Quicklook cache taking up almost 80gb of storage?

Hey all,

My mac was running out of storage space so i decided to investigate and check out what could be taking up so much space. Thats when I came across my quicklook thumbnail cache folder which is taking up almost 80gb of space. More specifically, the file "index.sqlite-wal" is the largest file by far. I could not find any useful information online for this problem. All I found was clearing cache tutorials which would clear the index.sqlite file and not the one I am looking for. I am skeptical to delete the file just because its a system file and this is clearly a red flag zone. I am running the 10.15 Beta (19A471t). I will attach a screenshot of the folder below:


couldnt add an image so heres a link: https://gyazo.com/de7e0505f9588ca7f9eb7dadeaf0d471


Take a look at the highlighted file and the size on the right. Any thoughts?


Thanks!

David

I'm not as confident as you are about posting external links. You can Google "sqlite-wal file growing" for lots of useful information. I suggest you file a bug report with Apple and include that Google search query as a suggestion.

David:


Only assumptions on my part, I'd guess that file is allowed to go big specifically related to the beta, via the ACM, or, at least in your example, it's borked - sqlite.org says this about size, otherwise:


"Avoiding Excessively Large WAL Files

In normal cases, new content is appended to the WAL file until the WAL file accumulates about 1000 pages (and is thus about 4MB in size) at which point a checkpoint is automatically run and the WAL file is recycled. The checkpoint does not normally truncate the WAL file (unless the journal_size_limit pragma is set). Instead, it merely causes SQLite to start overwriting the WAL file from the beginning. This is done because it is normally faster to overwrite an existing file than to append. When the last connection to a database closes, that connection does one last checkpoint and then deletes the WAL and its associated shared-memory file, to clean up the disk.


So in the vast majority of cases, applications need not worry about the WAL file at all. SQLite will automatically take care of it. But it is possible to get SQLite into a state where the WAL file will grow without bound, causing excess disk space usage and slow queries speeds. The following bullets enumerate some of the ways that this can happen and how to avoid them.

  • Disabling the automatic checkpoint mechanism. In its default configuration, SQLite will checkpoint the WAL file at the conclusion of any transaction when the WAL file is more than 1000 pages long. However, compile-time and run-time options exist that can disable or defer this automatic checkpoint. If an application disables the automatic checkpoint, then there is nothing to prevent the WAL file from growing excessively.
  • Checkpoint starvation. A checkpoint is only able to run to completion, and reset the WAL file, if there are no other database connections using the WAL file. If another connection has a read transaction open, then the checkpoint cannot reset the WAL file because doing so might delete content out from under the reader. The checkpoint will do as much work as it can without upsetting the reader, but it cannot run to completion. The checkpoint will start up again where it left off after the next write transaction. This repeats until some checkpoint is able to complete.However, if a database has many concurrent overlapping readers and there is always at least one active reader, then no checkpoints will be able to complete and hence the WAL file will grow without bound.This scenario can be avoided by ensuring that there are "reader gaps": times when no processes are reading from the database and that checkpoints are attempted during those times. In applications with many concurrent readers, one might also consider running manual checkpoints with the SQLITE_CHECKPOINT_RESTART orSQLITE_CHECKPOINT_TRUNCATE option which will ensure that the checkpoint runs to completion before returning. The disadvantage of using SQLITE_CHECKPOINT_RESTARTand SQLITE_CHECKPOINT_TRUNCATE is that readers might block while the checkpoint is running.
  • Very large write transactions. A checkpoint can only complete when no other transactions are running, which means the WAL file cannot be reset in the middle of a write transaction. So a large change to a large database might result in a large WAL file. The WAL file will be checkpointed once the write transaction completes (assuming there are no other readers blocking it) but in the meantime, the file can grow very big.As of SQLite version 3.11.0 (2016-02-15), the WAL file for a single transaction should be proportional in size to the transaction itself. Pages that are changed by the transaction should only be written into the WAL file once. However, with older versions of SQLite, the same page might be written into the WAL file multiple times if the transaction grows larger than the page cache."


As JD advises, you want to file a bug against the beta and see what, if anything, comes back. Note that other than QB being a bit brittle, I don't see this as a 'known issue' in the release notes.


Ken

Quicklook cache taking up almost 80gb of storage?
 
 
Q