ai-collab/bulk-image-organizer

Fork 0

[Bug]: 'Close Directory and Delete Cache' Fails on Windows #27

New issue

Closed

opened 2026-06-09 05:14:06 +00:00 by ValleyGeek · 3 comments

ValleyGeek commented

2026-06-09 05:14:06 +00:00

Owner

App version

0.1.1

Platform

Windows

Steps to reproduce

Open a directory
Let scans finish
Press the Close Directory and Delete Cache button
Wait 10 seconds

Expected behavior

The working directory is closed and the cache database is deleted from the disk for that directory

Actual behavior

'Delete Cache' dialog appears with error stating the process cannot access the file because it is being used by another process [DB_FILE_PATH]

Additional context

No response

### App version 0.1.1 ### Platform Windows ### Steps to reproduce 1. Open a directory 2. Let scans finish 3. Press the `Close Directory and Delete Cache` button 4. Wait 10 seconds ### Expected behavior The working directory is closed and the cache database is deleted from the disk for that directory ### Actual behavior 'Delete Cache' dialog appears with error stating the process cannot access the file because it is being used by another process [DB_FILE_PATH] ### Additional context _No response_

ValleyGeek added the

Kind/Bug

Priority

Medium

labels

2026-06-09 05:14:06 +00:00

Claude referenced this issue from a commit

2026-06-20 06:25:28 +00:00

fix(db): release WAL/mmap before deleting workspace cache (fixes #27)

Claude commented

2026-06-20 06:25:55 +00:00

Member

Pushed a fix on branch 0.1.37/issue-27-34-close-directory (not yet merged):

The per-directory working DB used WAL mode with a 1GB mmap_size. On Windows, a memory-mapped WAL file can stay briefly locked after its connection closes — the same reasoning the share-sync pragmas already accounted for (_apply_share_file_pragmas sets mmap_size = 0). mmap_size is now 0 on Windows for the local cache too (it's a read-perf optimization only).
Added checkpoint_and_release(), called right before "Close Directory and Delete Cache" attempts to unlink the file: it opens one last short-lived connection, runs PRAGMA wal_checkpoint(TRUNCATE), and switches back to a rollback journal so no -wal/-shm companions are left to interfere with deletion.
The existing 10s retry-with-backoff loop remains as a final fallback.

Covered by a new unit test (test_checkpoint_and_release_removes_wal_companions). I wasn't able to verify on actual Windows hardware, so I'd appreciate confirmation once this lands in a build — leaving the issue open until then.

Pushed a fix on branch [`0.1.37/issue-27-34-close-directory`](https://repos.valleygeek.net/ai-collab/bulk-image-organizer/src/branch/0.1.37/issue-27-34-close-directory) (not yet merged): - The per-directory working DB used WAL mode with a 1GB `mmap_size`. On Windows, a memory-mapped WAL file can stay briefly locked after its connection closes — the same reasoning the share-sync pragmas already accounted for (`_apply_share_file_pragmas` sets `mmap_size = 0`). `mmap_size` is now `0` on Windows for the local cache too (it's a read-perf optimization only). - Added `checkpoint_and_release()`, called right before "Close Directory and Delete Cache" attempts to unlink the file: it opens one last short-lived connection, runs `PRAGMA wal_checkpoint(TRUNCATE)`, and switches back to a rollback journal so no `-wal`/`-shm` companions are left to interfere with deletion. - The existing 10s retry-with-backoff loop remains as a final fallback. Covered by a new unit test (`test_checkpoint_and_release_removes_wal_companions`). I wasn't able to verify on actual Windows hardware, so I'd appreciate confirmation once this lands in a build — leaving the issue open until then.

Claude referenced this issue from a commit

2026-06-20 07:19:46 +00:00

fix(db): explicitly close every short-lived connection (fixes #27)

Claude commented

2026-06-20 07:19:58 +00:00

Member

Update — found the actual root cause. The mmap/WAL-checkpoint mitigation in my previous comment was a reasonable defense-in-depth measure but not the real bug. Thanks for the handle64.exe dump (run right after the timeout error) — it showed ~25 simultaneously open handles to the same .bulk_image_organizer.db file (plus ~16 -wal and a couple -shm), still open well past the 10s retry window, plus handles to config.db/config.db-wal/config.db-shm too. That's a genuine handle leak, not a transient Windows lock.

Root cause: every WorkingDB and GlobalConfigDB read/write opened a fresh sqlite3.Connection via a bare with self._connect() as conn:. sqlite3.Connection's own context manager only commits/rolls back the transaction — it never calls .close(). The code relied on CPython reference-counting to release the connection once it went out of scope. I reproduced this locally: looping 200 short-lived connections opened that way leaked ~100 file descriptors even after an explicit gc.collect(), while the same loop with an explicit conn.close() stayed flat. So every scan, thumbnail read, pref lookup, etc. across a session was leaking a handle to the cache file — which is exactly why the file was still "in use" no matter how long the retry loop waited.

Fix (pushed to the same branch, 0.1.37/issue-27-34-close-directory): both DB classes already had a properly-closing transaction() context manager; every bare _connect() call site now routes through it instead. Added a regression test in each (test_repeated_reads_do_not_leak_connection_handles, test_repeated_pref_reads_do_not_leak_connection_handles) that fails against the old pattern (confirmed: 62 leaked fds) and passes against the fix.

This is a much more confident fix than the earlier mmap mitigation, though I still can't verify on real Windows hardware — would appreciate a retest once it's in a build.

**Update — found the actual root cause.** The mmap/WAL-checkpoint mitigation in my previous comment was a reasonable defense-in-depth measure but not the real bug. Thanks for the `handle64.exe` dump (run right after the timeout error) — it showed **~25 simultaneously open handles to the same `.bulk_image_organizer.db` file** (plus ~16 `-wal` and a couple `-shm`), still open well past the 10s retry window, plus handles to `config.db`/`config.db-wal`/`config.db-shm` too. That's a genuine handle leak, not a transient Windows lock. Root cause: every `WorkingDB` and `GlobalConfigDB` read/write opened a fresh `sqlite3.Connection` via a bare `with self._connect() as conn:`. `sqlite3.Connection`'s own context manager only commits/rolls back the transaction — it never calls `.close()`. The code relied on CPython reference-counting to release the connection once it went out of scope. I reproduced this locally: looping 200 short-lived connections opened that way leaked ~100 file descriptors *even after an explicit `gc.collect()`*, while the same loop with an explicit `conn.close()` stayed flat. So every scan, thumbnail read, pref lookup, etc. across a session was leaking a handle to the cache file — which is exactly why the file was still "in use" no matter how long the retry loop waited. Fix (pushed to the same branch, `0.1.37/issue-27-34-close-directory`): both DB classes already had a properly-closing `transaction()` context manager; every bare `_connect()` call site now routes through it instead. Added a regression test in each (`test_repeated_reads_do_not_leak_connection_handles`, `test_repeated_pref_reads_do_not_leak_connection_handles`) that fails against the old pattern (confirmed: 62 leaked fds) and passes against the fix. This is a much more confident fix than the earlier mmap mitigation, though I still can't verify on real Windows hardware — would appreciate a retest once it's in a build.

Claude commented

2026-06-20 07:38:38 +00:00

Member

Confirmed fixed and validated. Resolved on branch 0.1.37/issue-27-34-close-directory (PR to main incoming) by closing every SQLite connection explicitly instead of relying on garbage collection — see the previous comment for the root-cause writeup. Closing.

Confirmed fixed and validated. Resolved on branch `0.1.37/issue-27-34-close-directory` (PR to `main` incoming) by closing every SQLite connection explicitly instead of relying on garbage collection — see the previous comment for the root-cause writeup. Closing.

Claude closed this issue

2026-06-20 07:39:13 +00:00

Claude referenced this issue from a pull request that will close it,

2026-06-20 07:39:41 +00:00

fix: Close Directory and Delete Cache reliability (#27, #34) #112

ValleyGeek referenced this issue from a commit

2026-06-20 07:41:27 +00:00

fix: Close Directory and Delete Cache reliability (#27, #34) (#112)