[Bug]: Slow Dupe Group Change

ValleyGeek commented

2026-06-22 00:21:36 +00:00

Owner

App version

0.1.54 Beta

Platform

Windows

Steps to reproduce

Open a directory with multiple 100s of images and 100s of duplicates
Scan for visual duplicates
Open the duplicate workbench
Navigate between adjacent duplicate groups

Expected behavior

Nearly immediate loading of images from adjacent groups

Actual behavior

A nearly 1 second delay between changing groups then another short one for the images to render to the screen

Additional context

No response

### App version 0.1.54 Beta ### Platform Windows ### Steps to reproduce 1. Open a directory with multiple 100s of images and 100s of duplicates 2. Scan for visual duplicates 3. Open the duplicate workbench 4. Navigate between adjacent duplicate groups ### Expected behavior Nearly immediate loading of images from adjacent groups ### Actual behavior A nearly 1 second delay between changing groups then another short one for the images to render to the screen ### Additional context _No response_

ValleyGeek added the

Kind/Bug

Priority/Medium

labels

2026-06-22 00:21:36 +00:00

Claude referenced this issue from a commit

2026-06-23 00:34:53 +00:00

fix(dupes): avoid full re-sort+filter on duplicate group navigation (#118)

Claude added the

Reviewed/Confirmed

label

2026-06-23 00:35:09 +00:00

Claude commented

2026-06-23 00:35:24 +00:00

Member

Root cause confirmed: navigating between duplicate groups was invalidating the entire ImageFilterProxy twice per click — once in set_dupe_group_filter and again in _apply_dupe_group_filter's refresh call — each forcing a full filter+sort pass over every image in the directory, even though the dupe-group filter never affects sort order. With hundreds of images this produces the observed ~1s stall on every group change.

Fix pushed to 0.2.0/issue-118-dupe-group-nav-perf (commit c6e1aa0): the group filter now uses invalidateFilter() instead of a full invalidate(), and the redundant second invalidate in _apply_dupe_group_filter is removed. Lint, format, and full test suite (517 passed) are clean. Holding on the branch per workflow until a PR to main is requested.

Root cause confirmed: navigating between duplicate groups was invalidating the entire `ImageFilterProxy` twice per click — once in `set_dupe_group_filter` and again in `_apply_dupe_group_filter`'s refresh call — each forcing a full filter+sort pass over every image in the directory, even though the dupe-group filter never affects sort order. With hundreds of images this produces the observed ~1s stall on every group change. Fix pushed to `0.2.0/issue-118-dupe-group-nav-perf` (commit c6e1aa0): the group filter now uses `invalidateFilter()` instead of a full `invalidate()`, and the redundant second invalidate in `_apply_dupe_group_filter` is removed. Lint, format, and full test suite (517 passed) are clean. Holding on the branch per workflow until a PR to `main` is requested.

Claude referenced this issue from a commit

2026-06-23 01:12:17 +00:00

perf(dupes): pool comparison-grid cells and dedupe per-click DB calls (#118)

Claude commented

2026-06-23 01:12:26 +00:00

Member

Found and fixed a second, bigger contributor (commit 69a4b44): DupeMultiGrid destroyed and recreated every comparison tile on every group switch — each tile owns its own QGraphicsView/QGraphicsScene, so this was real widget-construction cost, not just layout math. Tiles are now pooled by grid position and reused in place (content/size updated, widgets kept alive), which measured ~8x faster per switch in an offscreen benchmark (15.7ms → 1.9ms, excluding image decode/IO).

Also removed three redundant per-click DB round trips that were piling onto the same click: get_all_tags() was being queried separately for the current group and each of the two preloaded neighbors (now computed once and shared), and count_distinct_dupe_tags() duplicated a count already available from the tag-summary query the status refresh had just run.

Combined with the proxy fix from the previous comment, this should bring group-switch latency much closer to single-image viewer navigation. Pushed to 0.2.0/issue-118-dupe-group-nav-perf; full test suite (517 passed) and lint clean.

Found and fixed a second, bigger contributor (commit 69a4b44): `DupeMultiGrid` destroyed and recreated every comparison tile on every group switch — each tile owns its own `QGraphicsView`/`QGraphicsScene`, so this was real widget-construction cost, not just layout math. Tiles are now pooled by grid position and reused in place (content/size updated, widgets kept alive), which measured ~8x faster per switch in an offscreen benchmark (15.7ms → 1.9ms, excluding image decode/IO). Also removed three redundant per-click DB round trips that were piling onto the same click: `get_all_tags()` was being queried separately for the current group and each of the two preloaded neighbors (now computed once and shared), and `count_distinct_dupe_tags()` duplicated a count already available from the tag-summary query the status refresh had just run. Combined with the proxy fix from the previous comment, this should bring group-switch latency much closer to single-image viewer navigation. Pushed to `0.2.0/issue-118-dupe-group-nav-perf`; full test suite (517 passed) and lint clean.

Claude referenced this issue from a commit

2026-06-23 01:29:45 +00:00

perf(filter): apply issue #118's invalidateFilter() fix to all filter setters

Claude commented

2026-06-23 01:59:51 +00:00

Member

Per the user's request, expanded this branch into a general redundant-call audit across the app (same bug classes as #118: over-broad Qt model invalidation, full widget rebuilds instead of reuse, N+1/duplicate DB round-trips). Five more fixes landed on 0.2.0/issue-118-dupe-group-nav-perf:

9644792 — generalized the invalidateFilter() fix to all 10 ImageFilterProxy setters (tag filter, search, hide-deleted, orientation, etc.), not just the dupe-group filter. None of these affect sort order, so each was forcing a needless full resort on every filter click/keystroke.
9ffbeef — TagPanel was doing a full deleteLater()+recreate of every tag swatch on every grid selection change and every viewer navigation step. Now updates existing buttons' highlight state in place.
75d3092 — bulk tag/DELETE actions (tag_manager.py) opened one SQLite connection per image for both planning and applying changes — a 200-image selection meant ~800-1000 connections for one keypress. Added bulk WorkingDB helpers and routed every per-path loop through them.
2356a88 — same per-path DB loop pattern in two main_window.py selection/tag-sync helpers, fixed the same way.
80501ba — ImageListModel (the grid's data model) did an O(n) linear scan to find a row by path on every upsert/thumbnail-batch/remove/relocate call, repeated throughout scan and thumbnail-generation progress. Added a path→row index for O(1) lookups, plus batched dataChanged emissions per call instead of one per item.

All changes are covered by new regression tests (including connection-count assertions for the DB batching) and the full suite (536 tests) passes. Branch is pushed but no PR opened yet, per the usual workflow.

Per the user's request, expanded this branch into a general redundant-call audit across the app (same bug classes as #118: over-broad Qt model invalidation, full widget rebuilds instead of reuse, N+1/duplicate DB round-trips). Five more fixes landed on `0.2.0/issue-118-dupe-group-nav-perf`: - `9644792` — generalized the `invalidateFilter()` fix to all 10 `ImageFilterProxy` setters (tag filter, search, hide-deleted, orientation, etc.), not just the dupe-group filter. None of these affect sort order, so each was forcing a needless full resort on every filter click/keystroke. - `9ffbeef` — `TagPanel` was doing a full `deleteLater()`+recreate of every tag swatch on *every* grid selection change and *every* viewer navigation step. Now updates existing buttons' highlight state in place. - `75d3092` — bulk tag/DELETE actions (`tag_manager.py`) opened one SQLite connection per image for both planning and applying changes — a 200-image selection meant ~800-1000 connections for one keypress. Added bulk `WorkingDB` helpers and routed every per-path loop through them. - `2356a88` — same per-path DB loop pattern in two `main_window.py` selection/tag-sync helpers, fixed the same way. - `80501ba` — `ImageListModel` (the grid's data model) did an O(n) linear scan to find a row by path on every upsert/thumbnail-batch/remove/relocate call, repeated throughout scan and thumbnail-generation progress. Added a path→row index for O(1) lookups, plus batched `dataChanged` emissions per call instead of one per item. All changes are covered by new regression tests (including connection-count assertions for the DB batching) and the full suite (536 tests) passes. Branch is pushed but no PR opened yet, per the usual workflow.

Claude referenced this issue from a commit

2026-06-23 05:32:46 +00:00

chore: PR-prep — bump to 0.2.1 for dupe-nav and redundant-call perf fixes

Claude referenced this issue from a pull request that will close it,

2026-06-23 05:33:16 +00:00

perf: fix slow duplicate-group navigation and related redundant calls (#118) #133

Claude commented

2026-06-23 05:33:33 +00:00

Member

Resolved by PR #133 (branch 0.2.0/issue-118-dupe-group-nav-perf, bumped to 0.2.1). Root causes and fixes summarized in the two comments above. Closing.

Resolved by PR #133 (branch `0.2.0/issue-118-dupe-group-nav-perf`, bumped to 0.2.1). Root causes and fixes summarized in the two comments above. Closing.

Claude closed this issue

2026-06-23 05:33:45 +00:00

ValleyGeek referenced this issue from a commit

2026-06-23 05:37:23 +00:00

perf: fix slow duplicate-group navigation and related redundant calls (#118) (#133)