Fix duplicate workbench slowness with large duplicate sets (fixes #59) #65

Merged

ValleyGeek merged 4 commits from 0.1.24/issue-59-dupe-workbench-perf into main

2026-06-13 00:23:30 +00:00

Grok commented

2026-06-13 00:11:03 +00:00

Member

Summary

Fixes #59 — eliminates multi-second delays in the Duplicates workbench when hundreds of duplicate groups are present.

Problem

With 400+ duplicate groups and 3000+ images, every interaction (navigating groups, applying tags, opening the viewer, returning to Multi view) triggered a full rebuild of the duplicate tag sidebar with 2+ database queries per tag (~800+ queries), plus unnecessary filter invalidation and full-model tag reloads.

Solution

WorkingDB.get_dupe_tag_summaries() — batch member/delete counts and filter visibility in 1–2 queries
Incremental sidebar updates — refresh_statuses_from_db() updates counts and indicators without destroying widgets
Skip rebuild on navigation — changing the active group no longer rebuilds the full tag list
Fast viewer → Multi exit — dedicated _exit_viewer_to_dupes_multi() keeps the comparison grid and syncs only tag metadata
Keep Best — removed redundant full-model tag reload
Image enrichment — single tag-color lookup instead of per-image DB queries

Verification

ruff check / ruff format --check
Targeted pytest (dupe summaries, tag filter, viewer exit fast path)
Manual verification on large duplicate set (reported by reporter)

Version

Bumps to 0.1.25 (patch).

## Summary Fixes #59 — eliminates multi-second delays in the Duplicates workbench when hundreds of duplicate groups are present. ## Problem With 400+ duplicate groups and 3000+ images, every interaction (navigating groups, applying tags, opening the viewer, returning to Multi view) triggered a full rebuild of the duplicate tag sidebar with 2+ database queries per tag (~800+ queries), plus unnecessary filter invalidation and full-model tag reloads. ## Solution - **`WorkingDB.get_dupe_tag_summaries()`** — batch member/delete counts and filter visibility in 1–2 queries - **Incremental sidebar updates** — `refresh_statuses_from_db()` updates counts and indicators without destroying widgets - **Skip rebuild on navigation** — changing the active group no longer rebuilds the full tag list - **Fast viewer → Multi exit** — dedicated `_exit_viewer_to_dupes_multi()` keeps the comparison grid and syncs only tag metadata - **Keep Best** — removed redundant full-model tag reload - **Image enrichment** — single tag-color lookup instead of per-image DB queries ## Verification - [x] `ruff check` / `ruff format --check` - [x] Targeted pytest (dupe summaries, tag filter, viewer exit fast path) - [x] Manual verification on large duplicate set (reported by reporter) ## Version Bumps to **0.1.25** (patch).

Grok added 2 commits

2026-06-13 00:11:03 +00:00

perf: batch dupe tag queries and skip redundant workbench rebuilds

Windows Release / prune-dev-release (push) Has been skipped

Details

Windows Release / build-release (push) Successful in 8m41s

Details

f18f169a14

Large duplicate sets (400+ groups) triggered hundreds of per-tag DB
round-trips on every navigation and tag change. Add get_dupe_tag_summaries()
for one-shot counts/filter checks, update status indicators incrementally,
and avoid rebuilding the full tag list when only the active group changes.

Also remove redundant full-model tag reload after Keep Best and replace
per-image enrich queries with a single tag-color lookup.

perf: fast path when leaving dupe viewer for multi view

Windows Release / prune-dev-release (push) Has been skipped

Details

Windows Release / build-release (push) Successful in 7m39s

Details

CI / test (pull_request) Has been cancelled

Details

CI / nuitka-smoke (pull_request) Has been cancelled

Details

CI / lint (pull_request) Has been cancelled

Details

b1cb140597

Returning from Single viewer to Multi comparison was still running
filter invalidation, tag panel reload, and a full 400+ tag list rebuild.
Add a dedicated exit path that keeps the comparison grid intact, syncs
only tag metadata, and updates dupe status indicators incrementally.

Grok added 1 commit

2026-06-13 00:11:05 +00:00

docs: PR-prep for issue 59 — dupe workbench performance (0.1.25)

CI / lint (pull_request) Successful in 26s

Details

Windows Release / prune-dev-release (push) Has been skipped

Details

CI / test (pull_request) Has been cancelled

Details

CI / nuitka-smoke (pull_request) Has been cancelled

Details

Windows Release / build-release (push) Successful in 8m12s