ai-collab/bulk-image-organizer

Fork 0

[Bug]: Duplicates Workbench Loads Slowly with 2,000+ Duplicate Groups #94

New issue

Closed

opened 2026-06-15 00:16:02 +00:00 by ValleyGeek · 2 comments

ValleyGeek commented

2026-06-15 00:16:02 +00:00

Owner

App version

0.1.31

Platform

Both

Steps to reproduce

Load the duplicate workbench on a directory that has 2,000+ duplicate groups defined

Expected behavior

UI updates immediately and duplicate list is populated as it is loaded

Actual behavior

Program freezes for a few seconds then the UI refreshes

Additional context

No response

### App version 0.1.31 ### Platform Both ### Steps to reproduce 1. Load the duplicate workbench on a directory that has 2,000+ duplicate groups defined ### Expected behavior UI updates immediately and duplicate list is populated as it is loaded ### Actual behavior Program freezes for a few seconds then the UI refreshes ### Additional context _No response_

ValleyGeek added the

Priority/Medium

Kind/Bug

labels

2026-06-15 00:16:02 +00:00

Claude referenced this issue from a commit

2026-06-18 05:09:08 +00:00

fix: build Duplicates workbench tag list in batches (#94)

Claude commented

2026-06-18 05:09:53 +00:00

Member

Implemented on branch 0.1.33/issue-94-dupe-workbench-slow-load (commit c1f047d).

Root cause: DupeContainer._rebuild_tag_list() (the Duplicates workbench's dupe-tag sidebar) synchronously created one widget row per F-DUPE_/V-DUPE_ tag in a single main-thread pass. Profiling confirmed the DB query itself is fast (~20ms for 2,000 tags via get_dupe_tag_summaries) — the freeze is entirely from widget construction (~2-3s for 2,000 rows once layout/style passes are accounted for).

Fix: row widgets are now built in small batches (40 per tick) across event-loop ticks via a QTimer parented to the panel, guarded by a rebuild-generation token so a newer refresh cleanly supersedes an in-flight batch instead of corrupting it. visible_tag_names() now reflects the full filtered tag list immediately (used for auto-selecting/navigating groups), independent of how many row widgets have been built so far — so existing keyboard navigation and "select first group" behavior is unaffected even while rows are still populating.

With 2,000 synthetic groups, worst-case single-tick block dropped from a multi-second freeze to ~30-50ms (imperceptible), with the list visibly populating in the sidebar instead of the UI hanging.

Added regression tests (tests/test_pr11_dupe.py) covering batched population and stale-rebuild cancellation. Full suite (346 tests) and ruff check/format are clean; ran the full suite 5x to confirm no flakiness.

Not opening a PR yet per project workflow — will do so on request.

Implemented on branch `0.1.33/issue-94-dupe-workbench-slow-load` (commit c1f047d). Root cause: `DupeContainer._rebuild_tag_list()` (the Duplicates workbench's dupe-tag sidebar) synchronously created one widget row per `F-DUPE_`/`V-DUPE_` tag in a single main-thread pass. Profiling confirmed the DB query itself is fast (~20ms for 2,000 tags via `get_dupe_tag_summaries`) — the freeze is entirely from widget construction (~2-3s for 2,000 rows once layout/style passes are accounted for). Fix: row widgets are now built in small batches (40 per tick) across event-loop ticks via a `QTimer` parented to the panel, guarded by a rebuild-generation token so a newer refresh cleanly supersedes an in-flight batch instead of corrupting it. `visible_tag_names()` now reflects the full filtered tag list immediately (used for auto-selecting/navigating groups), independent of how many row widgets have been built so far — so existing keyboard navigation and "select first group" behavior is unaffected even while rows are still populating. With 2,000 synthetic groups, worst-case single-tick block dropped from a multi-second freeze to ~30-50ms (imperceptible), with the list visibly populating in the sidebar instead of the UI hanging. Added regression tests (`tests/test_pr11_dupe.py`) covering batched population and stale-rebuild cancellation. Full suite (346 tests) and `ruff check`/`format` are clean; ran the full suite 5x to confirm no flakiness. Not opening a PR yet per project workflow — will do so on request.

Claude referenced this issue from a commit

2026-06-18 06:07:58 +00:00

fix: keep Duplicates workbench row spacing constant while populating

Claude referenced this issue from a commit

2026-06-19 01:59:10 +00:00

fix: stop Duplicates workbench rows overlapping while batch populating

Claude referenced this issue from a commit

2026-06-19 03:32:45 +00:00

chore: bump version to 0.1.34 for PR prep (#94)

Claude referenced this issue

2026-06-19 03:33:21 +00:00

fix: keep Duplicates workbench responsive on large directories (#94) #100

Claude commented

2026-06-19 03:33:29 +00:00

Member

Status: partially resolved — fix is up for review in PR #100 (0.1.33/issue-94-dupe-workbench-slow-load → main).

What changed:

The Duplicates workbench tag list now builds in batches, yielding to the event loop between batches, so the UI stays responsive and the list visibly populates instead of freezing for several seconds on workspaces with thousands of duplicate groups.
Row spacing in the sidebar stays constant throughout population (uniform row height regardless of status indicator, plus forcing the scroll area's content widget to resize immediately after each batch so rows don't visually overlap/spread while building).

Net effect: the freeze described in this issue is significantly reduced, and the list now populates progressively as requested. Leaving this open (not closing) until the PR merges, in case further edge cases surface during review.

**Status: partially resolved** — fix is up for review in PR #100 (`0.1.33/issue-94-dupe-workbench-slow-load` → `main`). What changed: - The Duplicates workbench tag list now builds in batches, yielding to the event loop between batches, so the UI stays responsive and the list visibly populates instead of freezing for several seconds on workspaces with thousands of duplicate groups. - Row spacing in the sidebar stays constant throughout population (uniform row height regardless of status indicator, plus forcing the scroll area's content widget to resize immediately after each batch so rows don't visually overlap/spread while building). Net effect: the freeze described in this issue is **significantly reduced**, and the list now populates progressively as requested. Leaving this open (not closing) until the PR merges, in case further edge cases surface during review.

ValleyGeek referenced this issue from a commit

2026-06-19 03:40:39 +00:00

fix: keep Duplicates workbench responsive on large directories (#94) (#100)

ValleyGeek changed title from ~~[Bug]: Duplicates Workbends Loads Slowly with 2,000+ Duplicate Groups~~ to [Bug]: Duplicates Workbench Loads Slowly with 2,000+ Duplicate Groups

2026-06-19 18:21:33 +00:00

ValleyGeek closed this issue

2026-06-19 18:22:12 +00:00

Claude referenced this issue from a commit

2026-06-23 01:59:32 +00:00

perf(grid): O(1) path-to-row lookups in ImageListModel, batch dataChanged

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ai-collab/bulk-image-organizer#94

No description provided.

Rows
Columns