ai-collab/bulk-image-organizer

Fork 0

[Bug]: Duplicate Workbench Slowness with Large Number of Sets #59

New issue

Closed

opened 2026-06-12 07:14:28 +00:00 by ValleyGeek · 3 comments

ValleyGeek commented

2026-06-12 07:14:28 +00:00

Owner

App version

0.1.22

Platform

Windows

Steps to reproduce

Open a directory with 3,000+ images that contains 400 duplicates sets and identify the duplicates
Perform any interaction within the duplicate workbench
- Open the duplicate workbench
- navigate between duplicate sets
- open the image viewer
- apply a tag
- apply a delete

Expected behavior

Near instant response when interacting with the application.

Actual behavior

4 second delay between interaction and application response

Additional context

I'm seeing a CPU core spike in use while the operation is delayed
The core issue seems to be around large tag counts slowing the application. Include review of, and optimization for, high tag counts.

### App version 0.1.22 ### Platform Windows ### Steps to reproduce 1. Open a directory with 3,000+ images that contains 400 duplicates sets and identify the duplicates 2. Perform any interaction within the duplicate workbench - Open the duplicate workbench - navigate between duplicate sets - open the image viewer - apply a tag - apply a delete ### Expected behavior Near instant response when interacting with the application. ### Actual behavior 4 second delay between interaction and application response ### Additional context I'm seeing a CPU core spike in use while the operation is delayed The core issue seems to be around large tag counts slowing the application. Include review of, and optimization for, high tag counts.

ValleyGeek added the

Kind/Bug

Priority

High

labels

2026-06-12 07:14:28 +00:00

Grok commented

2026-06-12 23:17:43 +00:00

Member

Implemented performance fix on branch 0.1.24/issue-59-dupe-workbench-perf.

Root cause: The Duplicates workbench rebuilt the entire tag list (400+ buttons) on every interaction, running 2+ DB queries per tag (get_image_dicts_for_dupe_tag, count_images_for_dupe_tag). Navigation between groups and tag apply/delete also triggered full rebuilds and, in some paths, reloaded tags for all 3000+ images.

Fix:

WorkingDB.get_dupe_tag_summaries() — batch member/delete counts and filter visibility in 1–2 queries
Incremental refresh_statuses_from_db() — update counts/indicators without destroying widgets
Skip dupe list rebuild when only the active group changes (refresh_dupe_list=False)
Remove redundant _reload_all_model_tags() after Keep Best
_enrich_dupe_group_images() uses one tag-color map instead of per-image DB lookups
count_distinct_dupe_tags() uses COUNT instead of fetching all names

Tests added for summaries and status derivation. Awaiting manual verification on a large duplicate set before closing.

Implemented performance fix on branch `0.1.24/issue-59-dupe-workbench-perf`. **Root cause:** The Duplicates workbench rebuilt the entire tag list (400+ buttons) on every interaction, running 2+ DB queries per tag (`get_image_dicts_for_dupe_tag`, `count_images_for_dupe_tag`). Navigation between groups and tag apply/delete also triggered full rebuilds and, in some paths, reloaded tags for all 3000+ images. **Fix:** - `WorkingDB.get_dupe_tag_summaries()` — batch member/delete counts and filter visibility in 1–2 queries - Incremental `refresh_statuses_from_db()` — update counts/indicators without destroying widgets - Skip dupe list rebuild when only the active group changes (`refresh_dupe_list=False`) - Remove redundant `_reload_all_model_tags()` after Keep Best - `_enrich_dupe_group_images()` uses one tag-color map instead of per-image DB lookups - `count_distinct_dupe_tags()` uses `COUNT` instead of fetching all names Tests added for summaries and status derivation. Awaiting manual verification on a large duplicate set before closing.

Grok commented

2026-06-12 23:44:59 +00:00

Member

Follow-up fix on 0.1.24/issue-59-dupe-workbench-perf for the remaining delay when leaving Single viewer back to Multi view.

Cause: _restore_from_viewer_sub() used the generic exit path, which invalidated the 3000-row filter proxy, reloaded the tag panel, and fully rebuilt the duplicate tag list — even though the Multi comparison grid was already populated.

Fix:

New _exit_viewer_to_dupes_multi() fast path: keeps the grid, runs refresh_statuses_from_db() + sync_image_tags() only
Removed redundant _on_dupe_multi_requested() call after viewer double-click return
Multi button skips full grid rebuild when tiles are already loaded

Test added: test_dupe_viewer_exit_skips_full_dupe_list_rebuild.

Follow-up fix on `0.1.24/issue-59-dupe-workbench-perf` for the remaining delay when leaving Single viewer back to Multi view. **Cause:** `_restore_from_viewer_sub()` used the generic exit path, which invalidated the 3000-row filter proxy, reloaded the tag panel, and fully rebuilt the duplicate tag list — even though the Multi comparison grid was already populated. **Fix:** - New `_exit_viewer_to_dupes_multi()` fast path: keeps the grid, runs `refresh_statuses_from_db()` + `sync_image_tags()` only - Removed redundant `_on_dupe_multi_requested()` call after viewer double-click return - Multi button skips full grid rebuild when tiles are already loaded Test added: `test_dupe_viewer_exit_skips_full_dupe_list_rebuild`.

Grok referenced this issue from a pull request that will close it,

2026-06-13 00:11:03 +00:00

Fix duplicate workbench slowness with large duplicate sets (fixes #59) #65

Grok closed this issue

2026-06-13 00:11:08 +00:00

Grok commented

2026-06-13 00:11:08 +00:00

Member

Verified fixed. Closed via PR #65 (0.1.24/issue-59-dupe-workbench-perf → main, release 0.1.25).

Delivered:

Batch get_dupe_tag_summaries() replaces per-tag DB round-trips in the duplicate sidebar
Incremental status updates; no full tag-list rebuild on group navigation or tag apply
Fast path when leaving Single viewer back to Multi comparison
Removed redundant full-model tag reload after Keep Best

Manual verification confirmed near-instant response across navigation, tagging, viewer, and Multi return with 400+ duplicate sets.

Verified fixed. Closed via PR #65 (`0.1.24/issue-59-dupe-workbench-perf` → `main`, release **0.1.25**). **Delivered:** - Batch `get_dupe_tag_summaries()` replaces per-tag DB round-trips in the duplicate sidebar - Incremental status updates; no full tag-list rebuild on group navigation or tag apply - Fast path when leaving Single viewer back to Multi comparison - Removed redundant full-model tag reload after Keep Best Manual verification confirmed near-instant response across navigation, tagging, viewer, and Multi return with 400+ duplicate sets.

ValleyGeek referenced this issue from a commit

2026-06-13 00:23:32 +00:00

Fix duplicate workbench slowness with large duplicate sets (fixes #59) (#65)

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ai-collab/bulk-image-organizer#59

No description provided.

Rows
Columns