Add Sharpness/Blur Detection for Better Duplicate Selection

ValleyGeek commented

2026-06-08 05:06:55 +00:00

Owner

When the perceptual hashing step identifies near-duplicate images (same resolution but different files), we currently rely primarily on file size to decide which version to keep.

Describe the solution you'd like
Consider adding Sharpness / Blur Detection using the variance of the Laplacian (via OpenCV) as a secondary quality metric.

Higher variance = sharper image with more preserved detail (generally "better").
Works well across JPEG, PNG, and WebP.

Sample Implementation

import cv2
import os

def sharpness_score(image_path: str) -> float:
    """Calculate sharpness using Laplacian variance. Higher = better."""
    image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)
    if image is None:
        return 0.0
    laplacian_var = cv2.Laplacian(image, cv2.CV_64F).var()
    return laplacian_var


def select_better_image(path1: str, path2: str) -> str:
    """Select the better image between two perceptual duplicates."""
    # Primary heuristic: larger file size (less compression)
    size1 = os.path.getsize(path1)
    size2 = os.path.getsize(path2)
    
    if abs(size1 - size2) / max(size1, size2) > 0.05:  # >5% difference
        return path1 if size1 > size2 else path2
    
    # Tiebreaker: sharpness
    sharp1 = sharpness_score(path1)
    sharp2 = sharpness_score(path2)
    
    if abs(sharp1 - sharp2) > 50:  # Tunable threshold
        return path1 if sharp1 > sharp2 else path2
    
    # Fallback to size if sharpness is too close
    return path1 if size1 >= size2 else path2

Additional context

This method is fast (single-channel grayscale Laplacian).
Excellent for photos and most real-world images.
Should be applied only within duplicate groups found by perceptual hashing to avoid unnecessary computation.

Benefits

Better preservation of detail when choosing between similar images.
More intelligent decisions than file size alone.
Lightweight and easy to integrate into the existing duplicate processing pipeline.

When the perceptual hashing step identifies near-duplicate images (same resolution but different files), we currently rely primarily on file size to decide which version to keep. **Describe the solution you'd like** Consider adding Sharpness / Blur Detection using the variance of the Laplacian (via OpenCV) as a secondary quality metric. - Higher variance = sharper image with more preserved detail (generally "better"). - Works well across JPEG, PNG, and WebP. #### Sample Implementation ```python import cv2 import os def sharpness_score(image_path: str) -> float: """Calculate sharpness using Laplacian variance. Higher = better.""" image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE) if image is None: return 0.0 laplacian_var = cv2.Laplacian(image, cv2.CV_64F).var() return laplacian_var def select_better_image(path1: str, path2: str) -> str: """Select the better image between two perceptual duplicates.""" # Primary heuristic: larger file size (less compression) size1 = os.path.getsize(path1) size2 = os.path.getsize(path2) if abs(size1 - size2) / max(size1, size2) > 0.05: # >5% difference return path1 if size1 > size2 else path2 # Tiebreaker: sharpness sharp1 = sharpness_score(path1) sharp2 = sharpness_score(path2) if abs(sharp1 - sharp2) > 50: # Tunable threshold return path1 if sharp1 > sharp2 else path2 # Fallback to size if sharpness is too close return path1 if size1 >= size2 else path2 ``` **Additional context** - This method is fast (single-channel grayscale Laplacian). - Excellent for photos and most real-world images. - Should be applied only within duplicate groups found by perceptual hashing to avoid unnecessary computation. **Benefits** - Better preservation of detail when choosing between similar images. - More intelligent decisions than file size alone. - Lightweight and easy to integrate into the existing duplicate processing pipeline.

ValleyGeek changed title from ~~Enhancement: Add Sharpness/Blur Detection for Better Duplicate Selection~~ to Add Sharpness/Blur Detection for Better Duplicate Selection

2026-06-09 02:35:08 +00:00

ValleyGeek added the

Kind/Feature

Kind/Enhancement

labels

2026-06-09 02:35:26 +00:00

Claude referenced this issue from a commit

2026-06-20 16:28:33 +00:00

feat(dupe): persist reviewed groups; Pillow sharpness tiebreaker for Keep Best (#67, #21)

Claude referenced this issue

2026-06-20 16:28:41 +00:00

[Enhancement]: Reviewed Duplicate Groups Don't Persist Across Sessions #67

Claude referenced this issue

2026-06-20 16:28:52 +00:00

[Feature]: Duplicate Multi View Zoom / Pan #57

Claude commented

2026-06-20 16:29:05 +00:00

Member

Implemented on branch 0.1.45/issue-21-57-67-dupe-workbench-depth (chained off the previous dev branch, not merged to main yet) — with one substitution from the proposed implementation, recorded as DR-025 in docs/DECISIONS.md.

This project's AGENTS.md lists "No OpenCV dependency by default" as an explicit non-goal, so the cv2.Laplacian(...).var() approach in the issue couldn't be used as-is. Instead, core/dupe_detector.py gained sharpness_score(path), using Pillow's ImageFilter.FIND_EDGES (also a discrete Laplacian-style edge kernel) on a grayscale copy plus ImageStat.Stat pixel variance — the same underlying idea (edge-variance sharpness), staying within the existing Pillow dependency.

pick_keep_best_path() / keep_best_delete_paths() gained an optional root parameter: resolution (pixel count) is still the primary criterion exactly as before, but when multiple images in a group tie on resolution, sharpness now breaks that tie before falling back to date/path. Sharpness is only computed for the images actually tied on resolution, not the whole group, so the common case (one image with strictly more pixels) pays no extra cost. ui/main_window.py's Keep Best handler passes root=self._current_root so this is live.

Bundled with #67 and #57 in the same branch/PR.

Implemented on branch `0.1.45/issue-21-57-67-dupe-workbench-depth` (chained off the previous dev branch, not merged to `main` yet) — with one substitution from the proposed implementation, recorded as **DR-025** in `docs/DECISIONS.md`. This project's AGENTS.md lists "No OpenCV dependency by default" as an explicit non-goal, so the `cv2.Laplacian(...).var()` approach in the issue couldn't be used as-is. Instead, `core/dupe_detector.py` gained `sharpness_score(path)`, using Pillow's `ImageFilter.FIND_EDGES` (also a discrete Laplacian-style edge kernel) on a grayscale copy plus `ImageStat.Stat` pixel variance — the same underlying idea (edge-variance sharpness), staying within the existing Pillow dependency. `pick_keep_best_path()` / `keep_best_delete_paths()` gained an optional `root` parameter: resolution (pixel count) is still the primary criterion exactly as before, but when multiple images in a group **tie** on resolution, sharpness now breaks that tie before falling back to date/path. Sharpness is only computed for the images actually tied on resolution, not the whole group, so the common case (one image with strictly more pixels) pays no extra cost. `ui/main_window.py`'s Keep Best handler passes `root=self._current_root` so this is live. Bundled with #67 and #57 in the same branch/PR.

Claude closed this issue

2026-06-20 16:29:16 +00:00

ValleyGeek commented

2026-06-21 22:55:20 +00:00

Author

Owner

This needs a metric, rating, score, or indicator of some sort added to the meta data display below each images in the multi dupe view. Don't apply it only to the images with highest resolution, process the whole set.

ValleyGeek reopened this issue

2026-06-21 22:55:20 +00:00

Claude referenced this issue from a commit

2026-06-21 23:22:07 +00:00

feat(dupe): show sharpness score for every image in Multi view (#21)

Claude commented

2026-06-21 23:22:19 +00:00

Member

Added, pushed to 0.1.47/issue-111-86-64-89-feedback-fixes (0.1.56).

The sharpness score (same edge-variance metric from the original Keep Best tie-breaker) is now shown below every image in the Multi view's metadata row, next to resolution and file size — computed for the whole group, not just the images tied on highest resolution like Keep Best's internal use of it. The sharpest/blurriest image in the group gets the same green/red highlighting the other two metrics already use.

It's scored in the background per image (so it doesn't block opening a group), which means it can briefly show "Sharp —" right after a group loads before filling in — that's expected, not a bug.

One thing I want to flag honestly: I don't have a display in this environment, so I verified this through unit tests (real images, real background worker round-trip, not mocks) and an offscreen UI-exit smoke test, but I have not visually confirmed how it actually looks in a running window. Please give it a look and let me know if the layout/formatting needs any adjustment.

Added, pushed to `0.1.47/issue-111-86-64-89-feedback-fixes` (0.1.56). The sharpness score (same edge-variance metric from the original Keep Best tie-breaker) is now shown below every image in the Multi view's metadata row, next to resolution and file size — computed for the whole group, not just the images tied on highest resolution like Keep Best's internal use of it. The sharpest/blurriest image in the group gets the same green/red highlighting the other two metrics already use. It's scored in the background per image (so it doesn't block opening a group), which means it can briefly show "Sharp —" right after a group loads before filling in — that's expected, not a bug. One thing I want to flag honestly: I don't have a display in this environment, so I verified this through unit tests (real images, real background worker round-trip, not mocks) and an offscreen UI-exit smoke test, but I have not visually confirmed how it actually looks in a running window. Please give it a look and let me know if the layout/formatting needs any adjustment.

Claude referenced this issue

2026-06-22 00:35:27 +00:00

feat(dupe): persist reviewed groups; synchronized zoom/pan; sharpness tiebreaker (#21, #57, #67) #127

Claude referenced this issue from a commit

2026-06-22 00:38:18 +00:00

docs: PR-prep for the 0.1.x feature-tranche close-out - bump to 0.2.0

Claude referenced this issue

2026-06-22 00:38:38 +00:00

Feedback-fix rounds for #111, #86, #64, #89, #109, #57, #58, #21 — bump to 0.2.0 #129

Claude closed this issue

2026-06-22 00:39:44 +00:00

Claude commented

2026-06-22 00:41:37 +00:00

Member

Closing as part of wrapping up this branch chain into PRs. One implementation note for the record: this was built without OpenCV (per AGENTS.md's no-OpenCV-by-default policy) — sharpness_score() uses Pillow's ImageFilter.FIND_EDGES + pixel variance instead of cv2.Laplacian(...).var(), which is the same underlying idea (edge-detail variance) but without the extra dependency. See DR-025 for the substitution rationale and DR-032 for how the score got surfaced into the Multi view's UI for every image in a group, not just Keep Best's internal tiebreaker use of it.

Shipped across PR #127 (original sharpness tiebreaker for Keep Best) and PR #129 (the UI display for every image, addressing your "process the whole set" follow-up comment). Reopen or file a new issue if the display needs adjustment once you've had a chance to look at it in a running build.

Closing as part of wrapping up this branch chain into PRs. One implementation note for the record: this was built without OpenCV (per AGENTS.md's no-OpenCV-by-default policy) — `sharpness_score()` uses Pillow's `ImageFilter.FIND_EDGES` + pixel variance instead of `cv2.Laplacian(...).var()`, which is the same underlying idea (edge-detail variance) but without the extra dependency. See DR-025 for the substitution rationale and DR-032 for how the score got surfaced into the Multi view's UI for every image in a group, not just Keep Best's internal tiebreaker use of it. Shipped across PR #127 (original sharpness tiebreaker for Keep Best) and PR #129 (the UI display for every image, addressing your "process the whole set" follow-up comment). Reopen or file a new issue if the display needs adjustment once you've had a chance to look at it in a running build.

Claude referenced this issue from a commit

2026-06-22 01:51:47 +00:00

feat(dupe): persist reviewed groups; Pillow sharpness tiebreaker for Keep Best (#67, #21)

ValleyGeek referenced this issue from a commit

2026-06-22 01:52:44 +00:00

feat(dupe): persist reviewed groups; synchronized zoom/pan; sharpness tiebreaker (#21, #57, #67) (#127)

Claude referenced this issue from a commit

2026-06-22 02:35:05 +00:00

feat(dupe): show sharpness score for every image in Multi view (#21)

Claude referenced this issue from a commit

2026-06-22 02:35:05 +00:00

docs: PR-prep for the 0.1.x feature-tranche close-out - bump to 0.2.0