perf(redis_admin): drop clear-by-filter preview load; reuse chart numbers

[thomasrockhu-codecov]

Why

The lazy preview added in #907 still costs seconds because the
count requires a streaming LRANGE scan of the full queue —
the very work the preview was supposed to defer. On the
deepest queues (200k–500k+ messages) the operator-facing
confirmation page can take 5–15 seconds to first byte.

But the operator already saw the bucket count and queue depth
on the chart that linked them here. The chart's per-row "Clear
queue" button is the only way to land on this page in normal
use, and the chart computed both numbers to render its own
row. So the confirmation page can extrapolate an "approximately
N messages (~Y% of queue)" callout from those passed-through
hints — no further Redis call required, and no async polling /
skeleton placeholder either.

What

• Chart form (_frequency_chart.html): per-row "Clear
  queue" form now carries bucket_count, bucket_pct,
  total_visible, and total_depth as hidden inputs. Method
  is GET since these are non-secret display hints.
• clear_by_filter_view GET: rip out all lazy-preview
  wiring (no LLEN, no streaming_celery_count, no preview
  cache, no background job, no sample materialization). Read
  the four hint params via new _coerce_int / _coerce_float
  helpers (so a hand-edited URL never 500s) and render the
  confirmation page synchronously.
• clear_by_filter_view POST: all three actions
  (dry_run, clear_keep_one, clear_all) now uniformly
  spawn the chunked background-job worker and 302 to the
  progress page. The synchronous-dry_run-rerender shape is
  gone — the chunked worker walks the queue and computes the
  exact match count as it scans, regardless of whether the
  pass tombstones anything. Audit log lands at job completion
  under mode=chunked-dry-run for the dry-run case.
• clear_by_filter.html: skeleton placeholders, JS
  include, and CSS include all deleted. Replaced with a static
  filter fieldset (queue, task, repo with admin link, commit
  with full-SHA tooltip) and an "Approximate impact"
  fieldset showing either the extrapolated count or
  "Approximate count not available — the chunked clear will
  compute the exact match count as it scans." Typed-
  confirmation pattern + three submits unchanged.
• services.streaming_celery_count: deleted (only caller
  was the deleted preview surface; chunked worker uses
  _streaming_celery_clear directly with dry_run=True).
• README: clear-by-filter/ URLs row + "chart-driven
  targeted clear" section rewritten to document the chart-hint
  passthrough, the synchronous-no-Redis-I/O GET, and the
  uniform 302-to-progress shape across all three actions.
• Note on PR perf(redis_admin): lazy clear-by-filter preview (count off the request thread) #907: that PR's lazy-preview infrastructure
  (preview/skip-count routes, JS, CSS, cache helpers, the two
  REDIS_ADMIN_CLEAR_BY_FILTER_PREVIEW_* settings) was never
  merged to main, so there was nothing on the file system to
  delete beyond streaming_celery_count. If perf(redis_admin): lazy clear-by-filter preview (count off the request thread) #907 lands first,
  this PR will need a follow-up to drop those leftovers; until
  then the surface is already clean.

Approximate-count display

Rendered inline (no JS) when all four hints are present and
total_visible > 0:

Will tombstone approximately 78,500 messages (~37% of
queue at depth 211,052)

Falls back to:

Approximate count not available — the chunked clear will
compute the exact match count as it scans.

…when any hint is missing / non-numeric / divides by zero.

Tests

CeleryBrokerClearByFilterViewTest rewritten as a contract
suite over the new view shape:

• GET with chart hints renders the approximate-count callout
  with the right numbers and makes zero Redis calls
  (asserted via a MagicMock proxy on
  redis_admin_conn.get_connection whose method_calls must
  be empty after the request).
• GET without hints / with malformed hints / with zero
  total_visible falls back to "Approximate count not
  available", still no Redis calls.
• GET resolves the core.Repository admin link when repoid
  is set.
• POST clear_all / clear_keep_one / dry_run (with valid
  typed-confirm where required) each 302 to
  …/clear-by-filter/job/<uuid>/; start_celery_broker_clear_job
  is invoked once with the right dry_run / keep_one flags.
• Typed-confirm gate still rejects destructive POSTs with a
  200 + error message.
• Legacy action=confirm form shape still aliases through to
  the chunked-job path.
• Refusal of unscoped clears + superuser gating preserved.
• test_clear_by_filter_view_dry_run_preview_unchanged in
  test_celery_broker_clear_job.py renamed to
  …_dry_run_redirects_to_progress_page and updated to assert
  the new shape.

End-to-end exercise of the chunked worker (queue actually
drained, audit-log shape, keep_one survivor placement) lives
in test_celery_broker_clear_job.py (existing).

Migration note

This PR removes URL routes that #907 would have introduced
(clear-by-filter/preview/, clear-by-filter/skip-count/)
but those routes were never merged to main, so there's
nothing externally linkable to break. The clear-by-filter/
route itself is unchanged.

If #907 merges first, rebase this PR on top of it; the
leftover preview helpers / settings / static files / preview
test file that #907 introduces should then be deleted as part
of the rebase.

Test plan

• uv run ruff check redis_admin/
• uv run ruff format --check redis_admin/
• pytest -q apps/codecov-api/redis_admin/tests/ (294
  passed, run inside the api container which provides
  Postgres for the Django ORM tests)
• Manual smoke once deployed: open the celery_broker
  drill-down on a deep queue, click "Clear queue…" on a
  bucket, confirm the page renders sub-second with the
  approximate-count callout.

Made with Cursor

---

Note

Medium Risk
Changes the celery broker admin clear-by-filter flow and how destructive/dry-run actions are executed (now always asynchronous), which could affect operator workflows and queue-clearing behavior if the new job-spawn path misfires. Risk is mitigated by preserving typed-confirmation gates, legacy action=confirm compatibility, and expanded tests around zero-Redis GET and redirects.

Overview
Speeds up the celery broker clear-by-filter operator flow by making the confirmation page fully synchronous with zero Redis I/O, reusing count/depth hints passed from the frequency chart to show an approximate impact instead of doing a live scan.

Updates the chart’s per-bucket Clear queue… action to GET the confirmation URL with bucket_count/bucket_pct/total_visible/total_depth hints, and simplifies the confirmation template to a static filter summary (including a repo admin link) plus the approximate-impact callout.

Unifies POST handling so all three actions (dry_run, clear_keep_one, clear_all) now spawn the existing chunked background clear job and redirect to the progress page; removes the now-unused streaming_celery_count helper and updates docs/tests accordingly.

^{Reviewed by Cursor Bugbot for commit 25d9949. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 25d9949. Configure here.}

[cursor]

POST form drops chart hints on confirmation failure

Medium Severity

The POST form carries queue_name, task_name, repoid, and commitid as hidden inputs but omits the four chart-hint fields (bucket_count, bucket_pct, total_visible, total_depth). When a destructive POST fails typed confirmation, params is request.POST, so _coerce_int(params.get("bucket_count")) returns None for all four hints. The approx dict becomes None and the template falls back to "Approximate count not available" — even though the operator just saw the approximate count on the initial GET. The operator loses impact context at exactly the moment they need to re-confirm.

Additional Locations (1)

• apps/codecov-api/redis_admin/admin.py#L1992-L2000

 

^{Reviewed by Cursor Bugbot for commit 25d9949. Configure here.}

[sentry]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.90%. Comparing base (6fb832f) to head (25d9949).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #909   +/-   ##
=======================================
  Coverage   91.89%   91.90%           
=======================================
  Files        1316     1316           
  Lines       50586    50576   -10     
  Branches     1625     1625           
=======================================
- Hits        46485    46480    -5     
+ Misses       3795     3790    -5     
  Partials      306      306           

Flag	Coverage Δ
apiunit	94.92% <100.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[codecov-notifications]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

[thomasrockhu-codecov]

Folded into #905; keeping this open as the per-feature review record.