Have you checked the documentation for submitting an Issue?
What type of enhancement is this?
What does the enhancement do?
Background
Several Admin list APIs in admin-server currently follow the same query pattern:
- Load a full in-memory/local dataset into a
Vec.
- Apply generic filtering and sorting.
- Apply pagination at the end.
This works for small datasets, but can become expensive when the cardinality grows to hundreds of thousands or millions.
Problem statement
- High CPU cost: full scan plus global sort (
O(n log n)).
- High memory pressure: large intermediate vectors and frequent string allocation.
- Latency spikes: expensive requests can block runtime capacity and increase p95/p99.
- Inconsistent cluster view for some endpoints: some list APIs are node-local and may return incomplete data in multi-broker deployments.
Candidate optimization directions (without changing core storage/data structures)
- Add request guardrails: max
limit, max page * limit, and stricter behavior for no-filter queries.
- Split query into two phases: lightweight scan first, then enrich only paginated rows.
- Reduce expensive sorting paths on large result sets (or restrict sortable fields).
- Add timeout/degraded behavior for very large queries.
- Add per-endpoint observability for query phases (scan/filter/sort/paginate/enrich/serialize) to identify bottlenecks.
- For node-local endpoints, add cluster fan-out aggregation (from Admin layer) where global view is required.
Interfaces that may have similar issues
/api/mqtt/client/list
/api/mqtt/session/list
/api/mqtt/subscribe/list
/api/mqtt/topic/list
/api/mqtt/system-alarm/list
/api/mqtt/ban-log/list
/api/mqtt/flapping_detect/list
/api/mqtt/connector/list (lower risk by data volume, same query pattern)
/api/mqtt/acl/list (lower risk by data volume, same query pattern)
/api/mqtt/blacklist/list (lower risk by data volume, same query pattern)
/api/mqtt/topic-rewrite/list (lower risk by data volume, same query pattern)
/api/mqtt/auto-subscribe/list (lower risk by data volume, same query pattern)
Suggested acceptance criteria
- Define and document query guardrails for all Admin list endpoints.
- Add per-endpoint performance metrics and logs for query phases.
- Prioritize optimization rollout for high-risk endpoints (
client_list, session_list).
- Clarify which endpoints are node-local vs cluster-aggregated in API behavior docs.
Are you willing to submit PR?
Have you checked the documentation for submitting an Issue?
What type of enhancement is this?
What does the enhancement do?
Background
Several Admin list APIs in
admin-servercurrently follow the same query pattern:Vec.This works for small datasets, but can become expensive when the cardinality grows to hundreds of thousands or millions.
Problem statement
O(n log n)).Candidate optimization directions (without changing core storage/data structures)
limit, maxpage * limit, and stricter behavior for no-filter queries.Interfaces that may have similar issues
/api/mqtt/client/list/api/mqtt/session/list/api/mqtt/subscribe/list/api/mqtt/topic/list/api/mqtt/system-alarm/list/api/mqtt/ban-log/list/api/mqtt/flapping_detect/list/api/mqtt/connector/list(lower risk by data volume, same query pattern)/api/mqtt/acl/list(lower risk by data volume, same query pattern)/api/mqtt/blacklist/list(lower risk by data volume, same query pattern)/api/mqtt/topic-rewrite/list(lower risk by data volume, same query pattern)/api/mqtt/auto-subscribe/list(lower risk by data volume, same query pattern)Suggested acceptance criteria
client_list,session_list).Are you willing to submit PR?