Add per-tenant TSDB cardinality status API endpoint#7332
Open
CharlieTLe wants to merge 3 commits intocortexproject:masterfrom
Open
Add per-tenant TSDB cardinality status API endpoint#7332CharlieTLe wants to merge 3 commits intocortexproject:masterfrom
CharlieTLe wants to merge 3 commits intocortexproject:masterfrom
Conversation
dosubot
bot
added
the
type/observability
Expose TSDB head cardinality statistics through a new /api/v1/status/tsdb endpoint, enabling users to identify which metrics, labels, and label-value pairs contribute the most series. This follows the existing UserStats fan-out pattern: ingester calls Head.Stats(), distributor aggregates across the replication set (dividing replicated counts by RF), and an HTTP handler serves JSON with an optional ?limit=N parameter. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>
Document the new TSDB cardinality status endpoint in the HTTP API reference, including query parameters, example request/response, and field descriptions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>
End-to-end test that starts a single-binary Cortex cluster, pushes series with varying cardinality, and validates the TSDB status API returns correct series counts, metric name breakdowns, and limit truncation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Charlie Le <charlie_le@apple.com>
CharlieTLe
force-pushed
the
worktree-sleepy-sniffing-squid
branch
from
1404aa1 to
2536082
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| limit := int32(10) | ||
| if v := r.FormValue("limit"); v != "" { | ||
| if n, err := strconv.Atoi(v); err == nil && n > 0 { | ||
| limit = int32(n) |
Check failure
Code scanning / CodeQL
Incorrect conversion between integer types High
yeya24
reviewed
Mar 6, 2026
Contributor
yeya24
left a comment
yeya24
left a comment
There was a problem hiding this comment.
Thanks for the PR!
I think this worth a design proposal as it is not a simple API and we need to think about how to extend it long term. Several questions I have:
- Is
/api/v1/status/tsdbthe right API to support? Or we should have a more specific API for cardinality analysis. Even though the endpoint is the same but I see you are using different response format compared to what Prometheus has https://prometheus.io/docs/prometheus/latest/querying/api/#tsdb-stats. - What fields should we support?
memoryInBytesByLabelNamelike this might make it hard to extend to long term storage - Is it the right thing to expose the API via distributor? Worst case this API could impact writes and this is what I want to avoid. Exposing via Querier seems safer and we can utilize even Query Frontend caching in the future
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/api/v1/status/tsdbendpoint that returns per-tenant TSDB cardinality statistics (series count by metric name, label value counts, memory usage by label, series count by label-value pair, min/max time)Test plan
TSDBStatusgRPC methodTSDBStatusaggregation logicTestTSDBStatus) that starts a single-binary Cortex cluster, pushes series with varying cardinality, and validates correct series counts, metric name breakdowns, label stats, and limit truncation🤖 Generated with Claude Code