Metrics Pipeline

The dashboard (Metrics Dashboard) is backed by a durable capture pipeline: in-process System.Diagnostics.Metrics instruments are observed, batched, published as a message, and written to a partitioned Postgres table with automatic retention.

Capture Rule

A single MeterListener observes the ZeeqTelemetry.Metrics meter. A measurement is persisted only if it carries an organization_id tag. Measurements without that tag still flow to OpenTelemetry exactly as before — they are simply not stored.

This one rule makes the pipeline self-selecting: there is no allow-list to maintain, and the pipeline's own internal diagnostics are never persisted by its own listener (they carry no organization_id). This is the feedback-loop guard — prefer plain logging over self-instrumenting the pipeline; if a pipeline meter is ever needed it must be a separate Meter instance, never ZeeqTelemetry.Metrics.

Instrument Taxonomy

metric_type is the instrument name, verbatim — there is no mapping layer. Add a KPI by declaring an instrument and tagging it; nothing else in the pipeline changes.

A metric event is a sparse wide event: a handful of promoted columns (indexed) plus a tags jsonb bag for everything else. A provider or call site that does not populate a field simply leaves it null (for example, a provider that reports no token usage never emits zeeq_review_tokens, so the row is absent rather than zero).

Agent telemetry emitters use the existing promoted tag names, not new aliases: organization_id is required for capture, user is promoted into the stored user_email column, and tool_name is promoted into the stored tool column. Harness-specific dimensions such as harness, model, token_kind, cost_source, mcp_server, success, and link_origin remain in tags until a dashboard query needs to promote or group by them.

Agent session metrics are emitted only after the normalized domain transaction succeeds. The session counter uses the domain store's newly-created conversation keys, so replaying telemetry for an existing conversation does not increment it again. Prompt, tool, token, and cost metrics skip housekeeping events such as title generation and progress messages. USD costs and provider-native raw billing units are intentionally separate histograms so dashboards never mix currencies and opaque provider units.

Flow

Instrument.Add/Record (tagged organization_id)
  -> MeterListener on ZeeqTelemetry.Metrics
  -> bounded channel (capacity 10k, DropWrite when full)
  -> batch: flush at 1,000 samples OR after 2s linger
  -> publish MetricBatchMessage (ISystemMessage, low-priority) via IZeeqMessagePublisher
  -> MetricBatchMessageHandler
  -> PostgresMetricEventStore (AddRange + one SaveChangesAsync)
  -> zeeq_metric_events

The channel is bounded and drops on overflow (DropWrite): metrics are best-effort and must never block or take down a request path. Publish failures are swallowed and logged for the same reason.

Storage and Retention

zeeq_metric_events is range-partitioned on created_at_utc with a composite primary key (id, created_at_utc). Partitions are 7 days wide, created ahead of time by pg_partman (premake 2), and dropped automatically after 30 days (retention_keep_table = false), driven by the existing hourly run_maintenance_proc cron job. Partitions are logged (not UNLOGGED) because Cloud SQL failover truncates unlogged tables, which would silently empty the dashboard.

Indexes are (organization_id, metric_type, …, created_at_utc) with partial IS NOT NULL filters on the promoted columns, so every dashboard query is partition-pruned and index-covered.

Read API

Seven read endpoints back the panels — series, percentiles, scatter, leaderboard, review volume, review findings, and overview — plus a filter-options endpoint that returns the distinct users, tools, authors, and repositories present in the data (repositories include removed mappings). Every route is a single-organization, window-scoped, AsNoTracking query cached for 30s via HybridCache. Organization scope comes from the cookie-validated route parameter, never a free query parameter, and window/metric-type inputs are validated against closed allow-sets.

Review volume/findings and the review histograms read the code-review tables directly. Review token counts come from an LLM usage sink that accumulates usage across all tool-calling round-trips of a single reviewer run (per-provider usage shape varies; providers that report nothing simply do not emit the token metric).

Adding a Metric

1. Declare an instrument (counter or histogram) on ZeeqTelemetry.Metrics.
2. Emit it with an organization_id tag (and any promoted columns / tags you want).
3. That's it — capture, batching, storage, and retention all apply automatically.

To surface it, add a read query + endpoint (or reuse the generic series/percentile endpoints) and a dashboard panel.

Deploy-Time Verification

Partitioning and retention are not exercised by the container-based integration tests (the test image ships pg_partman, but retention/cron behavior is time-based). Verify against a real environment via psql:

SELECT retention, retention_keep_table, infinite_time_partitions
FROM partman.part_config
WHERE parent_table = 'zeeq.zeeq_metric_events';

Confirm premade child partitions exist, and that cron.job_run_details shows a successful run_maintenance_proc tick.