Pipelines

Retention.

Hot, warm, and cold storage tiers — what each costs, what each query feels like, and when to move signals between them.

The three tiers

Every signal in Meridian lives in exactly one of three tiers at any moment. Tiers are mostly invisible at query time — the console fans out to all three — but they decide how fast a query returns and how much you pay for it.

0–30 days

Hot

Uncompressed columnar files in NVMe. Sub-second queries, full text indexing, every attribute searchable.

30 days – 13 months

Warm

ZSTD-compressed parquet on object storage. 2–8s queries. Indexed on a curated set of attributes you nominate per workspace.

13 months – 7 years

Cold

Glacier-class deep archive. Restore window is 4h. Used for compliance, not for debugging.

Default policy

Every new workspace starts with the following retention policy. You can change it any time, including for already-stored data.

Field	type	description
logs	stream
metrics	rollup
traces	stream
events	stream

Changing the policy

Retention is a workspace-level setting. To change it, edit workspace.toml and push:

Config

[retention.logs]
hot  = "30d"
warm = "13mo"
cold = "off"

[retention.traces]
hot  = "14d"
warm = "13mo"
cold = "off"

Shell

meridian workspace apply

The control plane re-indexes in the background. Hot → warm migrations finish in minutes; warm → cold takes overnight.

What a query against each tier feels like

Sub-second on most queries. Free-text search, regex, full attribute faceting all work without configuration.

Query

SELECT trace_id, duration_ms
FROM traces
WHERE service = 'checkout'
  AND duration_ms > 2000
  AND ts > now() - interval '6 hours'

Caution

Cold restores count toward your monthly egress allowance. Restoring a full month of trace data for a busy service can run into the tens of thousands of dollars on the underlying object store.

What never moves

A handful of records stay in hot for the entire workspace lifetime, regardless of policy:

Budget events — every burn, every reset, kept forever.
Pipeline changes — every apply, with the diff and the actor.
Workspace audit log — who logged in, who changed what.

This is non-negotiable; it's the part of Meridian that has to outlast any single incident.

Retention ready