OTLP pipelines.

What a pipeline is

A pipeline is an ordered list of transforms applied to OTLP records as they arrive at the ingest edge. Pipelines run before storage, which means anything you drop or rewrite at this layer never touches your bill, your retention floor, or your downstream queries.

1. 01

   /03
   Define the pipeline

   Pipelines live in pipelines/<name>.toml inside your workspace configuration repo. Each file is a single pipeline; ordering inside the file is the execution order at the edge.

   [ ]Config

   CopyCopied

   match  = "service.name = 'api' AND env = 'prod'"
   sample = { rate = 0.1, keep_errors = true }
   redact = ["http.request.body", "user.email"]

2. 02

   /03
   Apply it

   Push the file via the CLI. The control plane validates the syntax, estimates the budget impact, and rolls the pipeline out to the edge in under a minute.

   $Shell

   CopyCopied

   meridian pipeline apply pipelines/api-prod.toml

3. 03

   /03
   Verify

   The CLI returns the new pipeline ID and the estimated change in record volume. Spot-check the math against the live ingest dashboard before you forget about it.

   $Shell

   CopyCopied

   meridian pipeline status api-prod
   # api-prod  ·  active  ·  -88.2% volume  ·  edge.us-east, edge.eu-west

Transform reference

sample = { rate = 0.05, keep_errors = true }

Pipeline file layout

A typical workspace keeps one pipeline per service-environment pair, with shared rules pulled into a _lib/ folder.

When pipelines aren't enough

Cost tip. Most teams over-collect debug logs by 50–100×. A single match = "level = 'debug'" → sample { rate = 0.01 } rule typically pays for the workspace inside the first week.