HTTP destination¶

OBSERVABILITY_TYPE=http
OBSERVABILITY_HTTP_URL=https://collector.example.com/axium/traces
OBSERVABILITY_HTTP_FORMAT=envelope          # or ndjson
OBSERVABILITY_HTTP_HEADERS='{"Authorization":"Bearer xxx"}'

The core POSTs each batch (up to 50 traces, at most 2 s apart) to the URL. Nothing is read back: the destination is write-only, and GET /v1/observability/traces answers 409 under this destination.

The request¶


Method	`POST`
Timeout	5 s per request (connection + response)
Retries	None. A failed batch is dropped and counted.
Success	Any `2xx`. Anything else is logged with the first 200 bytes of the body: `[observability] <url> answered 503 Service Unavailable <body>`
Headers set by Axium	`content-type: application/json` (envelope) or `application/x-ndjson` (ndjson) · `x-axium-trace-version: 1`
Your headers	`OBSERVABILITY_HTTP_HEADERS`, applied after Axium's, so they can override them

Errors are logged at most once every 5 minutes with a running count of lost traces. The URL is not probed at startup; a typo surfaces on the first flush.

Formats¶

envelope (default)ndjson

One JSON object per batch:

{
  "version": 1,
  "data": [
    { "trace_id": "…", "span_id": "…", "started_at": "…", "…": "…" },
    { "trace_id": "…", "span_id": "…", "started_at": "…", "…": "…" }
  ]
}

Good for webhooks, serverless functions and any receiver that wants a single document per request.

One JSON object per line, each line terminated by \n, no wrapper. The schema version travels only in the x-axium-trace-version header.

{"trace_id":"…","span_id":"…","started_at":"…", …}
{"trace_id":"…","span_id":"…","started_at":"…", …}

Directly ingestible by ClickHouse (FORMAT JSONEachRow), BigQuery load jobs, Vector, Fluent Bit, Loki, Splunk HEC raw endpoints and most log pipelines.

Every trace carries all 42 fields, always in the same order, with null where a value is unknown, [] for policies_applied and {} for tags and attributes when empty. The computed fields latency_ms, tokens_total and cost are not included; derive them at the receiver:

latency_ms   = ended_at - started_at           (milliseconds)
tokens_total = tokens_in + tokens_out          (null if either is null)
cost         = cost_in + cost_out              (null if either is null)

Field definitions: Formats & event schema.

Recipes¶

ClickHouse¶

CREATE TABLE axium.traces (
  trace_id UUID, span_id UUID,
  started_at DateTime64(3, 'UTC'), ended_at DateTime64(3, 'UTC'),
  source_module LowCardinality(String), proxy_app_id Nullable(UUID), api_key_id Nullable(UUID),
  client_ip Nullable(String), client_country Nullable(String), client_user_id Nullable(String),
  provider_id Nullable(UUID), provider_type LowCardinality(Nullable(String)), provider_region Nullable(String),
  model_id Nullable(UUID), model_used Nullable(String), model_requested Nullable(String),
  dialect LowCardinality(Nullable(String)), surface LowCardinality(Nullable(String)), upstream_path Nullable(String),
  tokens_in Nullable(Int32), tokens_out Nullable(Int32), tokens_cache_read Nullable(Int32),
  tokens_cache_write Nullable(Int32), tokens_reasoning Nullable(Int32), tokens_saved Nullable(Int32),
  cost_in Nullable(Decimal(14,6)), cost_out Nullable(Decimal(14,6)), cost_saved Nullable(Decimal(14,6)),
  stream Nullable(Bool), max_tokens Nullable(Int32),
  error_origin Nullable(String), status LowCardinality(String), http_status Nullable(Int16),
  error_category Nullable(String), error_message Nullable(String),
  cache_hit Nullable(Bool), fallback_used Nullable(Bool),
  policies_applied Array(String), denial_reason Nullable(String), meta_tag Nullable(String),
  tags Map(String, String), attributes String,
  latency_ms Int32 MATERIALIZED dateDiff('millisecond', started_at, ended_at)
) ENGINE = MergeTree ORDER BY (started_at, proxy_app_id)
SETTINGS input_format_json_read_objects_as_strings = 1;

OBSERVABILITY_TYPE=http
OBSERVABILITY_HTTP_FORMAT=ndjson
OBSERVABILITY_HTTP_URL='https://ch.example.com:8443/?query=INSERT%20INTO%20axium.traces%20FORMAT%20JSONEachRow'
OBSERVABILITY_HTTP_HEADERS='{"X-ClickHouse-User":"axium","X-ClickHouse-Key":"…"}'

Vector (to anywhere)¶

vector.toml

[sources.axium]
type = "http_server"
address = "0.0.0.0:8686"
decoding.codec = "json"            # one event per NDJSON line
path = "/axium"

[sinks.s3]
type = "aws_s3"
inputs = ["axium"]
bucket = "llm-traces"
key_prefix = "axium/date=%Y-%m-%d/"
encoding.codec = "json"
buffer.type = "disk"               # durable: survives a slow S3
buffer.max_size = 268435488

OBSERVABILITY_TYPE=http
OBSERVABILITY_HTTP_FORMAT=ndjson
OBSERVABILITY_HTTP_URL=http://vector:8686/axium

A local collector with a disk buffer is the recommended way to get at-least-once delivery, since Axium itself does not retry.

Datadog Logs¶

OBSERVABILITY_TYPE=http
OBSERVABILITY_HTTP_FORMAT=envelope
OBSERVABILITY_HTTP_URL=https://http-intake.logs.datadoghq.eu/api/v2/logs?ddsource=axium&service=axium
OBSERVABILITY_HTTP_HEADERS='{"DD-API-KEY":"…"}'

Datadog's intake accepts a JSON array, so send the envelope through a small transform (a Datadog pipeline or a Lambda) that unwraps data, or use the NDJSON format through the Datadog Agent's HTTP log listener.

A webhook of your own¶

// Node/Express receiver, envelope format
app.post('/axium', express.json({ limit: '2mb' }), (req, res) => {
  if (req.get('x-axium-trace-version') !== '1') return res.status(400).end();
  for (const trace of req.body.data) store(trace);
  res.sendStatus(204);               // any 2xx acknowledges the batch
});

Answer quickly: the core waits at most 5 s and does not retry. Accept, queue, process later.