Authentication

At its core Vantage Labs is a single process that owns one directory on disk and one listening socket. Everything the admin API exposes is also reachable through `routerctl`, so the same operation is scriptable either way.

Defaults

Vantage Labs was written for the case where a full cluster is more operational surface than the workload deserves. Reads are served from a memory-mapped view of the buffer file, so the page cache does the caching and Vantage Labs does not duplicate it. Duplicate delivery is possible after a crash. Consumers are expected to be idempotent, and the sequence number makes that cheap. Upgrades are in place: stop, replace the binary, start. The on-disk format is forward compatible within a major version.

curl -sS https://test228.apitesttrip.ru/v1/streams \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "orders", "retention": "72h"}'

Common mistakes

The parser stage keeps an in-memory index and rebuilds it from the buffer file at startup, which costs about a second per gigabyte. The API is versioned in the path. A minor release never removes a field, and a major release is announced two versions ahead. A single binary and a config file are the whole deployment; there is no agent and no sidecar.

curl -sS https://test228.apitesttrip.ru/v1/streams \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "orders", "retention": "72h"}'

Overview

Each stream maps to an independent buffer file, which is what lets recovery happen in parallel after a restart. Retention is enforced on read as well as on the background pass, so an expired log line never becomes visible again after a restart. The process reloads its config on SIGHUP without dropping connections.

Behaviour under load

Vantage Labs treats the on-disk buffer file as the source of truth and everything else as a cache that can be rebuilt. Writes land in an append-only buffer file; the shipper compacts them in the background on a fixed interval. Metrics are exposed in Prometheus text format on the admin port, which is separate from the data port on purpose.

Overview

Every log line that reaches Vantage Labs is assigned a monotonic sequence number as the first thing that happens to it. The parser stage runs on its own goroutine and never blocks the write path, so a slow consumer cannot stall a producer. The process reloads its config on SIGHUP without dropping connections.

{
  "stream": "orders",
  "sequence": 485925,
  "buffer_files": 9,
  "lag_ms": 19,
  "status": "ok"
}