The API documentation is available in English.

Timed labels for first-person video.

Trio-Lumen labels first-person (egocentric) video with timed activity and per-hand action segments. You submit a batch of videos, review the quote, and fetch one verified JSON result per video.

• Base URL: https://platform.machinefi.com/api/lumen/v1
• All requests and responses are JSON over HTTPS.
• All times are ISO 8601 UTC. All money amounts are decimal strings in USD.

Getting started

1. Sign up at platform.machinefi.com.
2. Add credits to your balance in the console (Billing). Lumen and Trio-Spark share one prepaid balance; there is no subscription. See Billing.
3. Create an API key with Lumen enabled. In the console (API Keys), check Trio-Lumen when you create the key (it is checked by default). The secret is shown once; store it on your server.
4. Submit a batch with POST /batches (see Workflow), using public https URLs as inputs (presigned URLs recommended) and the default hosted output, then poll GET /batches/{id} and fetch results with GET /items/{id}/result.

A complete, runnable example is in the Quickstart below. Steps 1–4 are fully self-serve. Two options need us for now; contact us to set them up:

• Your own bucket (read inputs from a prefix, or write results back): create credentials with the bucket permissions below and send the bucket details through a secure channel. We register it and give you its src_... id.
• Webhooks: we issue your signing secret through a secure channel. Until then, poll for status.

Neither is required to get started: any public https URL works as an input, and results are hosted for you.

Quickstart

Each sample below runs the whole flow end to end: create a batch with an Idempotency-Key, poll it until it finishes or waits for confirmation, confirm the quote if needed, list the items, and download each result's JSON from its short-lived link. Set your API key first:

export TRIO_API_KEY=tf_...
export VIDEO_URL="https://your-host.example/path/clip.mp4"
export MAX_AMOUNT_USD=...   # optional, Python and Node: your budget for this batch

VIDEO_URL is a public https URL to an MP4 or MOV file (see Supported inputs). Save the curl sample as a script and run it with bash.

Send a User-Agent header. If you call the API with a raw HTTP client, set a User-Agent header that names your app. Some default library user agents are blocked at the edge before the request reaches the API. The samples below set one.

curl

#!/usr/bin/env bash
# Requires bash, curl 7.76+ and jq. Run after exporting TRIO_API_KEY and VIDEO_URL.
set -euo pipefail
: "${TRIO_API_KEY:?export TRIO_API_KEY first}"
: "${VIDEO_URL:?export VIDEO_URL first (a public https URL to an MP4 or MOV file)}"
BASE=https://platform.machinefi.com/api/lumen/v1
api() { curl -sS --fail-with-body -H "Authorization: Bearer $TRIO_API_KEY" "$@"; }

# 1. Create a batch. Reuse the same Idempotency-Key if you retry this request.
BODY=$(jq -n --arg url "$VIDEO_URL" '{task: "egocentric-actions", inputs: [{url: $url}], output: {type: "hosted"}}')
BATCH=$(api -X POST "$BASE/batches" -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" -d "$BODY" | jq -r .id)
echo "batch $BATCH"

# 2. Poll until the batch finishes; 3. confirm the quote only after you approve it.
while true; do
  B=$(api "$BASE/batches/$BATCH")
  STATUS=$(jq -r .status <<<"$B")
  echo "status: $STATUS"
  case $STATUS in
    completed|completed_with_errors|failed|cancelled) break ;;
    awaiting_confirmation)
      jq -r '.quote | "quote: " + .amount_usd + " USD (" + .price_per_hour_usd + " USD per hour)"' <<<"$B"
      read -r -p "Confirm this quote? [y/N] " OK || OK=n
      if [ "$OK" = y ]; then
        api -X POST "$BASE/batches/$BATCH/confirm" >/dev/null || echo "confirm refused (for example quote_expired); polling again"
      else
        api -X POST "$BASE/batches/$BATCH/cancel" >/dev/null
        echo "Not confirmed; the batch was cancelled."; exit 1
      fi ;;
    blocked)
      REASON=$(jq -r .blocked_reason <<<"$B")
      case $REASON in
        insufficient_funds|over_limit) echo "blocked ($REASON): add credits in the console (Billing); the batch resumes on its own." ;;
        credentials_invalid) echo "blocked: your bucket credentials no longer work. Send us new credentials; the batch resumes."; exit 1 ;;
        *) echo "blocked ($REASON): contact support."; exit 1 ;;
      esac ;;
  esac
  sleep 15
done
jq -r '"failure_reason: " + (.failure_reason // "none")' <<<"$B"

# 4. List items; 5. download each succeeded item's result and check its sha256.
CURSOR=""
FOUND=0
while true; do
  PAGE=$(api "$BASE/batches/$BATCH/items?limit=100${CURSOR:+&cursor=$CURSOR}")
  jq -r '.data[] | [.id, .status, (.reason // "")] | @tsv' <<<"$PAGE"
  for ITEM in $(jq -r '.data[] | select(.status == "succeeded") | .id' <<<"$PAGE"); do
    RES=$(api "$BASE/items/$ITEM/result")
    curl -sS --fail-with-body -o "$ITEM.json" "$(jq -r .url <<<"$RES")"   # presigned link: no API key
    WANT=$(jq -r .sha256 <<<"$RES")
    GOT=$( (sha256sum "$ITEM.json" 2>/dev/null || shasum -a 256 "$ITEM.json") | cut -d' ' -f1)
    if [ "$GOT" != "$WANT" ]; then rm -f "$ITEM.json"; echo "checksum mismatch for $ITEM; fetch a fresh link and retry"; exit 1; fi
    echo "  saved $ITEM.json: $(jq '.activity_segments | length' "$ITEM.json") activity segments"
    FOUND=$((FOUND + 1))
  done
  CURSOR=$(jq -r '.next_cursor // empty' <<<"$PAGE")
  [ -n "$CURSOR" ] || break
done
[ "$FOUND" -gt 0 ] || { echo "No succeeded items; see the reasons above."; exit 1; }

Python (requests)

# pip install requests
import hashlib, json, os, sys, time, uuid
from decimal import Decimal
import requests

BASE = "https://platform.machinefi.com/api/lumen/v1"
VIDEO_URL = os.environ["VIDEO_URL"]               # a public https URL to an MP4 or MOV file
BUDGET = os.environ.get("MAX_AMOUNT_USD")         # optional: quotes within it are confirmed automatically
TERMINAL = {"completed", "completed_with_errors", "failed", "cancelled"}

s = requests.Session()
s.headers.update({
    "Authorization": "Bearer " + os.environ["TRIO_API_KEY"],
    "User-Agent": "my-app/1.0",
})

def check(r):
    if r.status_code >= 400:
        sys.exit(f"HTTP {r.status_code}: {r.text}")
    return r.json()

# 1. Create a batch. Reuse the same Idempotency-Key if you retry this request.
body = {"task": "egocentric-actions", "inputs": [{"url": VIDEO_URL}], "output": {"type": "hosted"}}
if BUDGET:
    body["max_amount_usd"] = BUDGET
batch = check(s.post(f"{BASE}/batches", json=body, timeout=30,
                     headers={"Idempotency-Key": str(uuid.uuid4())}))
batch_id = batch["id"]
print("batch", batch_id)

# 2. Poll until the batch finishes; 3. confirm the quote only after you approve it.
while True:
    batch = check(s.get(f"{BASE}/batches/{batch_id}", timeout=30))
    status = batch["status"]
    print("status:", status)
    if status in TERMINAL:
        break
    if status == "awaiting_confirmation":
        # With MAX_AMOUNT_USD set, a batch only waits here when its quote is over that budget.
        q = batch["quote"]
        over = BUDGET and Decimal(q["amount_usd"]) > Decimal(BUDGET)
        print(f"quote: {q['amount_usd']} USD ({q['price_per_hour_usd']} USD per hour)"
              + (f", over your budget of {BUDGET} USD" if over else ""))
        try:
            answer = input("Confirm this quote? [y/N] ")
        except EOFError:
            answer = ""
        if answer.strip().lower() != "y":
            check(s.post(f"{BASE}/batches/{batch_id}/cancel", timeout=30))
            sys.exit("Not confirmed; the batch was cancelled.")
        r = s.post(f"{BASE}/batches/{batch_id}/confirm", timeout=30)
        if r.status_code == 409:
            print("confirm:", r.json()["error"]["code"])  # e.g. quote_expired: a new quote follows
        else:
            check(r)
    elif status == "blocked":
        reason = batch["blocked_reason"]
        if reason in ("insufficient_funds", "over_limit"):
            print(f"blocked ({reason}): add credits in the console (Billing); the batch resumes on its own.")
        elif reason == "credentials_invalid":
            sys.exit("blocked: your bucket credentials no longer work. Send us new credentials; the batch resumes.")
        else:
            sys.exit(f"blocked ({reason}): contact support.")
    time.sleep(15)
print("failure_reason:", batch["failure_reason"])

# 4. List items; 5. download each succeeded item's result and check its sha256.
cursor, found = None, 0
while True:
    params = {"limit": 100, **({"cursor": cursor} if cursor else {})}
    page = check(s.get(f"{BASE}/batches/{batch_id}/items", params=params, timeout=30))
    for item in page["data"]:
        print(item["id"], item["status"], item["reason"] or "")
        if item["status"] != "succeeded":
            continue
        res = check(s.get(f"{BASE}/items/{item['id']}/result", timeout=30))
        dl = requests.get(res["url"], timeout=120)  # presigned link: no API key
        dl.raise_for_status()
        if hashlib.sha256(dl.content).hexdigest() != res["sha256"]:
            sys.exit("checksum mismatch; fetch a fresh link and retry")
        with open(f"{item['id']}.json", "wb") as f:
            f.write(dl.content)
        doc = json.loads(dl.content)
        print("  saved", f"{item['id']}.json:", len(doc["activity_segments"]), "activity segments")
        found += 1
    cursor = page["next_cursor"]
    if not cursor:
        break
if not found:
    sys.exit("No succeeded items; see the reasons above.")

Node.js 18+ (built-in fetch)

// Node 18+ (built-in fetch). Save as lumen-quickstart.mjs and run: node lumen-quickstart.mjs
import { createHash, randomUUID } from "node:crypto";
import { writeFile } from "node:fs/promises";
import { createInterface } from "node:readline/promises";

const BASE = "https://platform.machinefi.com/api/lumen/v1";
const VIDEO_URL = process.env.VIDEO_URL; // a public https URL to an MP4 or MOV file
const BUDGET = process.env.MAX_AMOUNT_USD; // optional: quotes within it are confirmed automatically
const TERMINAL = new Set(["completed", "completed_with_errors", "failed", "cancelled"]);
const HEADERS = { Authorization: "Bearer " + process.env.TRIO_API_KEY, "User-Agent": "my-app/1.0" };
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const fail = (message) => { console.error(message); process.exit(1); };
if (!process.env.TRIO_API_KEY || !VIDEO_URL) fail("export TRIO_API_KEY and VIDEO_URL first");

async function api(method, path, { body, headers = {}, allow = [] } = {}) {
  const res = await fetch(BASE + path, {
    method,
    headers: { ...HEADERS, ...(body ? { "Content-Type": "application/json" } : {}), ...headers },
    body: body ? JSON.stringify(body) : undefined,
  });
  const text = await res.text();
  if (!res.ok && !allow.includes(res.status)) fail("HTTP " + res.status + ": " + text);
  return { status: res.status, data: JSON.parse(text) };
}

// Resolves to the typed answer, or to "" (declined) if stdin closes first.
function ask(question) {
  const rl = createInterface({ input: process.stdin, output: process.stdout });
  return new Promise((resolve) => {
    let answered = false;
    rl.on("close", () => { if (!answered) resolve(""); });
    rl.question(question).then(
      (answer) => { answered = true; resolve(answer); rl.close(); },
      () => resolve(""),
    );
  });
}

// 1. Create a batch. Reuse the same Idempotency-Key if you retry this request.
const body = { task: "egocentric-actions", inputs: [{ url: VIDEO_URL }], output: { type: "hosted" } };
if (BUDGET) body.max_amount_usd = BUDGET;
const created = await api("POST", "/batches", { headers: { "Idempotency-Key": randomUUID() }, body });
const batchId = created.data.id;
console.log("batch", batchId);

// 2. Poll until the batch finishes; 3. confirm the quote only after you approve it.
let batch;
while (true) {
  batch = (await api("GET", "/batches/" + batchId)).data;
  console.log("status:", batch.status);
  if (TERMINAL.has(batch.status)) break;
  if (batch.status === "awaiting_confirmation") {
    // With MAX_AMOUNT_USD set, a batch only waits here when its quote is over that budget.
    const q = batch.quote;
    const over = BUDGET && Number(q.amount_usd) > Number(BUDGET);
    console.log("quote: " + q.amount_usd + " USD (" + q.price_per_hour_usd + " USD per hour)"
      + (over ? ", over your budget of " + BUDGET + " USD" : ""));
    const answer = await ask("Confirm this quote? [y/N] ");
    if (answer.trim().toLowerCase() !== "y") {
      await api("POST", "/batches/" + batchId + "/cancel");
      fail("Not confirmed; the batch was cancelled.");
    }
    const r = await api("POST", "/batches/" + batchId + "/confirm", { allow: [409] });
    if (r.status === 409) console.log("confirm:", r.data.error.code); // e.g. quote_expired: a new quote follows
  } else if (batch.status === "blocked") {
    const reason = batch.blocked_reason;
    if (reason === "insufficient_funds" || reason === "over_limit") {
      console.log("blocked (" + reason + "): add credits in the console (Billing); the batch resumes on its own.");
    } else if (reason === "credentials_invalid") {
      fail("blocked: your bucket credentials no longer work. Send us new credentials; the batch resumes.");
    } else {
      fail("blocked (" + reason + "): contact support.");
    }
  }
  await sleep(15000);
}
console.log("failure_reason:", batch.failure_reason);

// 4. List items; 5. download each succeeded item's result and check its sha256.
let cursor = null;
let found = 0;
do {
  const query = "?limit=100" + (cursor ? "&cursor=" + encodeURIComponent(cursor) : "");
  const page = (await api("GET", "/batches/" + batchId + "/items" + query)).data;
  for (const item of page.data) {
    console.log(item.id, item.status, item.reason ?? "");
    if (item.status !== "succeeded") continue;
    const res = (await api("GET", "/items/" + item.id + "/result")).data;
    const download = await fetch(res.url); // presigned link: no API key
    if (!download.ok) fail("download failed: HTTP " + download.status);
    const bytes = Buffer.from(await download.arrayBuffer());
    if (createHash("sha256").update(bytes).digest("hex") !== res.sha256) fail("checksum mismatch");
    await writeFile(item.id + ".json", bytes);
    const doc = JSON.parse(bytes.toString("utf8"));
    console.log("  saved", item.id + ".json:", doc.activity_segments.length, "activity segments");
    found++;
  }
  cursor = page.next_cursor;
} while (cursor);
if (!found) fail("No succeeded items; see the reasons above.");

The samples never confirm a quote on their own. When a batch waits in awaiting_confirmation, they show the quote and ask you to approve it; anything but y cancels the batch, and nothing is charged. With MAX_AMOUNT_USD set, the Python and Node samples send it as max_amount_usd: a quote within that budget is confirmed by us automatically, and a quote over it still waits for your approval (a manual confirm does not check max_amount_usd). With webhooks you can drop the polling loop.

Supported inputs

Every input is inspected during preflight: we read its container, streams and timestamps, without decoding the whole video. A file that fails these checks becomes a skipped item with the reason shown. A problem found only later, while processing (for example frames that cannot be decoded), ends the item as failed, with a reason such as decode_error. Neither skipped nor failed items are charged.

URL inputs. A bucket is not required: any public https URL that serves the video file works.

• We download the file once, from our servers, during preflight. The URL must be fetchable with a plain GET, without browser cookies, a login or a browser session, and must answer 200 with the file itself. Up to 5 redirects are followed.
• Link to the file, not to a web page that plays it. Hosts that require a login, or that reject non-browser clients, fail with fetch_failed.
• Presigned S3, GCS or R2 URLs are recommended. Make them valid for at least 24 hours, since preflight can retry.
• The URL must use https and must not contain a user name or password. Other URLs are refused with 422 invalid_request when you create the batch.

Bucket inputs. Under a registered source's prefix, every non-empty object whose key ends in .mp4 or .mov (any case) becomes an item. Other objects are ignored.

Processing time

• Processing is asynchronous. Creating a batch returns at once; preflight, the quote and processing follow in the background.
• Short clips typically finish within minutes of confirmation. Longer videos and larger batches take longer.
• The first job after a quiet period can take longer to start while processing capacity starts up.
• Use webhooks rather than tight polling. If you poll, every 15–30 seconds is plenty, and stays well inside the request rate limit.

Authentication

Send your API key as a bearer token:

Authorization: Bearer tf_...

Create the key in the platform console with Trio-Lumen checked. A key can be enabled for Trio-Spark, Trio-Lumen, or both; this is fixed when the key is created. A key that is not enabled for Lumen gets 403 missing_scope: create a new key with Lumen checked (you can then revoke the old one). Keys created through the console API without a products list are enabled for Trio-Spark only. Keep keys server-side and never embed them in clients.

Workflow

1. Create a batch

Send your video inputs. The batch starts in preflight: we fetch and inspect every video and compute a quote.

2. Quote

• If the quote is within max_amount_usd, the batch is confirmed automatically and starts running.
• A zero quote is confirmed automatically, even without max_amount_usd.
• If every input is skipped, there is nothing to quote: the batch fails with no_processable_items and nothing is charged.
• Otherwise it waits in awaiting_confirmation until you call POST /batches/{id}/confirm.
• 144 hours after creation, a batch still in awaiting_confirmation, or blocked before any funds were reserved for it, is cancelled automatically. Nothing is charged.
• Quotes are valid for 24 hours and are re-quoted when they expire.
• The quote is an upper bound: you are never charged more than the confirmed quote.

3. Processing

Each video becomes an item and is processed independently. Items that finish with a verified result are succeeded and are charged on the video time covered, rounded up to whole minutes, with a minimum per item set by the quote (currently 2 minutes). See Billing.

4. Results

• Fetch each result with GET /items/{id}/result, or have them written to your bucket (output.type = "bucket").
• Hosted results are kept for 30 days.
• Subscribe to webhooks instead of polling.

Only succeeded items are charged. Skipped, failed, cancelled and needs_review items are not charged. An item in needs_review is checked by us. If we approve it, it becomes succeeded and is charged then, unless it qualifies for reuse (see Items in needs_review).

Endpoints

Create a batch

POST https://platform.machinefi.com/api/lumen/v1/batches

{
  "task": "egocentric-actions",
  "inputs": [
    {
      "url": "https://storage.example.com/raw/clip-001.mp4?X-Amz-Signature=..."
    },
    {
      "source": "src_3f2a...",
      "prefix": "raw/2026-11/"
    }
  ],
  "output": {
    "type": "hosted"
  },
  "webhook_url": "https://hooks.example.com/lumen",
  "metadata": {
    "project": "warehouse-q4"
  }
}

Response: 202 Accepted with the batch object.

Bucket output. Every batch writes into its own folder, <prefix><batch_id>/, so batches that share an output prefix never touch each other's files:

• Each succeeded item is written to <prefix><batch_id>/<item_id>.json.
• When the batch ends as completed, completed_with_errors or failed, <prefix><batch_id>/_COMPLETE.json is written. It carries batch_id and is written only after every succeeded item's file is in place, so its presence means the set is complete.
• It is not written for a cancelled batch, or for a batch that fails during preflight (for example too_many_items), since no item file was written.
• It is rewritten if an approval changes the outcome. Its generation field increases with every new outcome; the file with the highest generation is current. Writes use conditional requests (If-None-Match / If-Match) so an older outcome never replaces a newer one. On storage that does not support conditional writes, a late older write is detected and corrected within about 10 minutes.
• _COMPLETE.json is a convenience marker. The API (GET /batches/{id}) and webhooks are the source of truth for a batch's status and items.

Idempotency.

• Repeating a request with the same Idempotency-Key returns the original batch with the header Idempotent-Replayed: true. No second batch is created.
• The request comparison ignores metadata. A retry that differs only in metadata replays the original batch, and the new metadata is not applied.
• Reusing a key with any other field changed returns 409 idempotency_conflict.

Batch object

{
  "object": "batch",
  "id": "8d0c7c1e-4b0f-4f1a-9a57-0f3f6a0f2b11",
  "status": "running",
  "blocked_reason": null,
  "failure_reason": null,
  "cancel_requested": false,
  "task": "egocentric-actions",
  "processing_version": "egocentric-actions@<version>",
  "created_at": "2026-11-01T18:02:11.204Z",
  "confirmed_at": "2026-11-01T18:04:40.017Z",
  "finished_at": null,
  "preflight": {
    "pending_inputs": 0
  },
  "quote": {
    "amount_usd": "...",
    "price_per_hour_usd": "...",
    "min_billable_minutes": 2,
    "quoted_at": "2026-11-01T18:04:39.950Z",
    "expires_at": "2026-11-02T18:04:39.950Z"
  },
  "max_amount_usd": "...",
  "amounts": {
    "reserved_usd": "...",
    "settled_usd": "..."
  },
  "counts": {
    "queued": 40,
    "running": 4,
    "publishing": 0,
    "succeeded": 12,
    "needs_review": 0,
    "failed": 0,
    "skipped": 1,
    "cancelled": 0
  },
  "output": {
    "type": "hosted"
  },
  "webhook_url": "https://hooks.example.com/lumen",
  "metadata": {
    "project": "warehouse-q4"
  }
}

• processing_version above is a placeholder. The actual value is the task version pinned on your batch; see the available versions in GET /schema.
• quote is null until preflight finishes.
• amounts.reserved_usd is the part of the quote still held on your balance.
• amounts.settled_usd is what has been charged so far.

List batches

GET /batches?limit=20&cursor=...

• limit is 1–100 (default 20).
• Returns { "object": "list", "data": [batch...], "next_cursor": "..." | null }.
• Pass next_cursor back as cursor to get the next page.

Confirm or resume

POST /batches/{id}/confirm, no body. Returns the batch object.

• In awaiting_confirmation: reserves the quoted amount and starts processing. Returns 409 quote_expired if the quote expired; a new quote follows automatically.
• In blocked for insufficient_funds or over_limit: retries the reservation after you add funds.
• During preflight: returns 409 not_ready.
• Already running or finished: returns 200 with the batch unchanged, so a repeated confirm is safe.
• After a cancel: returns 409 invalid_state.

Cancel

POST /batches/{id}/cancel, no body. Returns the batch object.

• Queued items are cancelled and not charged.
• Items already running finish and are charged normally.
• The batch becomes cancelled once every item has stopped.
• Cancelling again, or cancelling a batch that already finished, returns 200 with the batch unchanged.

If your Lumen access ends, you can no longer create or confirm batches, but you can still list, read and cancel your existing batches and download their results. Batches still open 24 hours after access ended are cancelled as above: queued items are not charged, and items already running finish and are charged normally.

List items

GET /batches/{id}/items?status=succeeded&limit=100&cursor=... · limit is 1–500 (default 100); status filters by item status.

{
  "object": "list",
  "data": [
    {
      "object": "item",
      "id": "5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
      "batch_id": "8d0c7c1e-4b0f-4f1a-9a57-0f3f6a0f2b11",
      "position": 0,
      "input": {
        "type": "url",
        "index": 0,
        "host": "storage.example.com"
      },
      "status": "succeeded",
      "reason": null,
      "duration_s": 600,
      "container_duration_s": 600.5,
      "covered_s": 600,
      "billing": "charged",
      "billable_minutes": 10,
      "charged_usd": "...",
      "result_available": true,
      "result_expires_at": "2026-12-01T18:40:02.000Z",
      "created_at": "2026-11-01T18:02:11.204Z",
      "finished_at": "2026-11-01T18:40:02.000Z"
    }
  ],
  "next_cursor": null
}

Get a result

GET /items/{id}/result

{
  "object": "result",
  "item_id": "5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
  "schema": "lumen.egocentric-actions.v1",
  "sha256": "9b1f...",
  "bytes": 48213,
  "url": "https://...",
  "url_expires_at": "2026-11-01T18:55:02.000Z",
  "retained_until": "2026-12-01T18:40:02.000Z"
}

• url is a download link valid for 15 minutes. Call this endpoint again for a fresh link.
• Check the downloaded bytes against sha256.
• 409 result_unavailable: the item has no result. 409 result_expired: the item is past its 30-day retention.

Schema

GET /schema returns id (lumen.egocentric-actions.v1), tasks (the available versions, with latest marked) and json_schema (the full result JSON Schema).

Bucket permissions

Your own storage (optional, set up with us). To read inputs from, or write results back to, your own storage, you provide an S3-compatible bucket you control (AWS S3, Cloudflare R2, or another S3-compatible store) and credentials with read access to the input prefix and read and write access to the output prefix. Credentials without the permissions below are refused when the source is registered. Without a bucket, use public https URL inputs (presigned URLs recommended) and hosted results.

The credentials you give us need:

When the bucket is registered, and whenever its credentials are replaced, we check them with a small test object at <write prefix>.lumen-selftest/<random>.json: read of a missing key, write, read back, and two conditional writes that must be refused. Credentials missing a required permission are not accepted. The test object is deleted afterwards; without s3:DeleteObject it stays (a few bytes) and can be removed at any time.

AWS IAM example (inputs under raw/, outputs under lumen-out/):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::<bucket>",
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "raw/*",
            "lumen-out/*"
          ]
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": [
        "arn:aws:s3:::<bucket>/raw/*",
        "arn:aws:s3:::<bucket>/lumen-out/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<bucket>/lumen-out/*"
    },
    {
      "Effect": "Allow",
      "Action": "s3:DeleteObject",
      "Resource": "arn:aws:s3:::<bucket>/lumen-out/.lumen-selftest/*"
    }
  ]
}

Cloudflare R2: an R2 API token with Object Read & Write on the bucket covers all of the above (R2 tokens are scoped per bucket, not per prefix).

Statuses and reasons

Batch status

A finished batch can change from completed_with_errors or failed to completed (or completed_with_errors) when we approve an item that was in needs_review. A batch.completed webhook announces the change.

blocked_reason

failure_reason

no_processable_items, all_items_failed, too_many_items, batch_too_large, listing_too_large, source_unavailable, credentials_invalid, preflight_timeout (inputs could not be inspected within 24 hours), processing_error.

Item status

queued, running, publishing (result being verified), succeeded, needs_review, failed, skipped, cancelled.

Items in needs_review

needs_review means the item finished processing but its result was held back instead of delivered. Its reason says why:

• qa_failed: the result did not pass our automated quality checks.
• exceeds_quote: the result would have cost more than this item's share of the confirmed quote.

While an item is in needs_review:

• It is not charged. Its billing is not_billed, and its share of the hold is released back to your balance.
• It has no result: result_available is false and GET /items/{id}/result returns 409 result_unavailable.
• The batch does not wait for it. A needs_review item counts as not succeeded: if no item in the batch succeeded, the batch ends failed with failure_reason all_items_failed; if some did, it ends completed_with_errors. counts.needs_review shows how many items are held, and an item.completed webhook with status needs_review is sent for each.

We review held items. If we approve one, it becomes succeeded and its result becomes available. It is charged then (billing charged), never more than its share of the confirmed quote, with one exception: if the same content was already charged within the reuse window, or the item was quoted at no charge, it is approved with billing reused and nothing is charged. The batch can then move to completed or completed_with_errors (a failure_reason is cleared), announced by item.completed and batch.completed webhooks. An item that is not approved stays in needs_review and is never charged.

Item reason

None of these is charged.

Webhooks

Set webhook_url on a batch to receive signed POST requests. We issue your signing secret through a secure channel. To rotate the secret, ask us for a new one; the old secret keeps working for 24 hours.

Events

Payload

{
  "event_id": "item.completed:5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
  "type": "item.completed",
  "batch_id": "8d0c7c1e-4b0f-4f1a-9a57-0f3f6a0f2b11",
  "item_id": "5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
  "status": "succeeded",
  "sequence": 7,
  "occurred_at": "2026-11-01T18:40:02.311Z"
}

• sequence numbers the events of one batch 1, 2, 3, … with no gaps. Events of one batch are delivered in that order: event n+1 is not sent until event n was accepted or given up on.
• An event that keeps failing holds back later events of its batch for at most 5 attempts (about 10 minutes). After that, later events are delivered while it keeps retrying, so order is no longer guaranteed. Always order by sequence on your side. Events of different batches are independent.
• Delivery is at least once. Deduplicate on event_id, which is also in the X-Lumen-Event-Id header.
• Respond with any 2xx within 10 seconds.
• 408, 429, 5xx, connection errors and timeouts are retried with exponential backoff for up to 24 hours. Any other 4xx is final: the event is not retried.
• A webhook_url that resolves to a private or reserved address is never delivered to. The event is given up on after 3 such attempts in a row, about 5 minutes apart.
• Webhooks are notifications. Read the current state from the API before acting on money or results.

Headers and signature

Content-Type: application/json
X-Lumen-Event-Id: item.completed:5a1e...
X-Lumen-Event-Type: item.completed
X-Lumen-Timestamp: 1793556002
X-Lumen-Signature: v1=5f0c...[,v1=...]

The signature is HMAC-SHA256(secret, "<X-Lumen-Timestamp>.<raw request body>"), hex-encoded with the prefix v1=. During a secret rotation the header carries several comma-separated v1= values; accept the request if any one matches. When verifying, reject timestamps more than 5 minutes from your clock, and verify against the raw body bytes, before any JSON parsing.

Verification: Node.js

import crypto from "node:crypto";
import express from "express";

const SECRET = process.env.LUMEN_WEBHOOK_SECRET;

// Returns true only for a fresh timestamp and a matching v1= signature.
// Malformed headers return false; they never throw.
function verifyLumenSignature(secret, ts, header, rawBody, nowSeconds = Date.now() / 1000) {
  if (typeof ts !== "string" || !/^[0-9]{1,12}$/.test(ts)) return false;
  if (Math.abs(nowSeconds - Number(ts)) > 300) return false;
  const expected = crypto
    .createHmac("sha256", secret)
    .update(Buffer.concat([Buffer.from(`${ts}.`), rawBody]))
    .digest(); // 32 bytes
  let ok = false;
  for (const token of String(header ?? "").split(",")) {
    const m = /^v1=([0-9a-fA-F]{64})$/.exec(token.trim());
    if (!m) continue; // skip malformed tokens
    ok = crypto.timingSafeEqual(Buffer.from(m[1], "hex"), expected) || ok;
  }
  return ok;
}

const app = express();
app.post("/lumen", express.raw({ type: "application/json" }), (req, res) => {
  const ok = verifyLumenSignature(SECRET, req.get("X-Lumen-Timestamp"), req.get("X-Lumen-Signature"), req.body);
  if (!ok) return res.sendStatus(400);
  const event = JSON.parse(req.body.toString("utf8"));
  // Deduplicate on event.event_id; order per batch_id by event.sequence.
  res.sendStatus(204);
});

Verification: Python

import hashlib, hmac, json, os, re, time
from flask import Flask, request, abort

SECRET = os.environ["LUMEN_WEBHOOK_SECRET"].encode()
app = Flask(__name__)

def verify_lumen_signature(secret, ts, header, raw_body, now=None):
    """True only for a fresh timestamp and a matching v1= signature.
    Malformed headers return False; they never raise."""
    if not isinstance(ts, str) or not re.fullmatch(r"[0-9]{1,12}", ts):
        return False
    if abs((time.time() if now is None else now) - int(ts)) > 300:
        return False
    expected = hmac.new(secret, ts.encode("ascii") + b"." + raw_body, hashlib.sha256).digest()
    ok = False
    for token in (header or "").split(","):
        m = re.fullmatch(r"v1=([0-9a-fA-F]{64})", token.strip())
        if not m:
            continue  # skip malformed tokens
        ok = hmac.compare_digest(bytes.fromhex(m.group(1)), expected) or ok
    return ok

@app.post("/lumen")
def lumen_webhook():
    body = request.get_data()  # raw bytes
    if not verify_lumen_signature(SECRET, request.headers.get("X-Lumen-Timestamp"),
                                  request.headers.get("X-Lumen-Signature"), body):
        abort(400)
    event = json.loads(body)
    # Deduplicate on event["event_id"]; order per batch_id by event["sequence"].
    return "", 204

Result format

Each succeeded item has one JSON document. It follows the JSON Schema lumen.egocentric-actions.v1, returned in full by GET /schema. In outline:

• Times are seconds on the file's player timeline.
• All timelines start at 0 and end together.
• The timelines cover the whole video stream, except that they may end slightly before its end: a final fragment shorter than 0.25 s can be left out (plus a measurement tolerance of 0.1 s). So the last end_s can be a fraction of a second below input.video_start_s + input.duration_s.
• Spans that could not be labeled have action: "unlabeled".
• If the video stream starts after the file does (for example, audio starts first), every timeline begins with one unlabeled segment [0, video_start_s) described as "no video". That segment is not counted in covered_seconds.

Limits

Errors

Errors use HTTP status codes and a JSON body with a stable code, a human-readable message and a retryable flag.

{
  "error": {
    "code": "invalid_request",
    "message": "inputs must hold 1–2000 entries.",
    "retryable": false
  }
}

Retry only when retryable is true (or on 429 and 503), with backoff. Reuse the same Idempotency-Key when retrying POST /batches.

Billing

Lumen is billed per minute of processed video, from the same prepaid balance as Trio-Spark. Add credits in the console (Billing); there is no subscription.

• Quote first. Before anything is charged, every batch returns a quote with the exact terms for that batch: quote.price_per_hour_usd (the rate), quote.min_billable_minutes (the minimum per item) and quote.amount_usd (the most the batch can cost). The terms are fixed when the quote is made and apply to the batch once you confirm it. If a quote expires before you confirm, the new quote carries the terms in effect then. Nothing is charged until a quote is confirmed.
• Per item. Each succeeded item is billed on the video time it covers, rounded up to whole minutes, with a minimum of min_billable_minutes per item, currently 2 minutes. For a charged item, billable minutes = the larger of the minimum and the covered time rounded up, where covered time is first capped at the video stream's duration (duration_s). For example, covered_s 180.05 on a 180-second video bills 3 minutes, not 4. A reused item bills 0 minutes; the minimum applies only to charged items.
• Not charged. Skipped, failed, cancelled and needs_review items are not charged. A result reused from content already charged within the reuse window (billing reused) is free.
• Budget. Set max_amount_usd on the batch to confirm quotes within that budget automatically. Without it, every non-zero quote waits in awaiting_confirmation for your confirm.
• Hold. The quoted amount is held on your balance when the batch is confirmed. You are never charged more than the confirmed quote, and only succeeded items are charged. Unused hold is released when the batch ends.
• Batch total. The charge for a batch is the sum over its succeeded items, never more than quote.amount_usd.
• Balance. There is no balance endpoint in the API. Check your balance and add credits in the console (Billing). A batch your balance cannot cover waits in blocked with insufficient_funds and resumes once the balance covers it.

Worked example (minimum of 2 minutes per item):

For volume or a higher credit limit, talk to us.