The API documentation is available in English.

Timed labels for first-person video.

Trio-Lumen labels first-person (egocentric) video with timed activity and per-hand action segments. You submit a batch of videos, review the quote, and fetch one verified JSON result per video.

• Base URL: https://platform.machinefi.com/api/lumen/v1
• All requests and responses are JSON over HTTPS.
• All times are ISO 8601 UTC. All money amounts are decimal strings in USD.

Getting started

1. Sign up at platform.machinefi.com.
2. Add credits to your balance in the console (Billing). Lumen and Trio-Spark share one prepaid balance; there is no subscription. See Billing.
3. Create an API key with Lumen enabled. In the console (API Keys), check Trio-Lumen when you create the key (it is checked by default). The secret is shown once; store it on your server.
4. Submit a batch with POST /batches (see Workflow), using presigned https URLs as inputs and the default hosted output, then poll GET /batches/{id} and fetch results with GET /items/{id}/result.

curl https://platform.machinefi.com/api/lumen/v1/batches \
  -H "Authorization: Bearer $TRIO_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "task": "egocentric-actions",
    "inputs": [{ "url": "https://storage.example.com/raw/clip-001.mp4?X-Amz-Signature=..." }],
    "output": { "type": "hosted" }
  }'

Steps 1–4 are fully self-serve. Two options need us for now; contact us to set them up:

• Your own bucket (read inputs from a prefix, or write results back): create credentials with the bucket permissions below and send the bucket details through a secure channel. We register it and give you its src_... id.
• Webhooks: we issue your signing secret through a secure channel. Until then, poll for status.

Authentication

Send your API key as a bearer token:

Authorization: Bearer tf_...

Create the key in the platform console with Trio-Lumen checked. A key can be enabled for Trio-Spark, Trio-Lumen, or both; this is fixed when the key is created. A key that is not enabled for Lumen gets 403 missing_scope: create a new key with Lumen checked (you can then revoke the old one). Keys created through the console API without a products list are enabled for Trio-Spark only. Keep keys server-side and never embed them in clients.

Workflow

1. Create a batch

Send your video inputs. The batch starts in preflight: we fetch and inspect every video and compute a quote.

2. Quote

• If the quote is within max_amount_usd, the batch is confirmed automatically and starts running.
• A zero quote is confirmed automatically, even without max_amount_usd.
• If every input is skipped, there is nothing to quote: the batch fails with no_processable_items and nothing is charged.
• Otherwise it waits in awaiting_confirmation until you call POST /batches/{id}/confirm.
• 144 hours after creation, a batch still in awaiting_confirmation, or blocked before any funds were reserved for it, is cancelled automatically. Nothing is charged.
• Quotes are valid for 24 hours and are re-quoted when they expire.
• The quote is an upper bound: you are never charged more than the confirmed quote.

3. Processing

Each video becomes an item and is processed independently. Items that finish with a verified result are succeeded and are charged on the video time covered, rounded up to whole minutes, with a minimum of one minute per item.

4. Results

• Fetch each result with GET /items/{id}/result, or have them written to your bucket (output.type = "bucket").
• Hosted results are kept for 30 days.
• Subscribe to webhooks instead of polling.

Only succeeded items are charged. Skipped, failed, cancelled and needs_review items are not charged. An item in needs_review is checked by us. If we approve it, it becomes succeeded and is charged then.

Endpoints

Create a batch

POST https://platform.machinefi.com/api/lumen/v1/batches

{
  "task": "egocentric-actions",
  "inputs": [
    {
      "url": "https://storage.example.com/raw/clip-001.mp4?X-Amz-Signature=..."
    },
    {
      "source": "src_3f2a...",
      "prefix": "raw/2026-11/"
    }
  ],
  "max_amount_usd": "250",
  "output": {
    "type": "hosted"
  },
  "webhook_url": "https://hooks.example.com/lumen",
  "metadata": {
    "project": "warehouse-q4"
  }
}

Response: 202 Accepted with the batch object.

Bucket output. Every batch writes into its own folder, <prefix><batch_id>/, so batches that share an output prefix never touch each other's files:

• Each succeeded item is written to <prefix><batch_id>/<item_id>.json.
• When the batch ends as completed, completed_with_errors or failed, <prefix><batch_id>/_COMPLETE.json is written. It carries batch_id and is written only after every succeeded item's file is in place, so its presence means the set is complete.
• It is not written for a cancelled batch, or for a batch that fails during preflight (for example too_many_items), since no item file was written.
• It is rewritten if an approval changes the outcome. Its generation field increases with every new outcome; the file with the highest generation is current. Writes use conditional requests (If-None-Match / If-Match) so an older outcome never replaces a newer one. On storage that does not support conditional writes, a late older write is detected and corrected within about 10 minutes.
• _COMPLETE.json is a convenience marker. The API (GET /batches/{id}) and webhooks are the source of truth for a batch's status and items.

Idempotency.

• Repeating a request with the same Idempotency-Key returns the original batch with the header Idempotent-Replayed: true. No second batch is created.
• The request comparison ignores metadata. A retry that differs only in metadata replays the original batch, and the new metadata is not applied.
• Reusing a key with any other field changed returns 409 idempotency_conflict.

Batch object

{
  "object": "batch",
  "id": "8d0c7c1e-4b0f-4f1a-9a57-0f3f6a0f2b11",
  "status": "running",
  "blocked_reason": null,
  "failure_reason": null,
  "cancel_requested": false,
  "task": "egocentric-actions",
  "processing_version": "egocentric-actions@<version>",
  "created_at": "2026-11-01T18:02:11.204Z",
  "confirmed_at": "2026-11-01T18:04:40.017Z",
  "finished_at": null,
  "preflight": {
    "pending_inputs": 0
  },
  "quote": {
    "amount_usd": "...",
    "price_per_hour_usd": "...",
    "min_billable_minutes": 1,
    "quoted_at": "2026-11-01T18:04:39.950Z",
    "expires_at": "2026-11-02T18:04:39.950Z"
  },
  "max_amount_usd": "250",
  "amounts": {
    "reserved_usd": "...",
    "settled_usd": "..."
  },
  "counts": {
    "queued": 40,
    "running": 4,
    "publishing": 0,
    "succeeded": 12,
    "needs_review": 0,
    "failed": 0,
    "skipped": 1,
    "cancelled": 0
  },
  "output": {
    "type": "hosted"
  },
  "webhook_url": "https://hooks.example.com/lumen",
  "metadata": {
    "project": "warehouse-q4"
  }
}

• processing_version above is a placeholder. The actual value is the task version pinned on your batch; see the available versions in GET /schema.
• quote is null until preflight finishes.
• amounts.reserved_usd is the part of the quote still held on your balance.
• amounts.settled_usd is what has been charged so far.

List batches

GET /batches?limit=20&cursor=...

• limit is 1–100 (default 20).
• Returns { "object": "list", "data": [batch...], "next_cursor": "..." | null }.
• Pass next_cursor back as cursor to get the next page.

Confirm or resume

POST /batches/{id}/confirm, no body. Returns the batch object.

• In awaiting_confirmation: reserves the quoted amount and starts processing. Returns 409 quote_expired if the quote expired; a new quote follows automatically.
• In blocked for insufficient_funds or over_limit: retries the reservation after you add funds.
• During preflight: returns 409 not_ready.

Cancel

POST /batches/{id}/cancel, no body. Returns the batch object.

• Queued items are cancelled and not charged.
• Items already running finish and are charged normally.
• The batch becomes cancelled once every item has stopped.

If your Lumen access ends, you can no longer create or confirm batches, but you can still list, read and cancel your existing batches and download their results. Batches still open 24 hours after access ended are cancelled as above: queued items are not charged, and items already running finish and are charged normally.

List items

GET /batches/{id}/items?status=succeeded&limit=100&cursor=... · limit is 1–500 (default 100); status filters by item status.

{
  "object": "list",
  "data": [
    {
      "object": "item",
      "id": "5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
      "batch_id": "8d0c7c1e-4b0f-4f1a-9a57-0f3f6a0f2b11",
      "position": 0,
      "input": {
        "type": "url",
        "index": 0,
        "host": "storage.example.com"
      },
      "status": "succeeded",
      "reason": null,
      "duration_s": 600,
      "container_duration_s": 600.5,
      "covered_s": 600,
      "billing": "charged",
      "billable_minutes": 10,
      "charged_usd": "...",
      "result_available": true,
      "result_expires_at": "2026-12-01T18:40:02.000Z",
      "created_at": "2026-11-01T18:02:11.204Z",
      "finished_at": "2026-11-01T18:40:02.000Z"
    }
  ],
  "next_cursor": null
}

Get a result

GET /items/{id}/result

{
  "object": "result",
  "item_id": "5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
  "schema": "lumen.egocentric-actions.v1",
  "sha256": "9b1f...",
  "bytes": 48213,
  "url": "https://...",
  "url_expires_at": "2026-11-01T18:55:02.000Z",
  "retained_until": "2026-12-01T18:40:02.000Z"
}

• url is a download link valid for 15 minutes. Call this endpoint again for a fresh link.
• Check the downloaded bytes against sha256.
• 409 result_unavailable: the item has no result. 409 result_expired: the item is past its 30-day retention.

Schema

GET /schema returns id (lumen.egocentric-actions.v1), tasks (the available versions, with latest marked) and json_schema (the full result JSON Schema).

Bucket permissions

Your own storage (optional, set up with us). To read inputs from, or write results back to, your own storage, you provide an S3-compatible bucket you control (AWS S3, Cloudflare R2, or another S3-compatible store) and credentials with read access to the input prefix and read and write access to the output prefix. Credentials without the permissions below are refused when the source is registered. Without a bucket, use presigned URL inputs and hosted results.

The credentials you give us need:

When the bucket is registered, and whenever its credentials are replaced, we check them with a small test object at <write prefix>.lumen-selftest/<random>.json: read of a missing key, write, read back, and two conditional writes that must be refused. Credentials missing a required permission are not accepted. The test object is deleted afterwards; without s3:DeleteObject it stays (a few bytes) and can be removed at any time.

AWS IAM example (inputs under raw/, outputs under lumen-out/):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::<bucket>",
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "raw/*",
            "lumen-out/*"
          ]
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": [
        "arn:aws:s3:::<bucket>/raw/*",
        "arn:aws:s3:::<bucket>/lumen-out/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::<bucket>/lumen-out/*"
    },
    {
      "Effect": "Allow",
      "Action": "s3:DeleteObject",
      "Resource": "arn:aws:s3:::<bucket>/lumen-out/.lumen-selftest/*"
    }
  ]
}

Cloudflare R2: an R2 API token with Object Read & Write on the bucket covers all of the above (R2 tokens are scoped per bucket, not per prefix).

Statuses and reasons

Batch status

A finished batch can change from completed_with_errors or failed to completed (or completed_with_errors) when we approve an item that was in needs_review. A batch.completed webhook announces the change.

blocked_reason

failure_reason

no_processable_items, all_items_failed, too_many_items, batch_too_large, listing_too_large, source_unavailable, credentials_invalid, preflight_timeout (inputs could not be inspected within 24 hours), processing_error.

Item status

queued, running, publishing (result being verified), succeeded, needs_review, failed, skipped, cancelled.

Item reason

None of these is charged.

Webhooks

Set webhook_url on a batch to receive signed POST requests. We issue your signing secret through a secure channel. To rotate the secret, ask us for a new one; the old secret keeps working for 24 hours.

Events

Payload

{
  "event_id": "item.completed:5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
  "type": "item.completed",
  "batch_id": "8d0c7c1e-4b0f-4f1a-9a57-0f3f6a0f2b11",
  "item_id": "5a1e9a42-0b9e-4c55-8f0e-2a7b0f9c1d33",
  "status": "succeeded",
  "sequence": 7,
  "occurred_at": "2026-11-01T18:40:02.311Z"
}

• sequence numbers the events of one batch 1, 2, 3, … with no gaps. Events of one batch are delivered in that order: event n+1 is not sent until event n was accepted or given up on.
• An event that keeps failing holds back later events of its batch for at most 5 attempts (about 10 minutes). After that, later events are delivered while it keeps retrying, so order is no longer guaranteed. Always order by sequence on your side. Events of different batches are independent.
• Delivery is at least once. Deduplicate on event_id, which is also in the X-Lumen-Event-Id header.
• Respond with any 2xx within 10 seconds.
• 408, 429, 5xx, connection errors and timeouts are retried with exponential backoff for up to 24 hours. Any other 4xx is final: the event is not retried.
• A webhook_url that resolves to a private or reserved address is never delivered to. The event is given up on after 3 such attempts in a row, about 5 minutes apart.
• Webhooks are notifications. Read the current state from the API before acting on money or results.

Headers and signature

Content-Type: application/json
X-Lumen-Event-Id: item.completed:5a1e...
X-Lumen-Event-Type: item.completed
X-Lumen-Timestamp: 1793556002
X-Lumen-Signature: v1=5f0c...[,v1=...]

The signature is HMAC-SHA256(secret, "<X-Lumen-Timestamp>.<raw request body>"), hex-encoded with the prefix v1=. During a secret rotation the header carries several comma-separated v1= values; accept the request if any one matches. When verifying, reject timestamps more than 5 minutes from your clock, and verify against the raw body bytes, before any JSON parsing.

Verification: Node.js

import crypto from "node:crypto";
import express from "express";

const SECRET = process.env.LUMEN_WEBHOOK_SECRET;

// Returns true only for a fresh timestamp and a matching v1= signature.
// Malformed headers return false; they never throw.
function verifyLumenSignature(secret, ts, header, rawBody, nowSeconds = Date.now() / 1000) {
  if (typeof ts !== "string" || !/^[0-9]{1,12}$/.test(ts)) return false;
  if (Math.abs(nowSeconds - Number(ts)) > 300) return false;
  const expected = crypto
    .createHmac("sha256", secret)
    .update(Buffer.concat([Buffer.from(`${ts}.`), rawBody]))
    .digest(); // 32 bytes
  let ok = false;
  for (const token of String(header ?? "").split(",")) {
    const m = /^v1=([0-9a-fA-F]{64})$/.exec(token.trim());
    if (!m) continue; // skip malformed tokens
    ok = crypto.timingSafeEqual(Buffer.from(m[1], "hex"), expected) || ok;
  }
  return ok;
}

const app = express();
app.post("/lumen", express.raw({ type: "application/json" }), (req, res) => {
  const ok = verifyLumenSignature(SECRET, req.get("X-Lumen-Timestamp"), req.get("X-Lumen-Signature"), req.body);
  if (!ok) return res.sendStatus(400);
  const event = JSON.parse(req.body.toString("utf8"));
  // Deduplicate on event.event_id; order per batch_id by event.sequence.
  res.sendStatus(204);
});

Verification: Python

import hashlib, hmac, json, os, re, time
from flask import Flask, request, abort

SECRET = os.environ["LUMEN_WEBHOOK_SECRET"].encode()
app = Flask(__name__)

def verify_lumen_signature(secret, ts, header, raw_body, now=None):
    """True only for a fresh timestamp and a matching v1= signature.
    Malformed headers return False; they never raise."""
    if not isinstance(ts, str) or not re.fullmatch(r"[0-9]{1,12}", ts):
        return False
    if abs((time.time() if now is None else now) - int(ts)) > 300:
        return False
    expected = hmac.new(secret, ts.encode("ascii") + b"." + raw_body, hashlib.sha256).digest()
    ok = False
    for token in (header or "").split(","):
        m = re.fullmatch(r"v1=([0-9a-fA-F]{64})", token.strip())
        if not m:
            continue  # skip malformed tokens
        ok = hmac.compare_digest(bytes.fromhex(m.group(1)), expected) or ok
    return ok

@app.post("/lumen")
def lumen_webhook():
    body = request.get_data()  # raw bytes
    if not verify_lumen_signature(SECRET, request.headers.get("X-Lumen-Timestamp"),
                                  request.headers.get("X-Lumen-Signature"), body):
        abort(400)
    event = json.loads(body)
    # Deduplicate on event["event_id"]; order per batch_id by event["sequence"].
    return "", 204

Result format

Each succeeded item has one JSON document. It follows the JSON Schema lumen.egocentric-actions.v1, returned in full by GET /schema. In outline:

• Times are seconds on the file's player timeline.
• All timelines start at 0 and end together.
• Spans that could not be labeled have action: "unlabeled".
• If the video stream starts after the file does (for example, audio starts first), every timeline begins with one unlabeled segment [0, video_start_s) described as "no video". That segment is not counted in covered_seconds.

Limits

Errors

Errors use HTTP status codes and a JSON body with a stable code, a human-readable message and a retryable flag.

{
  "error": {
    "code": "invalid_request",
    "message": "inputs must hold 1–2000 entries.",
    "retryable": false
  }
}

Retry only when retryable is true (or on 429 and 503), with backoff. Reuse the same Idempotency-Key when retrying POST /batches.

Billing

Lumen is billed per minute of processed video, from the same prepaid balance as Trio-Spark. Add credits in the console (Billing); there is no subscription. Before any processing, every batch gets a quote, and the quoted amount is held on your balance when the batch is confirmed. You are never charged more than the confirmed quote. Only succeeded items are charged, on the video time covered, rounded up to whole minutes with a minimum of one minute per item. For volume or a higher credit limit, talk to us.