The commands and events travel on the harness's wire like any other, and carry references only. A frame, a soundtrack window and the original reach the host over HTTP first, and a command names them by descriptor. Serve them with useVideoCommands from @lloyal-labs/rig; ingestVideo from @lloyal-labs/ui sends them.
Commands
| Command | Fields | Asks |
|---|---|---|
video:describe |
none | How this host composes a video. Answered by video:capabilities. |
video:compose |
requestId, frames, audio?, durationMs, source? |
Compose these into one video. Answered by video:composed, video:error or video:cancelled, with progress along the way. |
video:cancel |
requestId |
Stop a compose in flight. Nothing is composed. |
video:compose, field by field:
| Field | Is |
|---|---|
requestId |
The page's name for this compose. A second compose under an id already in flight is refused, and the first stays cancellable. |
frames |
{ root, tMs }[]: each sampled frame's admitted image root, and the time it was sampled at, in order |
audio |
{ root, startMs, endMs }[]: each soundtrack window's admitted recording, and the span of the clip it covers, in order and not overlapping |
durationMs |
The clip's length |
source |
The original, pushed as a blob, kept as the video's source |
Events
| Event | Fields | Says |
|---|---|---|
video:capabilities |
capabilities |
How this host composes a video (below) |
video:progress |
requestId, phase: "transcribing", done, total |
How many soundtrack windows are transcribed |
video:composed |
requestId, root, untranscribed |
The composed video's root, and each window the host could not transcribe, { startMs, endMs, reason } |
video:error |
requestId, message |
Why nothing was composed, in a sentence a reader can be shown |
video:cancelled |
requestId |
The compose stopped and nothing was composed |
capabilities, field by field:
| Field | Is |
|---|---|
sampling |
{ fps, timestampIntervalMs, maxFrames, maxEdgePx, soundtrackWindowMs }: how the host samples, from attachments.video. soundtrackWindowMs is the length the page cuts the soundtrack into, in milliseconds, from soundtrackWindowSeconds |
maxDurationMs |
The longest clip the host composes, when it sets a limit. The page refuses a longer one before pushing any byte. |
soundtrack |
The limits each soundtrack window must meet to be kept, or null when nothing the host runs hears. The page cuts the soundtrack only when there are limits. |
Routes
Uploading the original (OCI distribution blob push):
| Route | Does |
|---|---|
GET /v2/ |
The API version check |
POST /v2/<name>/blobs/uploads/ |
Opens an upload session, or uploads in one request with ?digest= |
PATCH /v2/<name>/blobs/uploads/<id> |
Appends the next chunk. Content-Range names its bytes, both ends inclusive, and starts one past the last chunk's end: after 0-99, the next starts at 100 |
GET /v2/<name>/blobs/uploads/<id> |
How far the session got, to resume from |
PUT /v2/<name>/blobs/uploads/<id>?digest= |
Closes the session: the whole is checked against the digest, then admitted |
DELETE /v2/<name>/blobs/uploads/<id> |
Cancels the session |
HEAD /v2/<name>/blobs/<digest> |
Whether a blob is stored |
Reading a video (Content-addressed video storage): GET /v1/media/<root>, /config, /representations/<i> and /source, the last with range requests.
Related
- Video input: the order these are sent in.