Lloyaldocs
Video

Video protocol reference

Every command and event a video travels on, field by field. References only: bytes travel over HTTP.

On this page

The commands and events travel on the harness's wire like any other, and carry references only. A frame, a soundtrack window and the original reach the host over HTTP first, and a command names them by descriptor. Serve them with useVideoCommands from @lloyal-labs/rig; ingestVideo from @lloyal-labs/ui sends them.

Commands

Command Fields Asks
video:describe none How this host composes a video. Answered by video:capabilities.
video:compose requestId, frames, audio?, durationMs, source? Compose these into one video. Answered by video:composed, video:error or video:cancelled, with progress along the way.
video:cancel requestId Stop a compose in flight. Nothing is composed.

video:compose, field by field:

Field Is
requestId The page's name for this compose. A second compose under an id already in flight is refused, and the first stays cancellable.
frames { root, tMs }[]: each sampled frame's admitted image root, and the time it was sampled at, in order
audio { root, startMs, endMs }[]: each soundtrack window's admitted recording, and the span of the clip it covers, in order and not overlapping
durationMs The clip's length
source The original, pushed as a blob, kept as the video's source

Events

Event Fields Says
video:capabilities capabilities How this host composes a video (below)
video:progress requestId, phase: "transcribing", done, total How many soundtrack windows are transcribed
video:composed requestId, root, untranscribed The composed video's root, and each window the host could not transcribe, { startMs, endMs, reason }
video:error requestId, message Why nothing was composed, in a sentence a reader can be shown
video:cancelled requestId The compose stopped and nothing was composed

capabilities, field by field:

Field Is
sampling { fps, timestampIntervalMs, maxFrames, maxEdgePx, soundtrackWindowMs }: how the host samples, from attachments.video. soundtrackWindowMs is the length the page cuts the soundtrack into, in milliseconds, from soundtrackWindowSeconds
maxDurationMs The longest clip the host composes, when it sets a limit. The page refuses a longer one before pushing any byte.
soundtrack The limits each soundtrack window must meet to be kept, or null when nothing the host runs hears. The page cuts the soundtrack only when there are limits.

Routes

Uploading the original (OCI distribution blob push):

Route Does
GET /v2/ The API version check
POST /v2/<name>/blobs/uploads/ Opens an upload session, or uploads in one request with ?digest=
PATCH /v2/<name>/blobs/uploads/<id> Appends the next chunk. Content-Range names its bytes, both ends inclusive, and starts one past the last chunk's end: after 0-99, the next starts at 100
GET /v2/<name>/blobs/uploads/<id> How far the session got, to resume from
PUT /v2/<name>/blobs/uploads/<id>?digest= Closes the session: the whole is checked against the digest, then admitted
DELETE /v2/<name>/blobs/uploads/<id> Cancels the session
HEAD /v2/<name>/blobs/<digest> Whether a blob is stored

Reading a video (Content-addressed video storage): GET /v1/media/<root>, /config, /representations/<i> and /source, the last with range requests.

↑
Search