The research template takes attachments: drop a photo, a PDF or a video on the composer and ask about it. This page is how that works, and what to copy into your own harness.
Three rules
- Bytes go to the store, never to the model or the wire. An attachment is posted to the host's content plane, which works out what it is, normalises an image or reads a PDF into pages, and commits the bytes to
media/in the project. From then on, commands, events and the kept record carry only its digest. - An image is projected once. It goes onto the trunk before the agents fork, so every agent attends the same cells: N agents cost one projection.
- A document is read, not rendered. Its text is staged — listed on the shared spine, costing nothing until an agent reaches for it — and a page becomes an image in the model's context only when an agent decides it needs to see it.
The store
media/ at the project root is a content-addressed store in the OCI Image Layout — the format container registries use. Every blob is named by the SHA-256 of its bytes, so the same bytes are stored once, a citation that names a digest can only mean them, and anything that reads OCI can read the store: oras can push a run's media to any registry.
It sits beside models/, apart from what the app writes, because it is what the app was given. It outlives every session: a reopened brief finds its pictures and documents there again. It is never switched off, because a run whose inputs were not addressed cannot be replayed.
Take an attachment
The command carries descriptors, not bytes:
import type { Descriptor } from "@lloyal-labs/media";
export type Command =
| { type: "submit_query"; query: string; attachments?: Descriptor[] }
// …A descriptor arriving from a client is a claim, so check it once with admitted, then project what needs projecting:
import { admitted } from "@lloyal-labs/rig";
import { waitUntilSettled } from "@lloyal-labs/lloyal-agents";
const seen = yield* admitted(attachments);
if ("refused" in seen) return yield* wire.send({ type: "ui:error", message: seen.refused });
// Images onto the trunk, once, before any agent forks.
yield* waitUntilSettled(session.prefillUserMultimodal(text, seen.bitmaps, { attachments: seen.projected }));admitted asks three questions in order: is this the kind of thing that can be an attachment; is the content really in the store (a forged descriptor fails here); and — only for what is actually pixels — is there a vision service to see it. A document needs no projector. A refusal moves nothing; say it and return.
Pass seen.roots to the pool as attachments, and every tool call can see which files are available to the run.
Documents
The documents Ability gives agents three tools over what was attached:
| Tool | Does |
|---|---|
search_documents |
Narrows lexically first, then reranks and keeps the best passages within a token budget. It always scores in explore stance: an attached document is the on-topic universe. |
read_document |
Returns the exact text — only the part this agent has not already read. Ask for a page and you get the whole sections that touch it, so a table across a page break stays whole. |
view_page |
Hands the model one page, or one figure on it, as an image. Only pages worth the cells qualify — the first page, a page with no text, one with images, drawings, tables or figures; a text-only page refuses and points back to read_document. |
All three hand the model a ready-made citation, attachment://<digest prefix>/page/<n>, so it copies a citation rather than inventing one. The view resolves the prefix against digests it already holds; click it and that page opens.
Enable it as the research template does:
import { createDocumentsAbility } from "@lloyal-labs/documents-ability";
export const abilities = [createCorpusAbility, createWebAbility, createDocumentsAbility];It declares two services — reranker and vision — so the harness needs both blocks under model:, as the research template's harness.yml has them. Without them it is refused at enable, by name — see Services.
Video
A video is an attachment like any other: admitted checks it, and seen.bitmaps carries it as one input, its frames in order with time marks and its transcript beside them. seen.videos names the videos, so the question can tell the model each one's address. The page decodes it, the host stores it by digest, and the embedding service can search it by what was seen, said and heard. Video is the whole of it: how a video is taken, stored, projected, searched, cited and played.
How it compares with llama.cpp
lloyal lays a video out as llama.cpp's own video helper does, and differs in everything before that layout: who decodes, what is sampled, what is heard. The comparison is on Video.
Tune what an image costs
How many tokens one image is projected into is yours: model.vision.minTokens for the detail floor, model.vision.maxTokens for the ceiling. Grounding tasks want detail; wide fan-outs want room. See harness.yml.
Cap what attachments take
A long video, or many images, can cost more of the context than a turn can spare: at the default frame cap an hour-long recording brings 64 frames and an hour of transcript. The share of the context a turn's attachments may fill is the harness's attachments.context; give it to admitted, and the attachments are priced exactly before anything is read in:
attachments:
context: 0.7const seen = yield* admitted(attachments, { context: config().attachments.context });A turn over the share is refused with what it would cost and what is allowed, and the conversation is untouched. So is a turn that fits the share but not what the context has free now, later in a long conversation. With attachments.context unset in harness.yml, the share is the whole context: the harness still passes it to admitted, so the attachments are still priced against what is free, and a turn that does not fit is still refused before anything is read in. Only admitted called with neither a context nor a reserve prices nothing, and then an attachment that does not fit fails while it is read in. When a sequence slot is free, the turn is written on a fork of the conversation, so only the fork is lost and the conversation carries on (see fast-forward), but the reader has waited for nothing. With no slot free, the turn is written in place and a failure releases the conversation.
Leave room for the run
An attachment that fits can still leave the run nothing to work in. The agent pool starts an Agent only when its task prompt fits above softLimit, so a turn whose media leaves softLimit or less free starts a run in which every spawn is refused (pressure_init), and it ends with no answer. Give admitted a reserve, the cells that must stay free once the media lands, and it refuses that turn before anything is read in:
// The pool's own line for the effort this run will use.
const reserve = BUDGETS.effort[config().defaults.effort].context.softLimit;
const seen = yield* admitted(attachments, { context: config().attachments.context, reserve });The refusal says what the attachments cost and the most they may take. Take the reserve from the same budget row the run will use, so the check and the pool agree: in the research template that is 2,048 cells at high effort and 8,192 at medium. A reserve of exactly softLimit keeps the pool's line clear but leaves nothing above it for the Agents' task prompts; add what they take to the reserve when a run must be able to start. Either way the check is about starting a run, not about room to finish it.
Judge against where the media lands
admitted measures what is free in the context as it stands. A turn that starts a new conversation lets the current one go before its media is read in, so judge it against the whole context instead, with free:
// A new brief lets the one on the canvas go first: its media lands in an empty context.
const free = warm ? undefined : new ContextPressure(yield* Ctx.expect()).nCtx;
const seen = yield* admitted(attachments, { context: config().attachments.context, reserve, ...(free !== undefined ? { free } : {}) });Without it, a long conversation on screen would refuse media that a new brief has room for.
The three refusals, as the reader sees them:
| When | The sentence |
|---|---|
| Over the share | That attachment would take n of the m-token context; this harness lets attachments fill up to p% of it. Try a shorter clip or fewer attachments. |
| Over what is free | That attachment would take n tokens, but only f of the m-token context are free now. Try a shorter clip, fewer attachments, or a new brief. |
| Into the reserve | Not enough space for work after attachments (n tokens). Try smaller attachments or adjust budgets (k max allowed for attachments). |