Docs · Loop pack format
docs/ZOEYOS-LOOP-PACK-FORMAT.md in the ZoeyOS source. Last verified 2026-10-02. State the code moved since: 2026-10-08. Since the document: an agent connects at /connect and asks through /v1/ask; the voice module and the MCP server arrived 2026-10-01 to 10-05.The loop pack format
The handoff between a clip site and the player. A pack is read-only to the player: it plays what it is handed, and every adjustment to a library comes from the Composer.
The product rule
The public player ships with authoring off, which refuses every authoring route: the drill, the tuner, re-pinning, preset writes, a tune with a body, the transport (except stop), scrubbing. The intake chain, the pick installer and the tuner are in the private Composer package and in no pack. So a clip site's job ends at a raw film plus metadata. Turning that into a loadable face is the Composer's job.
Layout of a pack
<pack>/ README.md launch lines, build ref, Composer link-out scripts/house-conductor.py the player (one process per face) scripts/house-speak.py the voice + mouth-sheet publisher, run per line lib/*.py the player's modules avatars/<face>/manifest.json per-face measurements (painter frame, mouth boxes) display/<face>-poses/ the face's library
inventory.json the rows: every clip, its role and hubs hub-<hub>.png the plates every loop starts and ends on clip-<stem>.mp4 films clip-<stem>.playlist.json mouth playlist: per-frame station rows + tile index clip-<stem>.mouth.json per-frame mouth box, painter space clip-<stem>.head.json per-frame 3D head fit mouth-dials.json the library's one tune mouth-dials.clips.json per-clip overrides visemes/phoneme-map/map.json every tile's harvest record and head block visemes/phoneme-map/tiles/*.png the mouth tiles visemes/picks/map-best-<loop>.json the hand-picked library per loop (Composer input)
What ships is derived, never listed: the runtime files, every hub plate, every inventory row's film and sidecars, every picks file, and only the tiles some shipped playlist names. A playlist with an empty index is a hard error, because that loop would paint nothing.
What the player reads at runtime
| manifest | at import; missing is an error |
|---|---|
| inventory | at boot only; a new row, film, hub or face needs a restart of that face's player |
| playlists, mouth boxes, tiles | per paint, keyed on the file's time; a reload call drops the caches, no restart |
| the tune | the library file at boot; per-clip sets at every clip entry |
| picks files | never read by the player: the Composer's installer turns them into playlist index entries |
State (flags, preferences, locks, caches) lives in one folder named by the environment, the manifest, or a default in the user's home.
The key files, in brief
The manifest names the face, its library folder, the painter frame (736 by 400 at 24 fps; every mouth box is in it), the listening loop, and per-engine voice names. The inventory validates every row: roles are loop, slide, glance, flourish or oneshot; every row needs a file; a mouth box is four numbers. A loop starts and ends on the same hub; a slide joins two hubs; a flourish is dealt from a pool and may be barred from starting while she speaks. The playlist is the only file that tells the painter which tile to paste on which frame: an index of station, yaw bin and class (19 classes from rest to rest-open) to a tile, and one row per film frame; the rows must cover the whole film or the index resolves nothing. A slide or flourish that paints carries two more keys: paint, and the loop whose tiles and tune it wears.
Film requirements
- Size. 1296 by 704, or 16:9 HD with a 4 % top and bottom margin. Intake centre-crops and scales. One size per face.
- Frame rate. 24 fps containers. Frames and seconds are not linear inside a take: generators emit content-dependent duplicate frames.
- Loops are perfect loops. First and last frame are the hub plate. Intake scores the ends: under 0.80 is refused, 0.97 or better gets an 8-frame ease, between gets a 20-frame dissolve.
- A loop she speaks in has a still filmed mouth: closed lips, the jaw not moving. The painter pastes the mouth over the film; a talking film under the tile reads jumpy. Hands stay off the face.
- Slides are generated first frame and last frame; one film serves both directions. Silent.
- Talking takes are the same shot as their loop, filmed with the voice, and must carry their soundtrack: the words and the tiles come from it.
| hub loop | start = end; paints the mouth always; allowed during speech |
|---|---|
| slide | start ≠ end; paints only when stamped; allowed during speech |
| flourish | start = end, in a pool; paints only when stamped; barred from speech unless the row allows it |
| talking take | no hubs; never played on the wall; tile source only |
The handoff boundary
The site supplies, per film: the raw MP4 (soundtrack kept on a talking take); the face slug; the row key and role; the start and end hub; a label, and for a slide its reverse key and share group; for a talking take its pose class and the loops it feeds; for a new face, the still that becomes the hub plate.
The Composer produces: the slide intake (crop, score the ends, bake the ease onto the plates, encode, stamp the mouth sidecar and head track, refresh the row); the talking chain of thirteen idempotent steps (soundtrack, words, phonemes, head track, sidecar, tiles, labels, tile heads, tile marks, playlist, bank the other loops, inventory, notice the players); the picks by eye; the stamping of slides and flourishes; the tune; and the pack itself, re-encoded, scrubbed of names, locked down.
The site never writes a file in the face's library.
How the player learns about new content
- New or changed playlist, tile or mouth sidecar: one reload call; the next paint re-reads them.
- New inventory row, film, hub or face: a restart of that face's player.
- A whole new pack: unpack, launch one player per face.
- The import door (built 2026-10-02, off by default): a pack folder with a manifest of hashed files, inventory rows and map entries. The door only adds, verifies hashes, refuses a clip on a hub the library does not have, and installs a new hub only through the Composer with every row off until the owner says yes.
Connecting an agent
From the pack README: any server that speaks the OpenAI chat-completions API (a local model server, a hosted API, an agent framework's gateway) is connected on the player's own page; the address, key and model are kept in the state folder. She answers what is sent to the ask route; the page has a box for it. To talk to her out loud, the ear script reads the turns a speech-to-text service heard and asks through the same route, dropping what it hears while she speaks. Behaviour is driven by routes: comms for a conversation, cue for a named clip, home on the first tool call, stop to cut her mid-line.
Word timing
Live mouths get real word boundaries from a forced aligner on loopback. The speaker is fail-open: a missing, cold or slow aligner leaves the letter clock, which spreads the letters evenly. Measured against hand marks the letter clock scored 22 %, the aligned clock 58 %. The aligner is started by the player on first use; the voice itself is outside the pack.