meme-creator

Meme Creator

Turn a video clip into a meme: images (logos, avatars, stickers) glued onto

moving objects so they follow the motion frame by frame, delivered as MP4 + GIF.

The failure mode that ruins this is not tracking drift — it is gluing the

wrong person's avatar on screen because a name was bound to the first

plausible account found. So step 0 is identity, not tooling.

For a static image meme, skip tracking entirely: put one manual_keys entry

per target ([[1, [x, y, w, h]]]) in the overlays config and render just that

frame — --tracks is optional when every position comes from keys.

Requirements

• ffmpeg on PATH
• uv — all bundled Python scripts carry inline dependencies; run them with

uv run <script> and deps resolve themselves

• yt-dlp only when downloading (uv run --with yt-dlp yt-dlp ...)
• Chrome/Chromium only when rasterizing an SVG logo (asset-binding.md)
• Script paths below are written scripts/… **relative to this skill's bundle

directory**. Resolve the bundle path once (SKILL_DIR=<path to this skill>)

and run "$SKILL_DIR"/scripts/make_sheets.py … — your working directory is

the scratch dir, not the bundle.

Pipeline

Work inside a scratch directory (/tmp/<meme-name>/ or similar), deliver only

the final MP4 + GIF.

0. Bind identities before fetching assets

When the user names who/what goes on screen ("Codex, Claude, and Tibo's

avatar"), every ambiguous name must survive the disambiguation gate in

[references/asset-binding.md](references/asset-binding.md) — enumerate

candidates with a handle-free search, discriminate from the request context

(the peer entities listed alongside are the strongest signal), and ask the

user when context cannot decide. That file also covers logo/avatar sourcing,

SVG→PNG with real transparency via headless Chrome, and badge styling. Fetch

and eyeball every image before compositing it.

1. Acquire the source

Local file: use it. URL: yt-dlp handles bilibili (incl. b23.tv short links),

YouTube, and most sites:

uv run --with yt-dlp yt-dlp -f "bv*+ba/b" --merge-output-format mp4 -o "source.%(ext)s" "<url>"

If a domestic-CN site fails with a proxy tunnel error, retry with --proxy ""

(a local HTTP proxy breaks some domestic CDNs); if an international site times

out, retry with the local proxy.

Check duration: ffprobe -v error -show_entries format=duration -of csv=p=0 source.mp4

2. Locate the segment

Never track "the whole video." Find the exact scene with a 1–2 fps contact

sheet, then expand around it:

ffmpeg -v error -ss <start_s> -t 40 -i source.mp4 -vf "fps=1,scale=240:-1,tile=5x8" -frames:v 1 contact.png

Read the sheet, pick the beat boundaries (entrance → action → exit), and cut

the final segment ±1 s of padding. Extract the frame sequence at native fps:

mkdir -p seq && ffmpeg -v error -ss <seg_start> -i source.mp4 -t <seg_dur> seq/f_%04d.png
ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of csv=p=0 source.mp4

3. Prep overlay assets

One image per target, PNG with alpha. Unify mixed assets (app icon + logo mark

• photo avatar) as circle-white badges — see asset-binding.md for why and for

the avatar circular-mask handling.

4. Track the targets

Full procedure and failure taxonomy:

[references/tracking-playbook.md](references/tracking-playbook.md). Short form:

1. Grid one frame where all targets are fully visible:

uv run scripts/make_sheets.py grid --frames 'seq/f_%04d.png' --index 30 --out grid.png

and read each target's [x, y, w, h] box off the grid.

1. Write the tracking config (targets + init frame), run:

uv run scripts/track_boxes.py --frames 'seq/f_%04d.png' --config track.json --out tracks.json --last <N>

1. Verify: uv run scripts/make_sheets.py tile --frames 'seq/f_%04d.png' --tracks tracks.json --start 1 --end <N> --step-frames 15 --out verify.png

and look at every tile.

1. Fix what the tiles show: re-anchor later segments (shot changes, scale

blowups, drift), or take over a smooth long shot with manual_keys. Expect

2–3 rounds. Then mark visible/fade_in/fade_out for every exit,

entrance, and empty-stretch — a logo parked on an empty frame is the most

visible possible bug.

5. Render

uv run scripts/render_overlay.py --frames 'seq/f_%04d.png' --tracks tracks.json \
  --overlays overlays.json --out 'out/o_%04d.png' --last <N>

Spot-check a rendered contact sheet (make_sheets.py tile over out/) before

encoding — much cheaper than re-encoding.

6. Encode + size budget

scripts/encode_outputs.sh --frames 'out/o_%04d.png' --fps <native_fps> \
  --source source.mp4 --ss <seg_start> --t <seg_dur> --mp4 meme.mp4 \
  --gif meme.gif --gif-width 400 --gif-fps 10 --gif-colors 96

GIF size is driven by frame count × dither noise, not palette size. To fit a

~10 MB chat-app budget, lower knobs in this order: --gif-fps (15→12→10),

then --gif-width (480→400), then --gif-colors (128→96).

7. Verify the encoded deliverables, then deliver

Extract frames from the final MP4 and the final GIF (`ffmpeg -ss <t>

-frames:v 1`) and look at them — the earlier checks verified intermediates;

this one verifies what the recipient will actually see. Deliver both files by

their exact paths.

Troubleshooting

| Symptom | Cause | Fix |

|---|---|---|

| yt-dlp: Tunnel connection failed 503 on a domestic site | local HTTP proxy intercepting | add --proxy "" |

| cairosvg/svglib crash: no library called cairo | missing system cairo | rasterize SVG via headless Chrome (asset-binding.md) |

| Tracker dies exactly at a shot change | CSRT cannot cross cuts | new segments entry at the new shot's first frame |

| Box drifts slowly off the target over ~1 min | template pollution | re-anchor with the tracker's own last-good box, or manual_keys |

| Logo parked mid-frame after subject exits | missing visibility window | fade_out before the exit, no visible range after |

| GIF massively over budget | dither noise × frames | lower --gif-fps first, then width, then colors |

| Logo looks fine in PNGs but wrong in the GIF | checked intermediates only | always verify frames from the encoded file (step 7) |

| Encode fails: Failed to set value '1:a' for option 'map' | source clip has no audio stream | the bundled script maps 1:a? (optional); if running ffmpeg by hand, do the same or drop --source |

References

• [references/asset-binding.md](references/asset-binding.md) — identity

disambiguation gate, logo/avatar sourcing, transparency, badge styling

• [references/tracking-playbook.md](references/tracking-playbook.md) — CSRT

failure taxonomy, re-anchor recovery, manual keyframes, fades, verify loop