Montefiore Urology · Nursing / NICU & L&D · build runbook

How "The Calm in the Storm" was made — build runbook

Contents1. What actually exists2. The input: a scene brief, not a video file3. The stack that did the work4. The build, step by step5. Making a new video — the fast path6. Pitfalls learned the hard way7. Measured performance (this host, re-verified Sep 2026)8. Where everything is

Reconstructed from the original build sessions, not from memory. Every script, command and

setting below was recovered verbatim from the June 2026 chat history and is preserved on disk at

/workspace/nursing-videos/recovered/original_scripts/.

The reused, generalized version of this pipeline lives in /workspace/nursing-videos/pipeline/

and was verified by rendering real video (pipeline/README.md).


1. What actually exists

filmpathsizeverified
v1 — generated stillsfinals/calm-in-the-storm-720p.mp440,090,678 B (38.2 MB)122s · 1280x720 · 24fps · audio yes
v2 — your clips mixed infinals/calm-in-the-storm-v2-with-user-clips-720p.mp440,105,460 B (38.2 MB)122s · 1280x720 · 24fps · audio yes

Both are the same 2:02 film. The difference is only the source material:

motion, phase labels, title card and credits. Script: calm-in-the-storm/build_video.py.

footage) and 4 Gemini-generated stills, mixed with generated scenes. Script:

calm-in-the-storm/build_v2.py.

Five other renders from the same period are gone (build folders were cleaned out) — listed on the

archive page. Their scripts are recoverable, so any of them can be rebuilt byte-for-byte from the

same inputs.

Rebuild proof (Sep 2026): v2's scene plan was re-rendered from scratch with the kit below —

pipeline/manifest.v2-reconstruction.json_smoketest/proof/calm-in-the-storm-v2-rebuilt-720p.mp4,

122.00s · 1280x720 · 24fps · 38.9 MB, i.e. the same film the original 122s / 38.2 MB / 1280x720

render describes. The only deliberate difference: this rebuild has no narration track, because the

original Edge-TTS mp3 lived in ~/.hermes/audio_cache/ and that cache has since been cleared.

2. The input: a scene brief, not a video file

Everything starts from one markdown document — `FROM CHAOS TO CLARITY: THE POWER OF A SAFETY

HUDDLE`. Original attachment preserved at

~/.hermes/webui/attachments/2a31996a5a44/pasted-text-2026-06-23_01-08-06-602.md.

Its shape (this is the format to reuse — see pipeline/scene_brief_template.md):

VIDEO LENGTH        60 seconds
STYLE               realistic hospital, L&D + NICU, blurred faces, documentary,
                    fast opening → calm coordinated ending
SCENE 1  THE SURGE            0-8s
  Canva AI Video Prompt   →  the image/video prompt
  Voiceover               →  the narration line
  Staff Dialogue          →  optional
  Text Overlay            →  the caption that appears on screen
SCENE 2 … SCENE 7
CLOSING  CLARITY / COORDINATION / SAFER CARE

Two things made it produceable:

1. One prompt + one narration line + one caption per scene. The prompt became a still image,

the narration became the timeline, the caption became the on-screen bar.

2. A 60s spec was allowed to run 122s. The narration, spoken at natural pace, set the real

length; scenes were stretched to fit it rather than trimming the script.

3. The stack that did the work

Counts are real tool invocations from the five build sessions (Jun 23 + Jun 27 2026).

ProgramRoleReal usage
Hermes image_generate (FLUX, hosted on fal.ai)One photographic still per scene72 calls
Hermes text_to_speech (Edge TTS)The narration; this is what set the 122s timeline8 calls
MoviePy 2.1.2 + ffmpegCompositing and encode: Ken Burns, overlays, grain, vignette, H.264409 terminal runs
Google Gemini (your subscription)4 stills you generated and suppliedyour account
Kling AIThe live-motion clip that opens the filmyour exports
Your phone / other recording5 clips mixed into v2your uploads

Scripts written during the build: 51 write_file calls (kept in recovered/original_scripts/).

Not used for the film, only evaluated: Veo 3 (8-second clip limit), InVideo AI, Canva, Runway,

LTX, ElevenLabs, Replicate.

Hard rule that came out of this project: fal.ai is banned for video generation (a Kling clip

rendered through it was unusable — black screen). Stills through Hermes image_generate are fine,

which is exactly what v1 used.

4. The build, step by step

Step 1 — Install the renderer

python3 -m pip install moviepy pillow numpy imageio-ffmpeg
# optional, for narration from the shell:  pip3 install edge-tts

ffmpeg is not on PATH. MoviePy reaches it through imageio-ffmpeg:

import imageio_ffmpeg; imageio_ffmpeg.get_ffmpeg_exe()
# .../imageio_ffmpeg/binaries/ffmpeg-linux-x86_64-v7.0.2

Step 2 — Generate the stills (one per scene)

For each scene, the brief's prompt goes straight into image generation, landscape aspect:

image_generate(prompt="Cinematic documentary style shot of charge nurse at crowded nurse "
                      "station in L&D, multiple staff approaching with questions, blurred "
                      "faces, realistic hospital environment, shallow depth of field",
               aspect_ratio="landscape")
# → {"image": "https://v3b.fal.media/files/b/.../x.png"}

Then download them into the project:

mkdir -p my-project/scenes
curl -sL -o my-project/scenes/scene1_chaos.png "https://v3b.fal.media/files/b/.../x.png"

Prompt formula that worked: `[shot type] of [subject] in [specific hospital area] +

[action/controlled chaos] + blurred faces, no patient identifiers + documentary style,

cinematic lighting, shallow depth of field + realism`. Keeping "blurred faces / no patient

identifiers" in every prompt was deliberate — it is the privacy constraint from the brief.

Step 3 — Record the narration and measure it

The narration is the clock. Narration text came from the brief's voiceover lines, concatenated.

text_to_speech(text="<full narration script>")
# → ~/.hermes/audio_cache/tts_20260623_044539.mp3
# scriptable equivalent (what pipeline/make_voiceover.py wraps):
python3 -m edge_tts -f script.txt -v en-US-ChristopherNeural \
  --write-media my-project/voiceover.mp3
# then read the real duration — build_v2.py used TOTAL = 121.85

Step 4 — Place your own footage and stills

v2's own copy command, verbatim from the session. Every asset was renamed to a scene name, so

the build script reads like the brief it came from:

SRC="/home/hermeswebui/.hermes/webui/attachments/0e5032b0f9df"
cp "$SRC/Gemini_Generated_Image_z692mez692mez692.png" scenes_v2/gemini_nicu.png
cp "$SRC/download-1.png"                              scenes_v2/download_workstation.png
cp "$SRC/Gemini_Generated_Image_afe95uafe95uafe9-1.png" scenes_v2/gemini_huddle.png
cp "$SRC/Gemini_Generated_Image_upd2yuupd2yuupd2-1.png" scenes_v2/gemini_team.png
cp "$SRC/download__1_-1.png"                          scenes_v2/download_nurse_baby.png
cp "$SRC/SCENE_ESCALATING_CHAOS_Dura-1.mp4"           scenes_v2/vid_chaos.mp4
cp "$SRC/kling_20260623_VIDEO_ROM_CHAOS__2494_0-1.mp4" scenes_v2/vid_kling_chaos.mp4
cp "$SRC/video-1.mp4"                                 scenes_v2/vid_bedside.mp4
cp "$SRC/video__1_-1.mp4"                             scenes_v2/vid_corridor.mp4
cp "$SRC/video__2_-1.mp4"                             scenes_v2/vid_team.mp4

Step 5 — Write the timeline

Explicit start + duration per scene, seconds from zero. Everything is placed with

with_start(...) on a black canvas, so a scene is simply "on screen between these two times".

scenes = [
    ("video", "vid_chaos.mp4",       0.0, 10.0, 0),      # escalating chaos (native 720p)
    ("video", "vid_kling_chaos.mp4", 10.0,  5.0, 0),
    ("image", "gemini_nicu.png",     15.0, 12.0, 0.07),  # extra = Ken Burns zoom amount
    ("image", "download_workstation.png", 27.0, 13.0, 0.06),
    ("image", "gemini_huddle.png",   40.0, 18.0, 0.08),  # the pivot
    ("video", "vid_team.mp4",        58.0,  8.0, 0),
    ("image", "gemini_team.png",     66.0, 16.0, 0.07),
    ("image", "download_nurse_baby.png", 82.0, 14.0, 0.06),
    ("video", "vid_corridor.mp4",    96.0, 10.0, 0),     # 480p phone clip, upscaled
    ("image", "gemini_nicu.png",    106.0, 10.0, 0.05),
    ("image", "download_nurse_baby.png", 116.0, 6.0, 0.04),
]

Four acts, matching the brief's emotional arc:

actwindowcolourmeaning
1 The Gathering Storm0–40s#E84855surge, tension, fragmentation
2 Finding the Calm40–58s#4ECDC4the safety huddle is called
3 Unifying for the Patient58–96s#45B7D1shared awareness, coordination
4 The Core Mission96–122s#F9C74Fcalm, clarity, closing

Step 6 — Assemble (the render)

The exact recipe from build_v2.py:

W, H, FPS = 1280, 720, 24
TOTAL = 121.85

# stills → Ken Burns: full-frame fit, then a slow zoom driven by time t
clip = ImageClip(path, duration=dur).resized((W, H))
clip = clip.resized(lambda t: 1 + zoom * (t / dur)).with_position(("center", "center"))

# your footage → fit height, centre-crop to width (protects against odd aspect ratios)
clip = VideoFileClip(path)
clip = clip.subclipped(start_trim, start_trim + dur) if start_trim else clip
clip = clip.resized(height=H)
if clip.w > W: clip = clip.cropped(x_center=clip.w / 2, width=W)
clip = clip.resized((W, H)).with_duration(dur)

# text: DejaVu Sans from matplotlib for EVERY overlay (no GUI, no font service)
TextClip(text=..., font_size=32, color="white",
         stroke_color="black", stroke_width=2, font=FONT_BOLD)
# captions/phase labels sit on a translucent black bar for legibility:
ColorClip(size=(W, 50), color=(0,0,0)).with_opacity(0.45)

# look: static film grain + vignette over the whole film
video = CompositeVideoClip([video, grain, vignette], size=(W, H))
# grain  = np.random.randint(0, 10, (H, W, 3)) @ opacity 0.05
# vignette = per-pixel darkened edges @ opacity 0.45

# audio: narration, padded with silence to the full length
if voiceover.duration < TOTAL:
    pad = AudioClip(lambda t: 0, duration=TOTAL - voiceover.duration)
    audio = CompositeAudioClip([voiceover, pad.with_start(voiceover.duration)])
video = video.with_audio(audio)

video.write_videofile(OUTPUT, fps=FPS, codec="libx264", audio_codec="aac",
                      preset="veryfast", bitrate="2500k", threads=4, logger=None)

Title card = darkened (brightness 0.30) first frame as backdrop + three staggered text lines.

Credits = CLARITY · UNITY · SAFER CARE over the closing scene. Both are just more overlays with

their own with_start/with_duration — there is no separate editing pass.

Step 7 — Run it

cd /workspace/calm-in-the-storm && python3 build_v2.py
# v1:  cd /workspace/calm-in-the-storm && python3 build_video.py

MoviePy prints progress bar-less output; the June sessions polled the output file while it ran

(sleep 60/120 && ls -lh ...) because the render takes minutes, not seconds.

Step 8 — Verify before shipping

from moviepy import VideoFileClip
v = VideoFileClip("calm-in-the-storm-v2-720p.mp4")
print(v.duration, v.size, v.fps, v.audio is not None)   # 121.85 (1280,720) 24.0 True

That check is why the surviving files are known-good: duration, resolution, fps and the presence

of an audio track.

Step 9 — Publish

The gallery and films are served from the VPS, not from a local path:

so browsers can scrub MP4s (python -m http.server returns the whole file and breaks seeking)

/docker/traefik/*.yml (no restart needed)

./pipeline/publish.sh          # dry run
./pipeline/publish.sh --go     # copy + verify 200 and HTTP 206 Range on every mp4

Verification must force IPv4 (curl -4) — this host resolves the subdomain AAAA-first and looks

dead otherwise.

5. Making a new video — the fast path

cd /workspace/nursing-videos/pipeline
cp scene_brief_template.md ../<project>/brief.md      # 1. brief: scenes + prompts + narration
python3 make_voiceover.py ../<project>/script.txt ../<project>/voiceover.mp3
                                                      # 2. narration FIRST, note the seconds
# 3. generate one still per scene from the brief's prompts → ../<project>/assets/
cp manifest.example.json ../<project>/manifest.json   # 4. timeline: starts, durations, zooms
python3 build_from_manifest.py ../<project>/manifest.json --dry-run   # 5. check the timeline
python3 build_from_manifest.py ../<project>/manifest.json             # 6. render + verify
./publish.sh --go                                     # 7. ship to the live URL

Everything the June build hard-coded is now data in the manifest: scene type, source, start,

duration, Ken Burns zoom, video trim, captions, phase labels, title card, credits, grain, vignette,

resolution, bitrate. A new film is a new manifest — no script editing.

6. Pitfalls learned the hard way

with FileNotFoundError.

The v1-era method names (set_duration, resize, fl_image) no longer exist.

GUI on the server. Resolve the path defensively.

whole toolchain; that is a feature, it is what makes it re-runnable.

otherwise the film either runs out of narration or cuts it off.

ratios mid-film. v2's vid_corridor.mp4 was the 480p one.

range_server.py exists.

image clip; do not compute it per frame.

write_videofile settings above; there is no GPU on this host.

7. Measured performance (this host, re-verified Sep 2026)

runoutputwall clock
kit smoke test (draft)14.0s · 1280x720 · 24fps · 3.1 MB77s (~5.5x realtime)
full v2 reconstruction, veryfast 2500k122.00s · 1280x720 · 24fps · 38.9 MB12 min 21 s (~6x realtime)

Budget for a new 2-minute film at 720p veryfast: ~12–15 minutes of CPU. 1080p roughly triples

that; ultrafast is about 5x faster and looks fine for review cuts.

8. Where everything is

/workspace/nursing-videos/
├── index.html                     live gallery (do not restyle casually)
├── how-it-was-made.html           this runbook as a styled page (live at /how-it-was-made.html)
├── HOW-THE-VIDEOS-WERE-MADE.md    this document — the source of truth
├── finals/                        the two surviving films
├── assets/                        all 21 source stills + clips
├── recovered/original_scripts/    every script the June build wrote, verbatim
│   └── workspace/{calm-in-the-storm,safety-huddle-video,nicu-video}/
├── _smoketest/                    kit verification
│   ├── manifest.json              14s draft (renders in 77s)
│   ├── finals/_smoketest.mp4
│   └── proof/calm-in-the-storm-v2-rebuilt-720p.mp4   122s full reconstruction
├── pipeline/                      the reusable kit (this runbook's subject)
│   ├── build_from_manifest.py     the renderer
│   ├── make_voiceover.py          Edge TTS + duration measurement
│   ├── build_doc_page.py          markdown → this page's HTML
│   ├── manifest.example.json      worked 11-scene / 122s timeline
│   ├── manifest.v2-reconstruction.json   the exact v2 rebuild
│   ├── scene_brief_template.md    the input format
│   ├── publish.sh                 ship + verify
│   └── README.md                  quickstart
└── range_server.py                Range-capable server (deployed to /opt/nursing-videos/)

Original build sessions (base profile history, ~/.hermes/state.db):

datesessionwhat happened
Jun 23 01:082a31996a5a44main build — chose the stack, generated the stills, wrote the MoviePy scripts, first renders, caption-free edit, rebuild with your files
Jun 23 04:420e5032b0f9dfdiversity pass — broadened the cast, built v2 from your clips + Gemini stills, 720p export
Jun 23 05:263965589d6027asset review — thumbnails of everything used
Jun 23 05:4020260623_054021_0aeb87NICU/L&D motion-clip request — 6 live-motion clips instead of Ken Burns
Jun 27 01:0465cf82e37511Canva-style prompt set for the 7 scenes

To regenerate a deleted film: pull its script out of messages.tool_calls in that session's

history, restore the assets named in it, run it. The scripts in recovered/ are already extracted.