The guide · paste into Claude Code

Swap your face into AI characters.

Drop your selfie video into Claude Code with this prompt. It asks four quick questions (which words trigger the swap, which characters and voices, how to pace the cut), then runs the full pipeline through the Higgsfield MCP. Nano Banana Pro handles the face and outfit swap, Kling 3.0 the talking-head with voice. Your real background stays.

Paste into Claude Code

🔒 Drop your email to reveal the prompt

Get the full prompt, free.

Drop your email and the prompt unlocks instantly, plus every other guide and resource in the library. One email a week, unsubscribe anytime.

Task: Swap my face and outfit into AI-generated characters at specific moments in this selfie video — while keeping my real background, lighting, and audio timing locked.

I will drag the video into this chat. Use ONLY the Higgsfield MCP for all image and video generation. No ElevenLabs, no CapCut, no editor.

Before generating anything, ask me these four questions:

1. Which word(s) in my speech should trigger each avatar swap?
2. For each swap, describe the character — visual (look, outfit, era, vibe) AND voice (accent, gender, tone).
3. Should the avatar say the trigger word themselves, OR appear silently for a beat and then deliver their line, OR jump in mid-sentence?
4. Final aspect ratio (9:16 / 16:9 / 1:1)?

After I answer, run this pipeline end-to-end:

Step 1 — Whisper API (word-level transcription) on the audio to locate every candidate swap moment.

Step 2 — ffmpeg silencedetect at -30dB to find the AUDIBLE end of each spoken word. Use silencedetect for cut points, not Whisper — Whisper marks words ~0.7s longer than they actually sound (the trailing "ssss" of a "thiss" falls below detection but Whisper still counts it). Cutting on Whisper alone gives you dead air at every transition.

Step 3 — ffmpeg split the source at the audible cut points.

Step 4 — For each swap segment, extract the first frame and run it through Nano Banana Pro (model: nano_banana_2, resolution: 2k, aspect: 9:16). Prompt format:

"Edit this selfie image: replace the face and shirt ONLY with [CHARACTER DESCRIPTION]. Keep EVERYTHING else exactly the same — the ceiling, walls, picture frame, lighting direction, camera angle, head position, shoulder line, framing. The new person must occupy the exact same pixel region as the original. Match the original lighting direction and shadows. Photorealistic, same phone-selfie grain as the source."

This locks the background pixels so the swap feels seamless.

Step 5 — Send each Nano Banana output to Kling 3.0 (model: kling3_0, mode: pro, sound: on, duration: 3 or 4 seconds) with the swapped frame as start_image. Kling generates voice + lip-sync inline from the prompt — no separate audio file needed. Prompt structure (lead with the dialogue at the TOP so the auto-enhancer doesn't drown it):

"DIALOGUE (the EXACT words spoken aloud, nothing else): \"[line]\"

The character speaks ONLY these words and nothing else. [optional pacing notes]. The dialogue ENDS on the word \"[last word]\". He/she holds [expression] in silence for the remainder of the clip with subtle natural head movement.

NO improvisation. NO additional words.

Scene: [character visual]. Background fully static: [enumerate the room].

Voice: [accent + tone].

Style: photorealistic smartphone selfie aesthetic. No subtitles, captions, overlays, text, watermarks, or burnt-in letters of any kind."

Do NOT use Veo 3 or Veo 3.1 for this. Veo's auto-enhancer pads short dialogue with hallucinated filler ("this, or" turns into "business or have free events where we had to shoot work") and burns flickering caption glyphs even when told not to. Kling 3.0 pro is the right model.

Step 6 — Normalize every clip to 1080×1920, 24fps, AAC 48kHz stereo before concat. Kling already outputs 1080×1920 24fps; only re-encode the kept original segments to match. If you trim a Kling clip, also strip any silent intro at the start (use silencedetect to find it) so the character speaks the instant they appear.

Step 7 — ffmpeg concat → save to ~/Downloads/[original-filename]_swapped.mp4. Open it for me when done.

Cost discipline: preflight every Higgsfield job with get_cost: true and tell me the total before submitting. A two-swap Reel typically runs ~30-50 credits on Higgsfield Ultra (~$1-2). Don't blow credits without confirmation.

Common pitfalls (already learned the hard way — don't repeat them):
- Cutting on Whisper timestamps instead of silencedetect → dead air at every transition.
- Using Veo for the talking-head step → hallucinated dialogue and caption-burn glitches.
- Searching for a programmatic Higgsfield Audio / Speak 2.0 endpoint → it doesn't exist, those products are web-UI only. Kling generates the voice inline from the prompt.
- Leaving Kling's natural ~0.3s silent intro in the final cut → the character feels late. Trim it.
- Forgetting to re-encode the original segments to match Kling's specs before concat → DTS warnings and audio drift.

Confirm you've received the video, then ask me the four questions and wait for my answers.