AI Video Prompt Guide: Structure, Camera Controls & Workflow

Effective AI video prompting requires a structured syntax, not just descriptive adjectives. This guide defines a universal prompt framework—Subject, Action, Camera, Lighting, Style, Technical Parameters—that works across Sora, Runway Gen-3, Luma Dream Machine, Kling, and Pika. Apply this formula to reduce rerolls and achieve controllable, cinematic output.

The Universal Prompt Formula

Every high-quality prompt follows a fixed hierarchy. Models prioritize tokens earlier in the string, so order matters. Use this sequence:

1. Subject + Context: Who/what and where (e.g., "Astronaut in a bioluminescent forest").
2. Action + Motion: Specific movement verbs (e.g., "walks slowly, turns head left").
3. Camera + Lens: Shot type, movement, focal length (e.g., "Low angle, 35mm, slow dolly in").
4. Lighting + Atmosphere: Source, quality, mood (e.g., "Volumetric god rays, mist, golden hour").
5. Style + Aesthetic: Render engine, film stock, color grade (e.g., "Unreal Engine 5, Kodak Vision3 500T, teal-orange grade").
6. Technical Parameters: Aspect ratio, duration, fps, seed (e.g., "--ar 16:9 --duration 5 --fps 24").

Separate blocks with commas or pipe characters (|) for readability. Avoid natural language sentences; use dense, token-efficient keyword strings.

Camera Language: The Control Layer

Camera tokens dictate spatial relationship and viewer psychology. Generic terms like "cinematic" are low-signal. Use precise cinema vocabulary:

• Shot Scale: Extreme Close-up (ECU), Close-up (CU), Medium (MS), Wide (WS), Extreme Wide (EWS).
• Angle: Low angle (power), High angle (vulnerability), Dutch angle (tension), Eye-level (neutral).
• Movement: Static, Pan (horizontal), Tilt (vertical), Dolly/Track (physical move), Zoom (optical change), Crane/Jib (vertical lift), Handheld (organic shake).
• Lens/Focal Length: 14-24mm (wide, distortion), 35-50mm (natural), 85-135mm (portrait, compression), 200mm+ (telephoto, flattening).
• Depth of Field: Shallow (bokeh, subject isolation), Deep (everything sharp), Rack focus (transition).

Combine them: Low angle WS, 24mm, slow dolly left, deep focus. This forces the model to solve 3D geometry rather than hallucinate pixels.

Motion & Physics: Directing the Simulation

Current models (Gen-3, Kling 1.5, Sora) simulate physics, not just frame interpolation. Prompt for cause and effect, not just the result.

• Weak: "Woman running, hair flying."
• Strong: "Woman sprinting forward, ponytail whipping backward from wind resistance, feet kicking up mud splashes, heavy breathing posture."

Specify motion speed (slow motion, timelapse, real-time, high-speed) and motion type (linear, ease-in/out, erratic, physics-based). For complex choreography, break action into sequential beats: "Frame 1: Sits. Frame 2: Stands abruptly, chair scrapes floor. Frame 3: Walks to window." Some models accept keyframe conditioning (Start/End images) which supersedes text motion prompts.

Lighting & Atmosphere: Defining Form

Lighting tokens define material properties (specularity, subsurface scattering) and depth. Name the source, quality, and direction.

• Sources: Sunlight (hard, parallel), Overcast (soft, wrap), Practical (neon, candle, screen), Studio (Key/Fill/Rim), Bioluminescent.
• Quality: Hard (sharp shadows), Soft (diffused), Dappled (through foliage), Volumetric (god rays, fog interaction).
• Direction: Front (flat), Side (modeling), Back (rim/silhouette), Top (noon/drama), Bottom (horror).
• Atmosphere: Fog, haze, rain, snow, dust motes, smoke, heat haze. These trigger particle simulation.

Example: Side lighting, golden hour, volumetric haze, dust motes dancing, high contrast ratio 8:1.

Style Anchors: Locking the Look

Style tokens prevent the "AI aesthetic" (plastic skin, oversharpening). Anchor to real-world references:

• Film Stock: Kodak Vision3 250D/500T, Fuji Eterna 250, Portra 400 (stills look).
• Lens Brands: Arri Signature Prime (clean), Cooke S4/i (warm, flare), Anamorphic (horizontal flare, oval bokeh), Vintage Soviet (Helios swirl).
• Color Grading: Bleach bypass (desaturated, high contrast), Teal-Orange (blockbuster), Monochrome, Pastel (Wes Anderson), Log/Flat (for post grading).
• Render/Engine: Unreal Engine 5 (sharp, game look), Octane/Redshift (3D render), VHS/CCTV (lo-fi texture), Analog Video (scanlines, tracking error).

Stack 2-3 anchors: Kodak 500T, Cooke Anamorphic, Teal-Orange grade, film grain 30%.

Model-Specific Syntax & Parameters

While the core formula transfers, each model has unique parameter flags and token sensitivity.

Tip: Always use --seed (or fixed seed) when iterating on camera/lighting to isolate variable changes.

Practical Workflow: From Idea to Final Clip

1. Define the Shot List: Write 3-5 specific shots (WS, MS, CU) for a scene. Don't prompt "a movie."
2. Draft Master Prompt: Apply the Universal Formula to the Hero shot (usually the Wide/Master).
3. Generate Keyframes (Image-to-Video): Use Midjourney/Flux with identical style tokens to generate Start/End frames for critical shots. Upscale to 1080p/4K.
4. First Pass (Low Res/Fast Mode): Run prompts in Standard/Fast mode (Kling Std, Runway Turbo, Luma Fast). Check geometry, physics, adherence.
5. Iterate with Negatives: Add negative prompts: blur, low quality, jitter, morphing, extra fingers, watermark, text, logo, static camera.
6. Lock Camera/Seed: Once motion is correct, fix seed and camera tokens. Regenerate in High Quality/Pro mode.
7. Extend & Stitch: Use "Extend" (Gen-3, Kling) or "Re-cut" (Sora) for longer takes. Edit in NLE (DaVinci/CapCut), match grade.
8. Post-Process: Topaz Video AI (upscale, interpolate to 60fps, stabilize), Color Grade (LUTs), Sound Design (Foley, Score).

Common Failure Modes & Fixes

• Morphing/Warping: Increase guidance scale (CFG), use Start/End keyframes, simplify motion complexity, add "stable, coherent geometry" to prompt.
• Ignored Camera Move: Model may default to static. Prefix prompt with Camera: [Move Type] or use explicit ControlNet/Camera Control sliders (Runway, Kling).
• Janus Problem (Two Faces): Add "single subject, profile view only, one face" to negative prompt; use 3/4 angle instead of front.
• Text/Gibberish on Screen: Negative prompt: "text, letters, numbers, watermark, logo, signature, username".
• Flickering Lighting: Specify "consistent lighting, temporal stability, locked exposure". Avoid "dynamic lighting" unless intended.

Closing Takeaway

Prompting is not creative writing; it is technical direction. Treat the model as a virtual cinematographer that requires precise instructions: lens choice, lighting ratios, physics parameters, and render pipeline specs. Master the Universal Formula, build a personal token library for your aesthetic, and iterate systematically using seeds and keyframes. Consistency comes from constraint, not chaos.