AI Video Prompt Guide: Structure, Camera Controls & Workflow

Effective AI video prompting relies on a fixed syntax: Subject + Action + Camera + Lighting + Style + Technical Parameters. Models like Sora, Runway Gen-3, Luma Dream Machine, and Kling interpret this hierarchy differently, but the structural logic remains constant. This guide breaks down each component, explains model-specific weighting, and provides a reusable template for professional results.

Why Prompt Structure Determines Output Quality

Video models process tokens sequentially. Early tokens define the subject and core action; middle tokens control camera and environment; late tokens apply style and technical constraints. When you front-load aesthetic keywords ("cinematic," "8k") before defining motion, the model hallucinates detail on static frames. A structured prompt forces the model to allocate compute to temporal coherence first.

Runway Gen-3 responds strongly to camera tokens in positions 3-5. Kling 1.6 prioritizes physical motion verbs. Luma Dream Machine weights lighting and atmosphere heavily. Sora (where accessible) follows a strict subject-motion-camera hierarchy. Understanding these biases lets you reorder the same components for each model without rewriting the concept.

The Universal Prompt Formula

Use this template for every generation. Fill each bracket; delete brackets you don't need.

[Subject Description] + [Action/Motion Verb] + [Camera Type + Movement] + [Lighting/Time of Day] + [Environment/Background] + [Style/Aesthetic] + [Technical Specs]

1. Subject Description (Tokens 1-15)

Define the primary element with visual specificity. Avoid abstract nouns. Instead of "a woman," use "a 30-year-old woman with shoulder-length dark hair, wearing a navy trench coat." Include distinguishing features: clothing texture, color palette, props. For characters, specify age, ethnicity, and posture. For objects, state material, scale, and state (pristine, weathered, wet).

2. Action & Motion Verbs (Tokens 16-30)

Video is motion. Use precise, physical verbs: "strides," "lunges," "pans left," "rotates clockwise," "dissolves into particles." Avoid "moves," "goes," "walks." Specify speed: "slow-motion," "real-time," "time-lapse." Define trajectory: "walks toward camera," "orbits subject at 2m radius." For physics-based motion, add "natural weight," "momentum preserved," "cloth simulation."

3. Camera Type & Movement (Tokens 31-50)

This is the strongest differentiator between amateur and professional output. Declare lens, angle, and movement in one clause.

• Lens: 16mm (wide, distortion), 24mm (environmental), 35mm (standard), 50mm (natural), 85mm (portrait, compression), 200mm (telephoto, flattening).
• Angle: eye-level, low-angle (heroic), high-angle (vulnerable), Dutch angle (tension), bird's-eye, worm's-eye.
• Movement: static, pan left/right, tilt up/down, dolly in/out, truck left/right, pedestal up/down, orbit clockwise/counterclockwise, handheld (micro-jitter), drone rise/reveal, FPV dive.
• Focus: deep focus, shallow focus (f/1.8), rack focus from [object A] to [object B].

Example: "35mm lens, eye-level, slow dolly in, shallow focus rack from foreground hands to face."

4. Lighting & Time of Day (Tokens 51-70)

Lighting defines mood and material response. Specify source, quality, direction, and color temperature.

• Source: golden hour sun, overcast sky, neon sign, practical lamp, LED panel, candlelight, bioluminescence.
• Quality: hard (sharp shadows), soft (diffused), dappled (through foliage), volumetric (god rays), rim/backlight.
• Direction: front, side (split), 45-degree (Rembrandt), top-down, bottom-up.
• Color: 5600K (daylight), 3200K (tungsten), teal/orange contrast, monochrome, specific gel (CTB, CTO).

5. Environment & Background (Tokens 71-85)

Ground the subject. "Blurred city street" is weak. "Rain-slicked Tokyo alley at night, neon reflections on wet asphalt, distant steam vents" gives the model spatial anchors. Include depth cues: foreground elements, mid-ground action, background landmarks. Specify weather: fog density, rain intensity, snow accumulation, wind direction on vegetation.

6. Style & Aesthetic (Tokens 86-100)

Apply look last. Reference film stocks (Kodak Vision3 500T, Fuji Eterna 250), directors (Wong Kar-wai color grading, Villeneuve scale), or genres (neo-noir, cyberpunk, minimalist documentary). Avoid stacking contradictory styles. One strong reference beats five weak ones.

7. Technical Specifications (Tokens 101+)

Model-specific controls. Resolution: 1280x720, 1920x1080. Aspect ratio: 16:9, 9:16, 2.39:1, 4:3. Frame rate: 24fps (cinematic), 30fps (broadcast), 60fps (smooth motion). Duration: 5s, 10s. Seed: fixed for consistency. Negative prompt: "blur, distortion, morphing, extra limbs, watermark, text, low quality."

Model-Specific Prompt Weighting

Identical prompts produce different results across models. Adjust component emphasis per platform.

Practical takeaway: For Runway, lead with camera. For Kling, lead with motion physics. For Luma, lead with lighting mood. For Pika, lead with style reference. For Hailuo, balance subject and motion equally.

Camera Language Cheat Sheet

Copy these clauses directly into prompts. They translate reliably across models.

Common Failure Modes & Fixes

1. Morphing limbs / extra fingers: Add "anatomically correct, five fingers, stable topology" to negative prompt. Reduce motion speed. Use image-to-video with a clean start frame.
2. Camera drift (unintended zoom/pan): Explicitly state "static camera" or "locked off." Avoid "smooth" without a defined movement vector.
3. Flickering lighting: Specify "consistent lighting, no exposure shift." Lock time of day. Avoid "dynamic lighting" unless intentional.
4. Background warping: Define background geometry: "static background, parallax only from camera movement." Use simpler environments for long durations.
5. Style bleeding (unwanted color grade): Move style tokens to the end. Use negative prompt: "no color grading, neutral white balance." Generate raw, grade in post.
6. Temporal inconsistency (character changes between frames): Use image-to-video with a consistent reference frame. Add "temporal consistency, identity preservation" to prompt. Keep clips under 5s for character work.

Step-by-Step Prompt Building Workflow

1. Define the shot goal in one sentence. Example: "A chef chops vegetables in a sunlit kitchen, close-up on knife skill."
2. Extract subject tokens. "Professional chef, 40s, focused expression, white chef coat, santoku knife, carrots and onions on wooden board."
3. Select the primary motion verb. "Rhythmically chops, precise downward cuts, onion pieces accumulate."
4. Choose camera clause from cheat sheet. "100mm macro, eye-level, static, shallow focus on blade contact point."
5. Set lighting. "Morning window light, 45-degree side, soft shadows, steam rising from vegetables."
6. Add environment depth. "Blurred stainless steel counter behind, herb pots on windowsill, out-of-focus pendant lights."
7. Apply style. "Food documentary aesthetic, natural color, high dynamic range."
8. Append technical specs. "1920x1080, 16:9, 24fps, 5s, seed 42."
9. Assemble full prompt. Combine in formula order. Count tokens; trim to model limit.
10. Generate 3 variations. Vary seed only. Select best temporal coherence. Upscale if needed.

Advanced: Image-to-Video Prompting

When using an input image, the prompt describes change, not the static scene. The model already sees subject, lighting, and composition. Your prompt only needs: Motion vector + Camera movement + Duration cue.

Example: Input image shows a parked car at sunset. Prompt: "Car accelerates forward, wheels spin, dust kicks up, camera low-angle truck left following, golden hour maintained, 5 seconds." Do not re-describe the car or sunset. Wasted tokens confuse the motion planner.

For character animation: "Head turns left, eyes track moving object, subtle smile forms, shoulders relax, breathing motion, 3 seconds." Micro-expressions require explicit enumeration; models default to frozen faces.

Closing Takeaway

Prompting is not creative writing—it is parameter engineering. The formula [Subject][Motion][Camera][Light][Environment][Style][Tech] works because it mirrors the model's latent space hierarchy. Memorize the camera cheat sheet. Learn your target model's weighting bias from the comparison table. Build every prompt through the 10-step workflow. Consistency comes from structure, not inspiration.