How to Write Prompts for AI Video Models

Most AI video prompts fail for a simple reason: they describe a picture when the model needs instructions for a shot. A prompt such as "A woman in a red coat in a busy market" establishes the subject and setting but gives the model little direction about what should happen over the next eight seconds. The model must improvise the movement, camera behaviour, and timing, often producing slow, generic motion or unnecessary camera movements. Better results come from approaching prompts the way a director briefs a camera crew: define the subject, break the action into clear beats, specify the camera movement, and establish the lighting and visual style. This guide introduces a five-part template for writing effective AI video prompts, explains how to adapt it for image-to-video generation, and outlines a practical iteration process that helps you improve results without wasting your generation budget.

What Makes AI Video Prompts Different

AI video prompts must describe how a scene changes over time, whereas image prompts primarily describe a single moment. This introduces three additional requirements: what moves, how the camera behaves, and the sequence in which events occur. Without these instructions, the model has to make its own decisions about motion and timing, often resulting in vague actions, unnecessary camera movement, or scenes that lack a clear beginning and ending. A strong video prompt therefore describes not only what appears in the frame but also how the scene develops throughout the shot.

Another important distinction is that some video characteristics should be configured through API parameters rather than described in the prompt. One major provider's documentation explicitly distinguishes between the settings that define the video itself, such as duration, resolution, and character references, and the instructions that determine what happens within the scene. Writing "make it longer" in a prompt, for example, generally won't override the configured duration. Setting these parameters separately gives you more predictable control over the output and makes prompts easier to reuse across different models.

On AVIS, these settings are available as separate fields on the video generation endpoint, including duration, resolution, ratio, seed, and cameraFixed. Image inputs can also be assigned specific roles, such as firstFrame, lastFrame, or referenceImage, depending on the model. Because supported values and parameter combinations vary across models, check the model list endpoint before implementing your integration rather than hardcoding the limits of a single model.

A Five-Part Template for AI Video Prompts

A reliable AI video prompt describes five elements: the subject, the action broken into beats, the setting and lighting, the camera, and the visual style. These elements appear consistently across the prompting guides published by major providers, making the template a useful starting point even when you switch between models. By keeping each element explicit, you give the model enough direction to produce a coherent shot without overloading the prompt with unnecessary detail.

The Template

Write the prompt as a sequence of plain descriptive sentences, following this order:

1. Subject. Identify who or what appears on screen and specify two or three defining characteristics that should remain consistent throughout the shot. For example: "A woman in her thirties wearing a long red wool coat, with dark hair tied back." These details establish the subject's appearance without introducing unnecessary complexity.
2. Action, in beats. Break the movement into small, observable steps that follow a clear sequence. Instead of writing "walks across the room," describe the action more precisely: "Takes four steps toward the window, pauses, and pulls the curtain open in the final second." This gives the model a sequence of events to follow rather than a vague action to extend across the entire clip.
3. Setting and lighting. Establish the location, time of day, primary light source, and a small set of colours that define the scene's visual palette. For example: "A small apartment at dusk, with soft window light and a warm lamp in the corner; a palette of amber, cream, and walnut brown." These details help maintain a consistent atmosphere throughout the shot.
4. Camera. Specify the framing first, followed by the camera movement. For example: "Medium shot at eye level, with a slow dolly in." If the camera should remain stationary, explicitly describe a static shot or use a fixed-camera setting when the model supports it. Clear camera instructions reduce the chance of unwanted movement.
5. Style. Describe the desired visual texture and rendering characteristics using concrete terms, such as "shot on 35mm film, shallow depth of field, natural grain." Describing the visual qualities directly makes the prompt easier to transfer between models than relying on the name of a particular film or director, and it avoids unnecessary dependence on specific creative references.

The following camera vocabulary provides a useful starting point. Most video models can interpret common filmmaking terms, although the accuracy of the results varies by model.

Basic framing and camera movements are generally more predictable than unusual angles or complex lens effects. When experimenting with less familiar techniques, introduce one at a time and evaluate the result before combining them with other instructions. This makes it easier to identify which elements the model follows reliably and which need further refinement.

Weak vs. Strong: The Same Idea, Rewritten

Consider the following prompt:

Weak: "A woman in a red coat in a busy market, cinematic."

This describes a subject, a location, and a broad aesthetic, but it leaves almost every production decision to the model. There is no clear action, no defined camera behaviour, and no indication of how the shot should develop over time.

Strong: "A woman in her thirties wearing a long red wool coat walks briskly through a crowded street market, weaving between two vegetable stalls before stopping and turning her head toward the camera in the final second. Late-afternoon golden sunlight comes from the left, casting long shadows across the street; the colour palette combines red, ochre, and slate grey. A low-angle tracking shot moves alongside her at walking pace. Shot on 35mm film, with shallow depth of field."

The stronger prompt gives the model a specific sequence of actions, a defined ending, a consistent lighting direction, a limited colour palette, and an explicit camera movement. Instead of asking the model to invent an entire shot from a general concept, you provide a clear visual plan while leaving enough creative freedom for it to generate the scene.

How Long Should an AI Video Prompt Be?

There is no universal word limit, but approximately 40–100 words is a useful starting point for a single shot. A five-word prompt leaves the model to determine nearly everything about the movement, camera, and timing, while a 300-word prompt may introduce unnecessary detail or conflicting instructions. The right level of detail depends on your objective: product demonstrations, branded content, and repeatable visual sequences generally benefit from more explicit instructions, whereas creative exploration often works better with shorter prompts that give the model room to experiment. Start with the essential elements, evaluate the output, and add detail only when it addresses a specific problem.

Adjusting the Prompt for Image-to-Video, Audio, and Exclusions

The five-part template is a useful foundation, but not every video-generation task requires all five elements. Image-to-video generation already receives visual information from an input image, audio generation introduces another layer of instructions, and exclusion requirements need to be expressed carefully to avoid reinforcing the very elements you want to remove. Adjusting your prompt to match the generation method helps reduce ambiguity and makes better use of the information the model already has.

Image-to-Video: Describe the Motion, Not the Picture

When you provide a first frame, the image already establishes much of the scene's visual information, including the subject, composition, colour palette, lighting, and style. Your prompt should therefore focus primarily on motion: what changes, how the subject moves, and what the camera does. Repeating details that are already clearly visible in the input image may add little value, and an overly elaborate scene description can distract from the movement you actually want the model to generate.

For example, a text-to-video prompt might describe the complete scene: "A ceramic coffee cup sits on a wooden table beside a rainy window, with steam rising from the cup under soft grey morning light. Slow push in." If you use the same image as the starting frame for image-to-video generation, a more focused prompt would be: "Steam rises and curls from the cup while rain streaks down the window behind it. Slow push in toward the cup." The second prompt concentrates on the changes that should occur instead of describing the existing composition again.

If the model supports both a first frame and a last frame, your prompt should describe the transition between those two visual states. For example: "The camera arcs left around the table as the lighting gradually shifts from cool grey to warm golden tones." The two images establish the endpoints, while the prompt guides the movement between them. Reference images serve a different purpose: they help define the appearance of a character or object across multiple shots, making them useful when you need to maintain visual consistency throughout a longer sequence.

Audio and Dialogue

When using a model that generates sound, describe audio explicitly rather than assuming the model will infer it from the visual scene. Separate the audio instructions from the visual description so that the two are easy to distinguish, and specify the ambient sounds, individual sound effects, and dialogue that should be present. For dialogue, place spoken lines in a separate block at the end of the prompt and use consistent speaker labels throughout. Keep the instructions appropriate for the clip's duration: a four-second shot, for example, will generally accommodate only one or two brief exchanges without making the scene feel rushed.

Audio requirements also need to match the model's actual capabilities. Some models generate synchronized audio and video, while others generate silent video or support only particular audio features. Check the model's documentation before designing a prompt format around dialogue, sound effects, or synchronization requirements.

Say What You Want, Not What You Don't

Negative instructions can be surprisingly unreliable in video generation. Writing "no people in the background" may cause the model to focus on the concept of people rather than reliably excluding them. Guidance from major providers generally favours describing the desired result positively, such as "an empty street with no visible pedestrians" or, more simply, "an empty street." When a model provides a dedicated negative-prompt field, use that field to specify unwanted elements as concise terms, such as "people, text, watermark," rather than relying entirely on exclusion instructions within the main prompt.

The same principle applies to camera instructions. Direct descriptions such as "The camera slowly pans left" are usually clearer than conversational requests such as "Please pan the camera to the left." A video prompt works best when it describes the intended visual result directly, using specific language that the model can translate into movement and composition.

How to Iterate on Video Prompts and Build Them into an App

Video generation takes longer and typically costs more per attempt than text generation, so a disciplined iteration process can save both time and budget. Rather than rewriting the entire prompt whenever a result falls short, treat each generation as an experiment designed to test a specific change. This makes it easier to understand how individual instructions affect the output and helps you build a repeatable process for improving results.

Use the following checklist when a generated video is close to what you want but still needs refinement:

1. Change one thing at a time. Make a single targeted adjustment, such as "same shot, but a slower dolly" or "same action, switch to a wide shot." Changing several elements simultaneously makes it difficult to determine which modification improved or worsened the result.
2. Keep the seed fixed while tuning, where supported. A fixed seed can help reduce variation between generations, allowing you to evaluate the effect of prompt changes more consistently. Once the prompt performs as expected, release the seed when you want to explore alternative results. Keep in mind that a fixed seed does not guarantee identical outputs across all models, settings, or versions.
3. Shorten the clip before lengthening the prompt. Longer clips can be harder to keep visually consistent because the model must maintain the subject, environment, and motion over more frames. One provider's guidance suggests that generating two four-second clips and stitching them together can produce better results than generating a single eight-second clip. Visual drift can also become more noticeable in longer generations, so plan complex sequences as several shorter shots rather than expecting one prompt to control an entire scene perfectly.
4. Check the settings before rewriting the prompt. If the output has the wrong duration, aspect ratio, or resolution, adjust the corresponding API parameters rather than trying to correct the problem through natural-language instructions. Separating configuration issues from prompt issues makes troubleshooting faster and more reliable.
5. Expect variation between generations. Even with an identical prompt, repeated generations can produce different results. When the cost allows, generate two or three candidates and compare them against your requirements instead of rewriting the prompt after every imperfect result. This is particularly useful when the overall composition is correct but small differences in movement or timing affect the final quality.

Turning the Template into Code

If your application allows users to enter free-form ideas, sending their text directly to the video model may produce inconsistent results. A more structured approach is to represent the five prompt elements as separate fields, populate them from user input, and assemble the final prompt in a predictable order. Alternatively, you can use a text model to expand a short user idea into the five-part structure before submitting it to the video-generation API. This makes it easier to maintain consistency, test individual prompt components, and enforce application-specific rules, such as removing unsupported brand references or rewriting ambiguous negative instructions.

For example, a simple Python function can combine the five elements into a single prompt:

def build_video_prompt(subject, beats, setting, camera, style):
    return " ".join([f"{subject} {beats}", setting, camera, style])

In a production application, you may want to add validation for missing fields, normalize whitespace, and handle optional elements such as audio instructions or negative prompts. You should also keep duration, aspect ratio, resolution, and seed in the API request body, sourcing them from application configuration or user settings instead of trying to extract them from the prompt text. Separating prompt content from generation parameters makes your implementation easier to test and adapt when you introduce additional video models.

Keeping Iteration Costs Under Control

Before submitting a generation request, estimate its cost whenever the API supports it. On AVIS, the video estimation endpoint accepts the same request body as the generation endpoint and returns an indicative cost without charging for the estimation itself. According to our API pricing page, system-side failures are also not billed. These capabilities make it easier to experiment while keeping spending predictable. During prompt development, use shorter durations and lower resolutions where appropriate, then render the final version using the intended production settings once the prompt produces satisfactory results.

If you prefer a visual workflow, the Video Node in AVIS Creative Studio brings the prompt, starting frame, reference images, and generation settings together in one interface. It also displays the estimated cost before each generation, making it easier to compare variations and refine a prompt without losing track of the associated expense.

FAQ

How long should an AI video prompt be?

Approximately 40–100 words is a reasonable starting point for a single shot, although the ideal length depends on the complexity of the scene and the model you use. Very short prompts leave important decisions about movement, camera behaviour, and timing to the model, while excessively long prompts can introduce unnecessary detail or conflicting instructions. Use more detail when consistency and precise control matter, and keep prompts lighter when you want to explore creative possibilities.

What should an AI video prompt include?

A strong prompt covers five elements: the subject, the action broken into clear beats, the setting and lighting, the camera framing and movement, and the visual style. These details tell the model what should appear on screen and how the shot should develop over time. Parameters such as duration, resolution, aspect ratio, and seed should generally be configured through the API rather than described in the prompt, since natural-language instructions typically cannot override those settings.

How do I write a prompt for image-to-video generation?

Focus on movement rather than describing the entire scene. The input image already establishes the subject, composition, colours, and lighting, so your prompt should explain what changes, how the subject moves, and what the camera does. Repeating the existing visual details may add little value and can distract from the intended motion. If you provide both a first frame and a last frame, describe the transition between them.

How do I write negative prompts for video?

Describe the result you want rather than relying exclusively on instructions about what the model should avoid. For example, use "an empty street" instead of "no people in the background." If the model supports a separate negative-prompt field, specify unwanted elements there as concise terms, such as "text, watermark, people." The exact behaviour depends on the model, so test negative prompts against representative examples rather than assuming they will always be interpreted literally.

Why does my AI-generated video change halfway through?

Visual consistency can weaken as clips get longer, causing subjects, faces, backgrounds, or other details to drift over time. Shorter clips are often easier to control because the model has fewer frames over which to maintain consistency. For longer sequences, consider generating several shorter shots, using reference images to maintain the appearance of recurring subjects, and carrying the last frame of one shot into the next when the model supports that workflow.

Can I prompt for audio and dialogue?

Yes, provided the selected model supports audio generation. Describe ambient sounds, sound effects, and dialogue explicitly, keeping audio instructions separate from the visual description. Put dialogue in a separate block with consistent speaker labels, and keep spoken lines short enough to fit naturally within the clip. Check the model's supported capabilities before relying on synchronized dialogue or specific sound-generation features.

Wrapping Up

Effective AI video prompts read more like a director's shot list than a description of a still image. Define the subject, break the action into clear beats, establish the setting and lighting, specify the camera movement, and describe the visual style. Keep duration, resolution, aspect ratio, and seed in the generation parameters rather than the prompt itself. For image-to-video generation, concentrate on the motion instead of repeating details already established by the input image, and refine results by changing one variable at a time on short, inexpensive clips.

For teams building AI video applications, a structured prompt template and a clear separation between prompt content and API parameters make generation more consistent, easier to test, and less expensive to optimize. AVIS provides a unified video generation endpoint across supported models, along with cost estimation before generation. This lets you experiment with different video models and prompting strategies while keeping integrations and cost management in one place.

Last updated: October 2026