Storyboarding with Midjourney
AI tools have significantly streamlined the storyboarding process, allowing creators to iterate quickly through different visual concepts without the need for extensive manual sketching. This technological advancement not only saves time but also enables experimentation with various styles, perspectives, and narrative approaches at the click of a button.
There are various AI tools available today that can simplify the storyboarding process. These tools can be divided into 2 categories:
Text-to-image tools
With text-to-image tools, you can easily create images based on text descriptions that you can use for making storyboards. You can then combine and animate these in another programs. These tools offer several advantages for storyboarding: they provide flexibility with complete control over visual style and experimentation with different looks, enable quick iterations by rapidly generating different versions of scenes, and offer creative freedom through extensive prompting options that let you specify everything from camera angles to specific emotions and atmospheres.
AI-Storyboarding tools
These are AI tools specifically developed for creating storyboards. You can often upload your script and instantly receive a storyboard based on it. Using camera movements, you can then create an animated storyboard. These tools are usually credit-based, so costs can add up quickly (especially if you already have a subscription to a text-to-image tool). Additionally, they often offer less freedom during prompting, and the resulting images tend to be rather generic. However, for those who prefer an all-in-one solution with integrated storyboarding features, these tools can be an excellent choice. For those who want to invest time in creating visually appealing storyboards, it’s better to opt for a workflow using text-to-image tools, which currently offer more prompting possibilities.
Discover our overview of AI tools
Among the various AI-powered image generation tools available today, we chose the text-to-image tool Midjourney for this tutorial about creating storyboards. Its ability to maintain consistency in characters and style makes it particularly valuable for visual storytelling.
Remember that you are still working with an AI tool where the results heavily depend on your input and prompting skills. While this part of the guide won't focus on detailed prompting techniques (like getting specific poses or perfecting single images), we'll concentrate on maintaining character and style consistency across multiple images to effectively tell your visual story. If you want to learn more about specific prompting techniques, check out our Midjourney prompt guide for beginners, which covers these aspects in detail.
A well-crafted prompt for storyboarding consists of these key elements:
- style: define the visual appearance and artistic direction of your storyboard
- camera angle/postion: specify the perspective and framing of each shot
- subject/action: describe what’s happening in the scene, including characters, locations, and activities
- parameters: include technical specifications like aspect ratio, style references, character references, and negative prompts
storyboard sketch, black and white, close-up shot, center-view, a happy polar bear, smiling, buildings in the background –ar 16:9 –sref
Style,
Camera angle/position
Subject/action
Parameters (In Discord, add after the prompt. In Web UI, adjust via Settings)
When creating storyboards, the main challenges lie in maintaining consistent style across all images and ensuring characters and environments remain uniform throughout. While keeping prompt descriptions consistent (using the same character descriptions, environment details, and style) is essential, you’ll quickly discover this alone isn’t sufficient.
Parameters play a crucial role in achieving consistent storyboards. In this section, we’ll explore all parameters that influence image consistency across your storyboard. Keep in mind that you don’t need to use all parameters at once – experiment to find which combination works best for your specific needs. Some parameters might work better than others depending on your desired outcome.
Remember that achieving 100% consistency with AI is still challenging. However, you can always refine details in Photoshop afterward to achieve better results.
For a comprehensive guide to all available parameters and their latest updates, please consult the official Midjourney documentation.
-> Midjourney Documentation
Style Reference
Style Reference
The style reference parameter allows you to match the visual characteristics of an existing image in your new Midjourney creations. It captures and applies elements like colors, medium, textures, and lighting from your reference image, helping maintain a consistent visual theme across your storyboard. While this parameter is excellent for maintaining visual consistency, note that it doesn’t ensure character consistency – it only replicates the artistic style, not specific objects or people.
Best Practices from Midjourney Style Reference
- Keep text prompts simple – Avoid adding style words that might conflict with your reference image’s look.
- Add style words selectively – If achieving a specific style is difficult, include descriptive words that match your reference image.
- Focus on content, not instructions – Use your text prompt to describe what you want to see, not how Midjourney should modify the reference image.
Bad Prompt Examples:
- the look of this image but a dog
- copy this style and make a bunny
Good Prompt Examples:
- detailed portrait of a dog
- ballpoint pen sketch of a bunny
storyboard sketch, black and white, medium shot, center of picture
a polar bear looking at the fish in the supermarket, he walks between 2 aisles, fish department–ar 16:9
storyboard sketch, black and white full body shot, center of picture
a full-body portet polar bear standing up, he is inside a refrigerator, door closed, lot’s of frozen food behind him, in the frozen section of the supermarket –ar 16:9
Use other images from your storyboard as style references. If the style doesn’t match perfectly, try using a different reference image or combine multiple style references to achieve the desired look.
Style Weight --sw
Style Weight
The style weight parameter --sw controls the intensity of your reference image’s style in the generated image. You can adjust this value between 0 and 1000, with --sw 100 being the default setting. Add this parameter at the end of your prompt in Discord.
storyboard sketch, black and white, medium shot, center of picture
a polar bear looking at the fish in the supermarket, he walks between 2 aisles, fish department–ar 16:9
Character Reference --cref
Character reference
To maintain character consistency across multiple images, you can use character references. You can upload these references through the image icon in the web version, or use the --cref parameter at the end of your prompt followed by the image URL. For multiple reference images, simply separate the URLs with spaces.
Character references help Midjourney understand and replicate specific features like hair color, clothing style, and facial characteristics, ensuring your character remains consistent throughout different scenes in your storyboard.
Best Practices from Midjourney Character Reference
- Use Midjourney Images: For best results, start with an image of a single character created by Midjourney. Images of real people typically won’t look exactly like them.
- Limit Character References: While you can use more than one image of the same character, it’s often not necessary.
- Account for Details: Be aware that intricate details like specific freckles or logos on clothing might not come out exactly right.
- Use Detailed Text Prompts: Combine your character reference with a clear text prompt. Text is just as important for conveying the full scene and additional details beyond what the reference image shows.
Bad Prompt Examples:
- put this man in a cafe
- add this man to a cafe setting
Good Prompt Examples:
- a man with blue hair and gold glasses sitting in a cafe
- illustration of a man sitting alone in a cafe
Storyboard, cinematic, close-up shot, center-view
a a girl with pink hair and make-up, she is drinking a smoothie, a forest in the background–ar 16:9
Be aware that 100% consistency is not possible when using character references. However, you can significantly improve character consistency by providing detailed descriptions of your character’s appearance. Not sure how to describe your character effectively? Use Midjourney’s describe function. By uploading a photo of your character, you can see which descriptive elements Midjourney would include in the prompt. This method can also be applied when determining your preferred style.
-> Midjourney Describe
Character Weight --cw
Character weight
The character weight parameter controls how closely the generated image follows your reference image. Add --cw to your prompt in Discord to adjust this. At the default setting of --cw 100, Midjourney incorporates comprehensive details including facial features, hairstyle, and clothing. Lowering it to --cw 0 makes the AI focus primarily on facial characteristics. This parameter gives you control over how much information is extracted from your reference image to create consistent characters across your storyboard.
Even when setting the weight parameter to 0, adding a style or character reference will still have a subtle influence on the generated image. As shown in the images below, at cw 0, certain elements like hair patterns and style still maintain a connection to the reference image.
Image prompts
Image prompts
Before character and style references were introduced, image prompts were the only way to customize your style. Image prompts analyze both style and character elements. By including an image alongside your text prompt, you can guide Midjourney’s creative process. The AI examines the essential elements of your reference image and uses them as inspiration for generating new, unique images. While style and character prompts now offer more precise control over these aspects, image prompts remain particularly useful for replicating specific compositions, while also incorporating both character and style elements from the reference image. It’s important to note that Midjourney uses these references as inspiration rather than attempting to create exact copies.
Using image reference
On the web interface, upload your image and choose the appropriate icon to specify whether you want to use it as a character reference, style reference, or image prompt.
Using image reference
For Discord users, simply paste the image URL at the start of your text prompt. To use multiple images, add their URLs separated by spaces.
cinematic, wide-angle shot
a enchanted forest with mirrors between the trees –ar 16:9
Even when setting the weight parameter to 0, adding a style or character reference will still have a subtle influence on the generated image. As shown in the images below, at cw 0, certain elements like hair patterns and style still maintain a connection to the reference image.
Image prompts VS Character reference
Storyboard, cinematic close-up shot, center-view,
a girl with pink hair and make-up, she is drinking a smoothie, a forest in the background –ar 16:9
Common issues
When using style and character references, sometimes the camera angle from the reference image is unintentionally carried over to the generated image. In the example below, the reference image was taken from above, while we wanted a different camera angle for our original photo.
simple storyboard sketch, black and white full body shot, center-view,
a polar bear sitting down, holding one hand before his eye, a few flowers lying around him, the northpole, sad, tired –ar 16:9 –sref
There are several solutions to address this issue:
- Add the desired camera angle explicitly in your prompt.
- Include a Image prompt that shows the correct viewing angle.
- Add an additional style reference (with an other point of view).
- First generate your image without any references to get the desired camera angle. Once you’re satisfied with the perspective, add your reference images to refine the style and character details.
These tips are also helpful when you’re having trouble achieving a specific camera angle. In the example below, the camera angle from the style reference became dominant, preventing us from getting a panoramic shot. To solve this, we added a reference image of our desired view (if you can’t find one online, create it in Midjourney by removing the style reference). Then, we added an extra style reference to emphasize the drawing style.
simple storyboard sketch, black and white panoramic shot, right in picture,polar bear walking with suitcase on back, facing left, north pole, sunny day –no people –ar 16:9
When using character references, achieving the desired framing (e.g., full body shot) can be challenging, even when specified in your prompt. This is particularly true when your reference image is a close-up and you want a full body shot. The main factor affecting this is the aspect ratio – portrait orientation tends to work better for full-body shots, while landscape orientation is more suitable for close-ups. If you encounter this issue, try changing the aspect ratio. After change the ratio, you can also use Midjourney’s zoom or expand features to generate additional content on both sides to achieve the desired format, or reframe your images in another program.
cinematic a full body shot,of a boy in the park eating ice cream
