By Dave DeFusco
For many people, today鈥檚 image-generating AI tools feel almost magical. You type a sentence鈥斺渁 dog running in a park鈥濃攁nd within seconds, a lifelike image appears. If you ask for a sequence of images that show what happens next, however, things fall apart. The dog may suddenly change shape, color or even species from one image to the next. The story doesn鈥檛 hold together.
That challenge is at the heart of a research paper, 鈥淐ausalDreamer: Temporal Image Generation through Causal Intervention Attention with Identity Preservation,鈥 led by Namrataben Patel, a Katz School Ph.D. student in Mathematics in the Graduate Department of Computer Science and Engineering. She will present her research at the IEEE International Conference on Multimedia and Expo (ICME) in July.
In her paper, Patel set out to solve a simple but important problem: how to make AI-generated images behave more like a story instead of a collection of unrelated snapshots.
鈥淢ost of the existing models are designed to generate one image at a time,鈥 said Patel. 鈥淲hen I provide a text prompt, it generates one image. But when I provide the next prompt, it won鈥檛 necessarily get the same object in that next frame.鈥
In other words, today鈥檚 systems don鈥檛 really understand time or continuity. They don鈥檛 鈥渞emember鈥 what came before. As Patel puts it, 鈥渋t will randomly pick up the best matched object in the next frame,鈥 which is why characters often change unexpectedly.
CausalDreamer approaches the problem differently. Instead of treating each image as separate, it tries to model how events unfold step by step鈥攎ore like a storyboard than a single photograph. To do this, Patel trained the system using only text descriptions of cause and effect. For example: 鈥渁 dog digging a hole鈥 might lead to 鈥渢he ground breaks,鈥 then 鈥渢he hole gets deeper.鈥 These sequences teach the model how one action leads to another.
Initially, she struggled to generate consistent sequences鈥攌eeping the same subject while meaningfully evolving the scene from one image to the next was difficult. The breakthrough came when she moved beyond simple prompts and introduced a learned 鈥渟emantic shift鈥 between steps. By training a lightweight adapter and injecting this learned progression directly into the image generation process, she was able to guide how each frame evolves, enabling controlled and consistent transitions across images.
This idea connects to the concept of causality鈥攖he idea that actions lead to consequences. Patel drew inspiration from this principle to guide how the AI 鈥減ays attention鈥 when generating images.
鈥淲hen we hear the term causal, we think there is an action and an effect,鈥 she said. 鈥淚f a cup drops to the floor, it will break. But current models don鈥檛 think this way. They take the prompt as it is. They will not predict the after-effects.鈥
CausalDreamer begins to change that. If given a prompt like 鈥渢wo cars crashing,鈥 the system can start to imagine not just the crash, but what follows鈥攄amage, movement and consequences in the scene. Another key challenge was balance. Some AI models can keep a character consistent, but nothing changes from frame to frame. Others create variety, but lose the character entirely. Patel aimed for both.
鈥淪ome models were preserving identity, but they were not changing the actions,鈥 she said. 鈥淔or our model, it is like a balance where we can see that there is a change in action while persisting the identity.鈥
The result is a system that can generate a sequence of images showing meaningful progression, like a child playing with a ball or a dog digging a hole, while keeping the same subject recognizable throughout. One surprising feature of CausalDreamer is that it is relatively lightweight and accessible. Instead of retraining massive AI systems, Patel built a small add-on component.
鈥淪calability is very easy,鈥 she said. 鈥淵ou can add as many actions and effects as you want using a simple file.鈥
She has also shared the project publicly, making it easier for other researchers to build on the work. The system works especially well with people and animals, where identity is easier to recognize. It struggles more with generic objects like 鈥渁 bottle鈥 or 鈥渁 hat,鈥 where distinguishing features are limited. Patel sees this as an area for future improvement.
Looking ahead, she hopes to extend the work into full video generation and educational tools. 鈥淚f I am giving a lecture, I may want to explain it using some images,鈥 she said. 鈥淚n the future, I may be able to do that.鈥
Honggang Wang, senior author of the paper and chair of the department, noted that teaching AI to understand sequences and cause-and-effect relationships is a major step forward for applications like storytelling, training and scientific visualization.
鈥淏y helping machines move from single images to meaningful sequences,鈥 said Wang, 鈥淐ausalDreamer opens the door to AI systems that don鈥檛 just create pictures but begin to tell coherent visual stories.鈥
