Hugging Face's logo

PLAN-Lab
/
GraphVid

Image-to-Video
Diffusers
Safetensors
English
video-generation
controllable-generation
graph-conditioning
computer-vision
Model card Files
xet
 Community
GraphVid / README.md
Vedant
Readme was updated to improve model card
cbd287e
preview code
|
Raw
History Blame Contribute Delete
3.79 kB
metadata 
license: apache-2.0
language:
  - en
arxiv: 2607.2158
tags:
  - video-generation
  - image-to-video
  - controllable-generation
  - graph-conditioning
  - computer-vision

GraphVid: Interactive Graph-Controllable Video Generation

Vedant Shah, Onkar Susladkar, Tushar Prakash, Kiet A. Nguyen, Tianjiao Yu, Adheesh Juvekar, Muntasir Wahed, Ismini Lourentzou

馃帀 Accepted to ECCV 2026

馃搫 Paper 路 馃寪 Project Page 路 馃捇 Code 路 馃 GraphVid-Bench

Overview

GraphVid is an interactive graph-conditioned framework for controllable image-to-video generation.

Rather than requiring users to specify exact motion trajectories, GraphVid captures semantic interaction intent using structured interaction graphs. Users interact directly with objects in an input image, and these interactions are translated into graph-based conditioning that guides video generation.

GraphVid turns semantic interaction intent into video through structured interaction graphs.

Key Idea

GraphVid represents a scene using an interaction graph, where objects are represented as nodes and relationships or intended interactions are represented through edges.

User interactions are performed directly on the input image rather than on the graph itself. These interactions are interpreted and translated into an updated graph representation that captures the user's intended action.

The resulting graph provides structured conditioning to the video generation model:

the graph specifies what should happen, while the pretrained video backbone determines how the generated video should look and move.

Highlights

  • Semantic interaction control: Represents intended object interactions explicitly rather than treating control solely as geometric motion.

  • Interaction graphs: Uses structured graph representations to encode entities, relationships, and intended dynamics.

  • Image-based user interaction: Users interact directly with objects in the input image; the underlying graph is constructed and updated automatically.

  • Single- and multi-object control: Supports interactions involving individual objects as well as relationships between multiple objects.

  • GraphVid-Bench: Provides an interaction-focused video benchmark with structured relational annotations.

  • Lightweight conditioning: Introduces graph-based control while leveraging a pretrained video generation backbone.

Resources

Citation

If you find GraphVid useful in your research, please consider citing our work:

@article{shah2026graphvid,
  title={GraphVid: Interactive Graph-Controllable Video Generation},
  author={Shah, Vedant and Susladkar, Onkar and Prakash, Tushar and Nguyen, Kiet A. and Yu, Tianjiao and Juvekar, Adheesh and Wahed, Muntasir and Lourentzou, Ismini},
  journal={arXiv preprint arXiv:2607.21580},
  year={2026}
}