Future Video Generation
Generate future video from observation and task instruction.
Input: context video + goal instruction
Output: future video
Overview
The ActPhysCause Challenge evaluates whether embodied world models can understand robot actions as causal interventions and generate future videos that are physically plausible, causally consistent, and action-controllable under task prompts, initial observations, and counterfactual action conditions.
The ECCV Workshop edition evaluates Track 0 only: future video generation from an observation and a task instruction. Tracks 1–3 remain a public research roadmap, not competition tracks in this edition. The dataset has two splits only: a training set and an evaluation set.
1Active ECCV track
3Future tracks
2Dataset splits
1Evaluation server
TBDRegistration date
2026 Competition Track & Future Roadmap
Track 0 is the only competition track for this ECCV Workshop edition. Tracks 1–3 describe future extensions of the benchmark.
Video-Action Joint Generation
Predict the next action chunk and the resulting visual rollout.
Input: context video + action history + goal instruction
Output: future action chunk + future video
Action Correction
Given a bad future action, output an improved action and corrected rollout.
Input: context video + action history + goal instruction + bad action chunk
Output: delta action / corrected action chunk + corrected future video
Causal Action Selection
Rank candidate actions by how likely they are to cause the target effect.
Input: context video + action history + target effect + candidate actions
Output: action scores / ranking / selected action
Only Track 0 is scored and ranked at the ECCV Workshop. Track 1–3 specifications are shown as a future research roadmap only.
Dataset
The current release is track0_v0. It contains 50 dual-arm tabletop manipulation tasks generated with a fixed head-camera view.
- Training set: 2,000 episodes, grouped by task, with first frame, instruction, metadata, and ground-truth video.
- Evaluation set: 500 anonymized samples, released with first frame, instruction, and submission metadata. Evaluation annotations are not released.
- Format: 640 x 480 RGB first frames and H.264 videos from a fixed head camera.
Dataset available: The Track 0 training and evaluation data are released on Hugging Face. Open the official dataset page.
Challenge Overview & Dataset Examples
The overview below presents the benchmark roadmap. The video gallery that follows contains representative Track 0 training samples, where the target output is the future manipulation video.
Page 1 / 3
click_alarmclock / episode0
Click Alarm Clock
Click the center button on the alarm-clock with digital display's top side.
click_bell / episode0
Press Bell
Press the center top of the bell with metallic top and plastic base.
turn_switch / episode0
Turn Switch
Activate the smooth tan switch with textured sides with the left arm.
beat_block_hammer / episode0
Beat Block with Hammer
Grab the silver hammer using the right arm, then beat the block.
open_laptop / episode0
Open Laptop
Open the medium foldable silver laptop completely.
grab_roller / episode0
Grab Roller
Securely grab the light brown wooden roller using both arms.
move_can_pot / episode0
Move Can to Pot
Take the sauce can with the right arm, bring it, and set it next to the silver kitchen pot.
rotate_qrcode / episode0
Rotate QR Sign
Grab the tabletop payment sign, lift it from the table, and rotate it QR front.
handover_block / episode0
Handover Block
Lift, switch arms, and place the red block.
place_burger_fries / episode0
Place Burger and Fries
Set the hamburger and fries box down on the bright orange rectangular tray after picking them.
shake_bottle / episode0
Shake Bottle
Use the left arm to pick and shake the bottle with brown body and white top.
stack_blocks_two / episode0
Stack Two Blocks
The left arm takes red block, sets it in the center, then places green block on red block.
Track 0 Evaluation
Only Track 0 is evaluated in the ECCV Workshop edition. The evaluation focuses on future-video quality and whether the generated rollout is consistent with the task instruction.
| Track | Main focus | Representative metrics |
|---|---|---|
| Track 0 | Future video quality and goal consistency | Video quality, temporal coherence, physical plausibility |
Submission
The official starter kit will include format-checking scripts. Participants will submit Track 0 future-video predictions for the evaluation set through the submission portal.
{
"sample_id": "scene_000123",
"track": "track0",
"future_video": "predictions/scene_000123.mp4"
}
Leaderboard
| Leaderboard | Status | Description |
|---|---|---|
| Track 0 | Coming Soon | Future video generation ranking. |
| Track 1–3 | To be continued | Future roadmap only. These tracks are not evaluated or ranked at the ECCV Workshop. |
Important Dates
| Item | Date |
|---|---|
| Website online | 2026-07-07 |
| Registration opens | TBD |
| Training and evaluation data release | TBD |
| Submission portal opens | TBD |
| Final submission deadline | TBD |
| ECCV Workshop presentation | TBD |
Organizers
Organizer Institutes
- X-Era AI
- Sun Yat-sen University
- University of Technology Sydney
- The Hong Kong University of Science and Technology (Guangzhou)
Challenge Organizers
- Tianshui Chen, X-Era AI
- Jusheng Zhang, Sun Yat-sen University
- Qinhan Lyu, Sun Yat-sen University
- Tongyu Mo, Sun Yat-sen University
- Ruichong Gao, Sun Yat-sen University
- Zijie Liang, Sun Yat-sen University
- Keze Wang, Sun Yat-sen University
- Wenhao Wang, University of Technology Sydney
- Sidi Liu, The Hong Kong University of Science and Technology (Guangzhou)
Contact
The challenge website is online. The Codabench page, contact email, and discussion channel are under preparation.
Website Online Codabench Under Preparation Registration Coming Soon Dataset Coming Soon