LeBoardLeaderboard / vlabench unified
vlabench unified
Camera dropoutShare of frames from live episode cameras that are almost entirely one flat black or white value. A camera blank for a whole episode is counted as absent, shown underneath, not as a dropout. Method →
<0.1%
4 frames
BlurShare of camera frames inside an interval at least 0.5 s long where the image is much blurrier than that camera's typical frame. Method →
<0.1%
4,096 frames
DarknessShare of camera frames inside an interval at least 0.5 s long where 60% or more of the pixels are near-black. Method →
0
0 frames
FreezeShare of camera frames inside an interval at least 1 s long where the video repeats the exact same image. Method →
<0.1%
3,711 frames
Duration mismatchShare of assessed episode-camera pairs whose metadata video length disagrees with the Parquet frame count by more than 1 frame. A hatched cell with * means only some pairs could be assessed (the small number shows how many); the rest are unknown, not clean. –* means none could be assessed. Method →
0
0 of 32,931 assessed
Flagged episodesEpisodes flagged by any episode check: actions that never change, a robot that barely moves, non-finite values, broken row indexes or timestamp gaps. The count is on top, its share of assessed episodes underneath. Method →
0
of 10,977 assessed episodes
All checks
| Check | Flagged | Share | Coverage |
|---|---|---|---|
| Files 140 | |||
| Video decoding | 0 | 0 | complete |
| Episodes 10,977 | |||
| Episode boundaries | 0 | 0 | complete |
| Finite action and state | 0 | 0 | complete |
| Timestamp gaps | 0 | 0 | complete |
| Static actions | 0 | 0 | complete |
| Idle episodes | 0 | 0 | complete |
| Episode-camera pairs 32,931 | |||
| Duration mismatch | 0 | 0 | complete |
| Video/action sync | – | – | not measured |
| Frames 9,344,616 | |||
| Camera absent | 0 | 0 | complete |
| Camera dropout | 4 | <0.1% | complete |
| Blur | 4,096276 intervals | <0.1% | complete |
| Darkness | 0 | 0 | complete |
| Freeze | 3,711206 intervals | <0.1% | complete |
By camera
| Camera | Frames | Absent | Dropout | Blur | Darkness | Freeze |
|---|---|---|---|---|---|---|
| image | 3,114,872 | 0 | 0 | <0.1%119 intervals | 0 | 0.1%206 intervals |
| second_image | 3,114,872 | 0 | 0 | <0.1%13 intervals | 0 | 0 |
| wrist_image | 3,114,872 | 0 | <0.1% | 0.1%144 intervals | 0 | 0 |
Evidence
Longest video intervals up to 4 per signal · times are within the video file
| Signal | Camera | Episode | File | Start | End | Duration | |
|---|---|---|---|---|---|---|---|
| Blur | image | 9932 | file-039.mp4 | 95:54.3 | 96:00.8 | 6.5 s | Inspect → |
| Blur | image | 5087 | file-007.mp4 | 83:55.2 | 84:01.2 | 6.0 s | Inspect → |
| Blur | wrist_image | 7774 | file-024.mp4 | 53:05.5 | 53:10.8 | 5.3 s | Inspect → |
| Blur | wrist_image | 5565 | file-009.mp4 | 83:35.8 | 83:40.6 | 4.8 s | Inspect → |
| Freeze | image | 10510 | file-043.mp4 | 76:37.2 | 76:43.2 | 6.0 s | Inspect → |
| Freeze | image | 8141 | file-027.mp4 | 101:49.4 | 101:55.2 | 5.8 s | Inspect → |
| Freeze | image | 6572 | file-017.mp4 | 53:13.4 | 53:18.4 | 5.0 s | Inspect → |
| Freeze | image | 10851 | file-046.mp4 | 1:14.5 | 1:19.3 | 4.8 s | Inspect → |
| Camera dropout | wrist_image | 5027 | file-006.mp4 | 18:07.6 | 18:07.6 | 0.0 s | Inspect → |
| Camera dropout | wrist_image | 5372 | file-008.mp4 | 48:02.9 | 48:02.9 | 0.0 s | Inspect → |
| Camera dropout | wrist_image | 8153 | file-027.mp4 | 2:13.8 | 2:13.8 | 0.0 s | Inspect → |
| Camera dropout | wrist_image | 10788 | file-044.mp4 | 55:53.7 | 55:53.7 | 0.0 s | Inspect → |
Provenance
- Revision
- f61ad898199e746c20af3112f278cc86eb3ca3c4 (main)
- Scanned
- 2026-10-10 01:05 UTC · NVIDIA L40S
- Flag method
- rule-based-video-events-v1
- Episode method
- episode-motion-v1
- Code
- 9f54cd5fa0437e2c lock 282482d5a82a
Pinned dataset on Hugging Face ↗Result JSONMethodologyReport an issue with this result →