Skip to content
Vision Arena

All Benchmarks

Perception Test

A benchmark by DeepMind for low- and mid-level visual perception, covering memory, physics, and semantics across video, audio, and text.

Project PageGitHubPaper

Background

The Perception Test measures multimodal video models against scripted videos that capture perceptually interesting situations. It defines several tasks, including object and point tracking, action and sound localization, and multiple-choice and grounded video question answering. The challenge has changed over time, with an additional 1h+ walking-tour track being added in 2024, then merged overlapping tracks and extending the question-answering tasks in 2025. The latest version has 4 tracks and introduced new long-form spatial and multimodal video tasks.

Methodology

The benchmark centers on four skills (memory, abstraction, physics, and semantics) and four types of reasoning (descriptive, explanatory, predictive, and counterfactual), evaluated across video, audio, and text. It is designed to test how well pre-trained models transfer their capabilities in zero-shot, few-shot, or limited fine-tuning settings.

Dataset Structure

The dataset includes 11,609 real-world videos, averaging 23 seconds each, split roughly 20/50/30 across training (2,184 videos), validation (5,900 videos), and test (3,525 videos).

Benchmark Measured Capabilities

Task Examples

  • Question

    What letters did the person write on the paper?

    Question type
    Multiple choice
    Options
    • A.No letters
    • B.txo
    • C.wqrx
    Correct answer
    A. No letters
    Capability
    Perception
  • Question

    Which object was removed by the person from the tabletop?

    Question type
    Multiple choice
    Options
    • A.Tissue Roll
    • B.Fork
    • C.Steel Stand
    Correct answer
    A. Tissue Roll
    Capability
    Information Retrieval

@inproceedings{heyward2026perceptiontest2025,
title     = {Perception Test 2025: Challenge Summary and a Unified VQA Extension},
author    = {Joseph Heyward and Nikhil Parthasarathy and Tyler Zhu and Aravindh Mahendran and João Carreira and Dima Damen and Andrew Zisserman and Viorica Pătrăucean},
booktitle = {arXiv preprint},
year      = {2026},
url       = {https://arxiv.org/abs/2601.06287},
}