DOCCI: Descriptions of Connected and Contrasting Images
Skip to main content
Explore our many areas of focus
Explore all research areas
Applied AI & sciences
Earth AI
Health AI
Science AI
Sustainability & crisis resilience
Foundational ML & algorithms
Algorithms & theory
Information retrieval
Machine intelligence
Machine perception
Natural language processing
People, systems & quantum AI
Human-computer interaction and visualization
Networking
Quantum AI
Responsible AI
Anti abuse
Software engineering
Software systems
Learn More
Publications
Projects
Building a collaborative ecosystem
Datasets
Access high-quality datasets to accelerate your research.
Tools & services
Explore our latest AI models and products.
Open source
Discover open-source code and collaborate with the community.
Shaping the future together
See all programs
Faculty programs
Participating in the academic research community through meaningful engagement with university faculty.
Student programs
Supporting the next generation of researchers through a wide range of programming.
Locations
Find your place in our global offices and research labs.
Translating discovery into real-world impact
People
Our researchers drive advancements in computer science through both fundamental and applied research.
Teams
Collaborative groups tackling the world's most challenging AI problems.
Research
Explore our many areas of focus
Explore all research areas
Applied AI & sciences
Earth AI
Health AI
Science AI
Sustainability & crisis resilience
Foundational ML & algorithms
Algorithms & theory
Information retrieval
Machine intelligence
Machine perception
Natural language processing
People, systems & quantum AI
Human-computer interaction and visualization
Networking
Quantum AI
Responsible AI
Anti abuse
Software engineering
Software systems
Learn More
Publications
Projects
Resources
Building a collaborative ecosystem
Datasets
Access high-quality datasets to accelerate your research.
Tools & services
Explore our latest AI models and products.
Open source
Discover open-source code and collaborate with the community.
Conferences & events
Careers
Shaping the future together
See all programs
Faculty programs
Participating in the academic research community through meaningful engagement with university faculty.
Student programs
Supporting the next generation of researchers through a wide range of programming.
Locations
Find your place in our global offices and research labs.
Blog
About
Translating discovery into real-world impact
People
Our researchers drive advancements in computer science through both fundamental and applied research.
Teams
Collaborative groups tackling the world's most challenging AI problems.
Google
Research
Google AI
Learn about all our AI
Google DeepMind
Explore the frontier of AI
Google Labs
Try our AI experiments
Research
Resources
Conferences & events
Careers
Blog
About
Search
Home
Publications
DOCCI: Descriptions of Connected and Contrasting Images
Alex Ku
Garrett Tanzer
Jaemin Cho
Jason Baldridge
Jordi Pont-Tuset
Roopal Garg
Su Wang
Sunayana Rane
Yasumasa Onoe
Yonatan Bitton
Zack Berger
Zarana Parekh
(2024)
Download
Google Scholar
Copy Bibtex
Abstract
Despite recent advancements, text-to-image (T2I) models still exhibit critical limitations, such as errors in understanding spatial relationships, object counting, text rendering, and more. One challenge in overcoming these failure modes is the lack of resources; the majority of existing image-text datasets provide only brief captions that do not offer sufficient detail to discrepancies between images and their descriptions. To advance the development of T2I models further, we introduce \textbf{Descriptions of Connected and Contrasting Images (DOCCI)}, a dataset of 15k images taken by a single person with detailed human-annotated descriptions in English. We meticulously annotated detailed and coherent descriptions, averaging 136 words, which sufficiently differentiate images from related or similar ones. We intentionally curated images that showcase a diverse range of visual properties, including entities with their attributes, various orientations, and lighting effects, many of which are related to each other. We thoroughly analyze the quality and characteristics of the image-description pairs, and assess the performance of the latest T2I and I2T models. The experimental results indicate that the current state-of-the-art T2I models still struggle with the aforementioned challenges, and even the SOTA models have not fully addressed them. DOCCI is publicly available, and we believe that this dataset will be a valuable benchmark for vision-language research.
Research Areas
Machine perception
Natural language processing
Meet the teams driving innovation
Our teams advance the state of the art through research, systems engineering, and collaboration across Google.
See our teams
Follow us
Explore our other initiatives
Google AI
Discover how Google AI is committed to enriching knowledge and solving complex challenges
Products
Build
Research
Responsibility
Societal Impact
About
Google Cloud
High-performance infrastructure for cloud computing, data analytics & machine learning
Overview
Solutions
Products
Pricing
Resources
Google DeepMind
Our mission is to build AI responsibly to benefit humanity
Models
Research
Science
About
Google Labs
Explore the future of AI responsibly with Google Labs
About
Experiments
Stay connected
About Google
Google Products
Privacy
Terms
Cookies management controls
×