Common Crawl - Erratum - Content is truncated
The Data OverviewCDXJ IndexURL IndexWeb GraphsLatest CrawlCrawl StatsGraph StatsErrata
ResourcesGet StartedAI AgentBlogExamplesCCBotInfra StatusOpt-Out RegistryFAQ
CommunityResearch PapersMailing List ArchiveHugging FaceDiscordCollaborators
AboutAboutTeamJobsPrivacy PolicyTerms of Use
SearchAI Agent
Contact Us
< Back to Errata
ErratumContent is truncated
Originally reported by
.
Some archived content is truncated due to fetch size limits imposed during crawling. This is necessary to handle infinite or exceptionally large data streams (e.g., radio streams). Prior to March 2025 (CC-MAIN-2025-13), the truncation threshold was 1 MiB. From the March 2025 crawl onwards, this limit has been increased to 5 MiB.
For more details, see our truncation analysis notebook.
Affected Crawls
No items found.
Affected Web Graphs
No items found.
The DataOverviewCDXJ IndexURL IndexWeb GraphsLatest CrawlCrawl StatsGraph StatsErrata
ResourcesGet StartedAI AgentBlogExamplesCCBotInfra StatusOpt-Out RegistryFAQ
CommunityResearch PapersMailing List ArchiveHugging FaceDiscordCollaborators
AboutAboutTeamJobsPrivacy PolicyTerms of Use
© 2026 Common Crawl