FinePDF

The data were collected from 105 CommonCrawl snapshots spanning summer 2013 to February 2025. For PDF extraction, two pipelines were used: a text-based Docling (CPU) pipeline and an image-based RolmOCR (GPU) pipeline, with an XGBoost model automatically classifying which extraction method best fits each PDF. The process also applied GlotLID-based language identification, exact + MinHash deduplication, and PII anonymization. Compared to the HTML dataset, the average document length is about 2× longer, and it includes many very long documents (100k+ characters), which is advantageous for improving long-context capability. However, model-based filtering was applied only to the English subset, and no heuristic filtering was applied to other languages, which may lead to large quality variance across languages.