PRX Part 4: Our Data Strategy
This is Part 4 of the PRX series. Parts 1 to 3 covered model architectures , training design , and a 24-hour speedrun .
Key Takeaways
- A Blog post by Photoroom on Hugging Face Back to Articles a]:hidden"> PRX Part 4: Our Data Strategy Team Article Published July 6, 2026 Upvote 4 Roman Frigg photoroman Follow Photoroom David Bertoin Bertoin Follow Photoroom Jon Almazán jon-almazan Follow Photoroom Welcome back!
This time we're pulling back the curtain on the part that quietly underpins all of it: the data.
- Guiding principles A diverse dataset for pre-training The goal was to assemble a large, diverse dataset for pre-training.
At this stage the model is learning how the world looks: the visual concepts, the objects and scenes, how things are composed and lit, and the sheer range of what images can contain.
- A mix of data sources We assemble our pre-training data from a mix of public and internal datasets.
The priority at this stage is breadth, diversity, and standing on curation that already exists rather than redoing it ourselves.
- Our captions philosophy In our experience, what matters most for pre-training is to use long captions that accurately describe everything in the image.
We saw this directly in Part 2 , where switching from short captions to long ones substantially improved sample quality.
- In combination with Mosaic Composer we found it to be a very low-maintenance, flexible, and well-performing framework for distributed training.
Stats & Key Facts
- #Parts 1 to 3 covered model architectures , training design , and a 24-hour speedrun .
This time we're pulling back the curtain on the part that quietly underpins all of it: the data. Of all the things that shaped PRX's quality, the data pipeline was one of the least glamorous parts to build but nevertheless an important piece to get right. Here's what we did, what we'd do differently, and a few things we only learned the slow way.
In one sentence: we assemble training data from a mix of public and internal datasets, re-caption the images with a VLM, and turn the result into the streamable corpus we trained PRX on. At a high level, the data pipeline looks like this: In the following we will dive into it in detail. Guiding principles A diverse dataset for pre-training The goal was to assemble a large, diverse dataset for pre-training.
At this stage the model is learning how the world looks: the visual concepts, the objects and scenes, how things are composed and lit, and the sheer range of what images can contain. That is a problem of coverage and diversity, not of per-image perfection. A broad, representative corpus teaches the model far more about the structure of the visual world than a smaller, prettier one would, even if many of the individual images are ordinary snapshots or slightly compressed.
For more details please read the original article at Hugging Face.
Continue Learning
Comments
Sign in to join the conversation