Crusoe raised its fine-tuning upload limit to 16 GiB on October 22 with multipart support in Intelligence Foundry.
The prior 3 GiB cap is gone, replaced by parallel part uploads through the OpenAI SDK.
Staff engineer Abhishek Srikanth, group product manager Piyush Kadam and director Suman Debnath said, "We heard that our customers wanted to train on much larger datasets, and removing the per-upload size ceiling was a requested feature," per Crusoe.
The three-step flow creates an upload, adds parts in any order, then completes with an ordered list and optional MD5 checksum.
Crusoe allows 128 MiB per part and a 2 hour session, versus OpenAI's 64 MB and 1 hour, per the company's comparison table.
A 16 GiB file uploaded in about 79 seconds with 16 workers versus about 11 minutes single-threaded, per Crusoe's in-region benchmark at 5 ms from the API.
The OpenAI-compatible shape is documented elsewhere as POST /v1/uploads, POST /v1/uploads/:id/parts and POST /v1/uploads/:id/complete, per the DICOMPUTE Open issue.
Assembly now runs asynchronously with single-pass merge, checksum checks and encrypted storage that survives restarts, per Crusoe.
For LoRA fine-tuning, tokenization fell to 33 minutes from 4 hours on a 10 GiB dataset and load time fell to 82 seconds from 40 minutes while peak memory fell to 228 GiB from about 900 GiB, per Crusoe.
The fine-tuning API went generally available in July, and any dataset larger than 16 GiB still requires contacting Crusoe.
