Max individual file size for LFS files is 46.6GB
Hi, There seems to be a new limit for datasets, and I was just wondering if this is expected behavior. I’ve been pushing yearly zipped-Zarr stores for US precipitation radar data to openclimatefix/mrms · Datasets at Hugging Face successfully, each one being around 100-130GB each. I just tried to push an updated and fixed one for 2018, and am now having a new error that the max size is 46.6GB? I can split the Zarr stores into smaller ones, but it is simpler and easier to have a single large Zarr...
https://discuss.huggingface.co/t/max-individual-file-size-for-lfs-files-is-46-6gb/17950
Seonglae Cho
