Consider external file storage?

[lwhsu]

The repository size is over 400MB, soon it will be hard to clone and maintain. It's time to think about a more efficient way to add new content.

[ppaeps]

We could simply put every year in a separate Git submodule, making the initial clone faster.

I don’t really mind the initial time to clone. But I can see how it would turn people off submitting their slides as pull requests. (Cloning the repo from Australia was not a pleasant experience.)

Does GitHub have any kind of external storage? Or should we use a magic e.g. public_papers on freefall?

[ppaeps]

Note that using freefall would limit us to speakers with FreeBSD.org accounts. By keeping the storage on GitHub, members of the wider FreeBSD community have a lower barrier to entry way of sharing their presentations.

[lwhsu]

I'm thinking of https://help.github.com/en/github/managing-large-files/configuring-git-large-file-storage and have them also backup to freebsd.org infra.

[lwhsu]

But LFS doesn't solve the issue when you just want to attach a new big file. Another thought is structure the directory layout to be more suitable for sparse checkout.

[ppaeps]

Simply restructuring the repository to enable sparse checkouts is probably the best way forward. Could you also update README.md with instructions for casual users who simply want to submit a pull request with a presentation they did, without downloading the entire repository?

[ppaeps]

Will anyone get deeply upset if I upload the slides for An Introduction to hardware hacking with FreeBSD (Tom Jones)? It would add 193M to the repository.

[lwhsu]

I suggest let's hold off this a little bit. I haven't had time to check a suitable directory layout or a way to handle the large files.

[allanjude]

I think doing the years as the point of the sparce checkout makes sense. Since if you are just doing a checkout to add new stuff, you only need 1 or 2 years checked out

[ppaeps]

It sounds like sparse checkouts with Git are still very experimental¹ according to the documentation as of today. They're probably a better long-term solution than git-lfs though. There are distinct advantages to having all the data in the repository so that everyone with a full clone has a backup. The alternative is relying on whoever provides the LFS storage to be around forever. Though I don't see GitHub disappearing imminently.

Should I just go ahead and commit the huge presentation under the assumption that sparse checkouts will become possible in the near future?