Hugging Face repositories

Models, datasets and Spaces for huggingface_hub, the hf command, transformers and datasets

Hugging Face repositories are endpoints for the huggingface_hub library, the hf command and everything built on them (transformers, datasets, diffusers). Point HF_ENDPOINT at a Depotex repository and models, datasets and Spaces live there instead of on huggingface.co. One Depotex repository holds any number of Hugging Face repositories (org/name). Hosted repositories keep a real history (commits, branches and tags); proxy repositories cache the Hugging Face Hub or another Depotex; groups combine them.

Each commit's model card becomes an AI BOM: the model or dataset, its base models and training datasets, and its license, as an SBOM in the browser's Security tab and in Components search.

1. Set up

export HF_ENDPOINT=https://depotex.example.com/repository/my-space/hf
export HF_TOKEN="$DEPOTEX_TOKEN"     # an API token
export HF_HUB_DISABLE_XET=1          # for uploads, when hf_xet is installed

hf repos create acme/demo
hf upload acme/demo ./model .
hf download acme/demo
hf repos tag create acme/demo v1.0

from huggingface_hub import snapshot_download
snapshot_download("acme/demo", revision="v1.0")   # reads HF_ENDPOINT and HF_TOKEN

Libraries that load from the Hub (from_pretrained, load_dataset) use the same variables. The repository's Usage tab shows them with your address filled in.

Depotex doesn't offer Xet storage. Downloads fall back to plain HTTP on their own, but uploads of large files need HF_HUB_DISABLE_XET=1 when the hf_xet package is installed, so they go through Git LFS.

Uploads need huggingface_hub 0.11 or later; older versions send commits in a form Depotex doesn't read, and can only download.

2. Repositories, branches and tags

Creating a repository needs Publish permission and makes its first commit on main. Uploads are commits: small files go inside the commit, larger ones (over HUGGINGFACE_REGULAR_MAX_SIZE, 10 MiB by default) through Git LFS.
Branches and tags are created and deleted as on the Hub (hf repos branch create, hf repos tag create). A commit is refused if its branch moved since the client read it, so two uploads can't overwrite each other.
Downloads carry the headers the client's cache needs (X-Repo-Commit, ETag and, for large files, X-Linked-Etag), and a cached file is revalidated with a 304 when it hasn't changed.
Creating, deleting, changing a repository's private flag, and creating or deleting branches and tags raise the huggingface.repo.update event, so policies can refuse them.

3. Versions and policies

Each commit is a version: cleanup policies, quotas, quarantine and the block list act on commits. Downloading at a quarantined or blocked commit is refused.
A file's content is stored once per Hugging Face repository, whatever commits use it. Content no commit uses any more is removed by the nightly huggingface-gc job after HUGGINGFACE_GC_GRACE (24 hours).

Cleanup sees commits, not branches: a rule can remove the commit a branch points to, and the branch then answers 404 until it moves. Keep cleanup rules away from Hugging Face repositories whose branches must stay readable, or keep enough recent versions.

4. Proxy repositories

Add an upstream with the Hugging Face Hub preset (https://huggingface.co), allow it for the space, then create a proxy repository with it. A proxy caches each file by commit, checked against the Hub's SHA-256 for large files; listings, trees and history are forwarded live. Proxies are read-only. A proxy can also reach a Hugging Face repository of another Depotex (Depotex link).

5. Groups

A Hugging Face group merges the lists of models, datasets and Spaces of its members, reads each repository from the first member that has it, and sends writes to its write member.

6. In the artifact browser

Models, datasets and Spaces show by id, with their commits as versions. The Install bar shows hf download and snapshot_download at that commit.

7. Server settings

Variable	Default	Description
HUGGINGFACE_REGULAR_MAX_SIZE	10485760	Largest file sent inside a commit (1 KiB–100 MiB); larger ones go through Git LFS
HUGGINGFACE_UPLOAD_LINK_TTL	2h	How long a large-file upload link works (1m–48h)
HUGGINGFACE_GC_GRACE	24h	How long content no commit uses is kept before it's removed (1h–30 days)

The artifact size limit (Settings → Repositories) applies to every file.

8. Limits

No Xet storage (see above).
No git access: git clone and git push of a model, dataset or Space aren't supported. Use the Hugging Face libraries, hf or huggingface-cli, which use the HTTP API.
No moving repositories, pull requests, discussions or the Hub's conversions.
Through a proxy, a revision with / in its name (other than refs/pr/N) may not resolve: use the commit id.