SafeTensors Zero-Copy Model Weights
Hugging Face's memory-safe, zero-copy tensor serialization format designed to replace Pickle-based PyTorch checkpoints without arbitrary code execution risks.
Starts with an 8-byte unsigned little-endian integer N giving the exact byte length of the UTF-8 JSON header at [8 .. 8+N], followed immediately by raw aligned tensor byte buffers. Apple Silicon MLX quantized checkpoints reuse this exact framing (`__metadata__.format = 'mlx'`): 4-bit/8-bit weights are packed into U32 tensors (8 or 4 nibbles per lane) alongside per-group F16/BF16 `scales` and `biases` entries, with `quantization_bits`/`quantization_group_size` declared in `__metadata__`. There is no separate `.mlx` container extension—`.mlx` is MATLAB Live Script (a ZIP/OPC package); MLX weight files are distributed as `.safetensors` or `.npz`.
Drop any .safetensors file (even 100 GB+) to inspect locally
Reads tiny File.slice() header/footer pages in a sandboxed Web Worker · 0 Bytes Uploaded
.safetensors Magic Bytes & Header Offset Map
UTF-8 JSON header object opening immediately after uint64_le header_size
| Byte Offset | Length | Data Type | Field Name | Engineering Description |
|---|---|---|---|---|
| 0x00 - 0x07 | 8 B | uint64_le | header_size | Byte length N of the UTF-8 JSON header |
| 0x08 - 8+N | N B (dynamic) | utf8_json | json_header | Tensor dtype, shape, data_offsets & optional __metadata__ |
.safetensors Security & Polyglot Risk Matrix
Cap header_size N (e.g. <= 100 MB) before allocating buffer, parse UTF-8 JSON without duplicate keys, and verify every tensor's [begin, end] offset is non-overlapping and <= file_size - 8 - N.
Read .safetensors Header & Metadata Without Heavy Dependencies
import json, struct, sys
with open(sys.argv[1], "rb") as f:
(n,) = struct.unpack("<Q", f.read(8))
header = json.loads(f.read(n).decode("utf-8"))
tensors = [k for k in header if k != "__metadata__"]
print(f"SafeTensors header={n}B, tensors={len(tensors)}")Deterministic .safetensors Sample Files & Edge Cases
minimal-bf16.safetensors
Real safetensors.torch output with __metadata__ and 3 tensors across BF16, F16, and F32 dtypes.
curl -fsSL https://fileformat.co/fixtures/v1/safetensors/minimal-bf16.safetensors -o minimal-bf16.safetensors \
&& echo "0d9894b505b77ec0d719fa112eead1695c8a2be9e1820c81267dfe1a64a2214b minimal-bf16.safetensors" | shasum -a 256 -cweights-bf16.safetensors
Multi-layer transformer weight shard written by safetensors.torch with exact contiguous data_offsets.
curl -fsSL https://fileformat.co/fixtures/v1/safetensors/weights-bf16.safetensors -o weights-bf16.safetensors \
&& echo "7594cbd039f2978e74012d1e761368a7d18f2f6efe6038e11a869f6d1c5a6b7f weights-bf16.safetensors" | shasum -a 256 -cunaligned-truncated.safetensors
Negative fixture with non-zero starting data_offsets [3, 8195] and truncated 24-byte body — safetensors must raise.
curl -fsSL https://fileformat.co/fixtures/v1/safetensors/unaligned-truncated.safetensors -o unaligned-truncated.safetensors \
&& echo "70d504e4e3dc7cc38bf0d72fe83b78badaf96f16be32402100c1450a9bb2e004 unaligned-truncated.safetensors" | shasum -a 256 -cRelated Specifications & Alternatives
Self-contained binary model weight format for llama.cpp and local LLM inference engines, storing architecture KV metadata, BPE vocabulary, and quantized tensor blocks in a single mmap-ready file.
Default PyTorch `torch.save()` archive wrapping a Python Pickle instruction stream (`data.pkl`) alongside raw tensor storage files inside a ZIP container.
ZIP container bundling multiple named NumPy `.npy` tensor arrays, widely used for embeddings, calibration datasets, and JAX/MLX checkpoints.