.avro Tier 1 · Deep AST Parser

Apache Avro Object Container File

Row-oriented binary data serialization container widely used in Kafka/Confluent event streaming and Iceberg manifest lists, embedding full JSON writer schema in the file header.

Begins with `Obj\x01` (`4F 62 6A 01`), followed by a ZigZag varint-encoded metadata map (`avro.schema`, `avro.codec`) and a 16-byte random sync marker repeated after every block.

MIME Types application/avro, application/octet-stream

Byte Endianness little

Canonical Spec Official Specification ↗

0xFF

Drop any .avro file (even 100 GB+) to inspect locally

Reads tiny File.slice() header/footer pages in a sandboxed Web Worker · 0 Bytes Uploaded

Binary Signature & Wire Layout

.avro Magic Bytes & Header Offset Map

Offset: 0x00 (0 B) ASCII: Obj.

4F 62 6A 01

Avro Object Container v1 magic ('O','b','j', 0x01)

Byte Offset	Length	Data Type	Field Name	Engineering Description
0x00 - 0x03	4 B	uint8[4]	magic	'Obj\x01' container signature
0x04+	var	avro_map	file_metadata	ZigZag varint map containing avro.schema & avro.codec
hdr_end-16	16 B	uint8[16]	sync_marker	16-byte random block boundary sync token

Upload Gateway & AppSec Advisory

.avro Security & Polyglot Risk Matrix

Risk Level: low

Arbitrary Code / Macros

NO (Inert Data)

Pickle Deserialization

SAFE (No Pickle VM)

XML External Entity (XXE)

NONE

Polyglot Prepend Risk

LOW (Strict 0x00 Magic)

Backend Upload Validation Recommendation

Validate `Obj\x01` prefix and cap maximum `avro.schema` JSON recursion depth and block decompression size.

Compiler-Verified Reference Code

Read .avro Header & Metadata Without Heavy Dependencies

✓ Zero-Dependency .avro Header Reader

import sys
with open(sys.argv[1], "rb") as f:
    buf = f.read(4096)
    assert buf[:4] == b"Obj\x01"
    has_schema = b"avro.schema" in buf
    print(f"Avro v1 container, embedded_schema={has_schema}")

CI/CD & Unit Test Fixtures

Deterministic .avro Sample Files & Edge Cases

Open Full .avro Sample Matrix & Mutator →

minimalexpect: valid

minimal-null.avro

Real Avro OCF from fastavro with embedded record schema, null codec, and 16-byte sync markers.

↓ Download (198 B)

SHA-256: 8f5420609b6bb63e55d37521ce5163e81467a3ec798784bf5857e3c4131acdfd

Exact: 198 bytes

curl -fsSL https://fileformat.co/fixtures/v1/avro/minimal-null.avro -o minimal-null.avro \
  && echo "8f5420609b6bb63e55d37521ce5163e81467a3ec798784bf5857e3c4131acdfd  minimal-null.avro" | shasum -a 256 -c

realisticexpect: valid

events-deflate.avro

fastavro-written OCF with deflate-compressed block of 10 UserEvent records.

↓ Download (241 B)

SHA-256: bc90476179aca51aae52611b266a87a02d16fd5392fa551554874ce3218f341a

Exact: 241 bytes

curl -fsSL https://fileformat.co/fixtures/v1/avro/events-deflate.avro -o events-deflate.avro \
  && echo "bc90476179aca51aae52611b266a87a02d16fd5392fa551554874ce3218f341a  events-deflate.avro" | shasum -a 256 -c

edge-caseBroken 16B Sync Markerexpect: parser_error

corrupted-sync-marker.avro

Avro block cut off mid-sync-marker — fastavro reader must raise on block-boundary failure.

↓ Download (186 B)

SHA-256: 6534e49af5871a7f976a650fb164f50be0a9231d34ec2a6e46600354ca5da803

Exact: 186 bytes

curl -fsSL https://fileformat.co/fixtures/v1/avro/corrupted-sync-marker.avro -o corrupted-sync-marker.avro \
  && echo "6534e49af5871a7f976a650fb164f50be0a9231d34ec2a6e46600354ca5da803  corrupted-sync-marker.avro" | shasum -a 256 -c

Related Specifications & Alternatives

.parquet little

Parquet

De-facto standard hybrid columnar storage format for modern data lakes (Iceberg, Delta, DuckDB, Spark), placing Thrift Compact Protocol schema and statistics in the file trailer.

.orc little

Apache ORC

Stripe-based columnar storage format originating in the Hive/Trino/Presto ecosystem with Protobuf-encoded Footer and a 1-byte PostScript length at the very last byte of the file.

.jsonl text

JSONL

Streaming line-oriented text format where every newline-separated record is a standalone UTF-8 JSON value—standard for LLM fine-tuning (SFT/DPO) datasets and batch APIs.