.arrow .ipc Tier 1 · Deep AST Parser

Apache Arrow IPC Random-Access File

Zero-copy in-memory columnar format serialized to disk with 8-byte aligned `ARROW1\x00\x00` framing and FlatBuffers schema/record-batch descriptors.

Distinguished from Arrow Streaming IPC (`.arrows`, which starts directly with `0xFFFFFFFF` continuation) by the 8-byte `ARROW1\x00\x00` header and trailing `ARROW1` footer.

MIME Types application/vnd.apache.arrow.file

Byte Endianness little

Canonical Spec Official Specification ↗

0xFF

Drop any .arrow file (even 100 GB+) to inspect locally

Reads tiny File.slice() header/footer pages in a sandboxed Web Worker · 0 Bytes Uploaded

Binary Signature & Wire Layout

.arrow Magic Bytes & Header Offset Map

Offset: 0x00 (0 B) ASCII: ARROW1..

41 52 52 4F 57 31 00 00

6-byte ARROW1 magic padded to 8-byte SIMD alignment

Byte Offset	Length	Data Type	Field Name	Engineering Description
0x00 - 0x07	8 B	char[6]+pad	header_magic	'ARROW1' + 2 zero padding bytes
0x08 - 0x0B	4 B	uint32_le	continuation	0xFFFFFFFF IPC message continuation marker
EOF-10 .. EOF-7	4 B	int32_le	footer_size	FlatBuffers Footer byte length
EOF-6 .. EOF-1	6 B	char[6]	footer_magic	Trailing 'ARROW1' magic

Upload Gateway & AppSec Advisory

.arrow Security & Polyglot Risk Matrix

Risk Level: low

Arbitrary Code / Macros

NO (Inert Data)

Pickle Deserialization

SAFE (No Pickle VM)

XML External Entity (XXE)

NONE

Polyglot Prepend Risk

LOW (Strict 0x00 Magic)

Backend Upload Validation Recommendation

Verify both leading 'ARROW1\x00\x00' and trailing 'ARROW1', and validate FlatBuffer buffer offsets before zero-copy pointer arithmetic.

Compiler-Verified Reference Code

Read .arrow Header & Metadata Without Heavy Dependencies

✓ Zero-Dependency .arrow Header Reader

import os, struct, sys
with open(sys.argv[1], "rb") as f:
    head = f.read(8)
    f.seek(-10, os.SEEK_END)
    flen, tail = struct.unpack("<i6s", f.read(10))
    assert head == b"ARROW1\x00\x00" and tail == b"ARROW1"
    print(f"Arrow IPC valid: footer={flen}B")

CI/CD & Unit Test Fixtures

Deterministic .arrow Sample Files & Edge Cases

Open Full .arrow Sample Matrix & Mutator →

minimalexpect: valid

minimal-ipc.arrow

Real Arrow IPC file-format output from pyarrow RecordBatchFileWriter with ARROW1 magic + FlatBuffers footer.

↓ Download (666 B)

SHA-256: b86279b7e38e81af141b12db151638fa780d57567b64de10e4a4e1c1610e6b64

Exact: 666 bytes

curl -fsSL https://fileformat.co/fixtures/v1/arrow/minimal-ipc.arrow -o minimal-ipc.arrow \
  && echo "b86279b7e38e81af141b12db151638fa780d57567b64de10e4a4e1c1610e6b64  minimal-ipc.arrow" | shasum -a 256 -c

realisticexpect: valid

record-batch.arrow

pyarrow-written 100-row IPC file with int64/float64/string columns in a single record batch.

↓ Download (3.3 KB)

SHA-256: 469652943f52989ed6b08a1ebea08fe5755d9fed2be0a6e9d4ee8f22a8775121

Exact: 3,402 bytes

curl -fsSL https://fileformat.co/fixtures/v1/arrow/record-batch.arrow -o record-batch.arrow \
  && echo "469652943f52989ed6b08a1ebea08fe5755d9fed2be0a6e9d4ee8f22a8775121  record-batch.arrow" | shasum -a 256 -c

edge-caseMissing ARROW1 Footerexpect: parser_error

missing-footer.arrow

Arrow IPC file-format bytes truncated inside the FlatBuffers footer — RecordBatchFileReader must raise.

↓ Download (650 B)

SHA-256: 2c7855bd0e3ae0dc6aad8260b0c03c8c3aeed62adcd2c5211a7fd6ab98bd6e5d

Exact: 650 bytes

curl -fsSL https://fileformat.co/fixtures/v1/arrow/missing-footer.arrow -o missing-footer.arrow \
  && echo "2c7855bd0e3ae0dc6aad8260b0c03c8c3aeed62adcd2c5211a7fd6ab98bd6e5d  missing-footer.arrow" | shasum -a 256 -c

Related Specifications & Alternatives

.feather little

Feather v2

Fast, language-agnostic DataFrame persistence format for Python (Pandas/Polars) and R, standardized since v2 directly atop the Apache Arrow IPC file specification.

.parquet little

Parquet

De-facto standard hybrid columnar storage format for modern data lakes (Iceberg, Delta, DuckDB, Spark), placing Thrift Compact Protocol schema and statistics in the file trailer.

.duckdb little

DuckDB

Single-file vectorized OLAP database format storing compressed columnar row-groups in 4 KB / 256 KB block-managed pages.