HDC-AM Master Fold

Hyperdimensional Computing Address Mapping master ingestion. Streams HuggingFace master corpora (FineWeb-Edu / English Wikipedia / ArXiv) straight into a RAM clamp, folds to bipolar {-1,+1}^4096 fast-weights and emits fast_weights_master.safetensors at the 444.4 MiB budget.

parameter	value
dimension D	4096
alphabet	{-1,+1}^D (int8)
slots (rows)	113,766
bytes / slot	4096
total	465,987,174 B = 444.4 MiB
streaming loaders	FineWeb-Edu, English Wikipedia, ArXiv (RAM-only)

Validation gates

mmap zero-copy open latency < 10 ms
TR-1 strict bipolarity: every slot row in {-1,+1}^D
associative recall sanity (top-1 Hamming margin)

Local recipe (master fold)

python hdc_am_pipeline.py ingest --d 4096 --slots 113766 --chunk-docs 512 \
  --glob "corpus/*.txt" --filename fast_weights_master.safetensors
python hdc_am_pipeline.py stats --overlay fast_weights_master.safetensors
python hdc_am_pipeline.py recall --overlay fast_weights_master.safetensors \
  --queries "Taj Mahal mausoleum Yamuna Agra" --topk 8

CLI (equivalent)

python hdc_am_pipeline.py ingest --d 4096 --slots 113766 \
  --chunk-docs 512 --glob "corpus/*.txt" \
  --filename fast_weights_master.safetensors

Engine: hdc_am_pipeline.py · ingest_streamer.py · run_master.sh