Technical Writing & Research

Deep dives into AI and LLM internals, cryptography, reverse engineering, and systems programming.

About

Languages

Frameworks

Databases & ORMs

Tooling & Infrastructure

Technical Writing & Research

Deep dives into AI and LLM internals, cryptography, reverse engineering, and systems programming.

13 min read

System One Models: How Non-Autoregressive Typed Decision Architectures Work

September 20, 2026

A technical explanation of the 2026 System One model class: parallel sampling instead of token-by-token decoding, schema-bound outputs that cannot produce type errors, and calibration trained as a first-class objective rather than prompted after the fact.

Highlights

• Parallel classification heads over an encoder-style representation, evaluated in one forward pass
• Schema-bound output primitives: bounded-cardinality choice, ordinal score, Bernoulli boolean
• Calibration metrics: Brier score, expected calibration error, reliability diagrams
• Reward design contrasts across RLHF, RLVR, and calibration-targeted reinforcement learning
• Hot-path / cold-path partitioning in agent control loops

#System One Models#Non-Autoregressive#Calibration#AI Systems

Read Full Article

26 min read

Inference-Efficient LLMs: How Sparse Compute and Compressed Memory Work

September 10, 2026

A technical breakdown of DeepSeek V4.1 Flash: asymmetric causal encoder-decoder prefill, CSA2 cross-layer KV reuse, FP4 caching, bounded replay, Engram memory, Single-Pass mHC, and DSpark speculative decoding.

Highlights

• 40-layer causal encoder-decoder split: 20 causal encoder layers plus 20 decoder layers
• Compressed Sparse Attention 2 with cross-layer KV reuse, index reuse, and hierarchical sparse indexing
• FP4 global KV cache at about 890 bytes per token and bounded reconstruction of short-lived SWA state
• 196B-parameter sparsely accessed Engram conditional memory
• Single-Pass manifold-constrained Hyper-Connections and DSpark five-token speculative drafting
• Native multimodal pre-training with a DeepSeek-ViT vision encoder

#DeepSeek#LLM Architecture#Mixture of Experts#Inference Systems

Read Full Article

14 min read

World Models: The Architecture Behind Spatial Intelligence

September 1, 2026

How world models actually work — latent dynamics, joint-embedding prediction, video simulators, and the 2026 shift to spatially grounded context where every token carries a 3D camera pose.

Highlights

• Recurrent state-space models: deterministic recurrence plus stochastic latents for dynamics
• Joint-embedding prediction in representation space instead of pixel space
• Autoregressive diffusion transformers over rectified flow in a compressed latent
• 3D Gaussian splatting, point clouds and depth maps as first-class model outputs
• Persistence and drift: why consistency is a representation problem, not a sampling problem

#World Models#Spatial Intelligence#3D Reconstruction#AI Architecture

Read Full Article

11 min read

Multi-Million-Token Context LLMs: How Long Context and Large Parameter Models Work

May 24, 2026

A practical explanation of context windows, model parameters, attention, KV cache memory, and why a million-token prompt is useful but not the same as perfect memory.

Highlights

• Tokenization and embeddings
• Self-attention, KV cache, positional encoding, and context caching
• Dense versus mixture-of-experts parameter usage
• Long-context limits and retrieval trade-offs

#LLM#Transformers#Long Context#AI Systems

Read Full Article

13 min read

System One Models: How Non-Autoregressive Typed Decision Architectures Work

September 20, 2026

Highlights

• Parallel classification heads over an encoder-style representation, evaluated in one forward pass
• Schema-bound output primitives: bounded-cardinality choice, ordinal score, Bernoulli boolean
• Calibration metrics: Brier score, expected calibration error, reliability diagrams
• Reward design contrasts across RLHF, RLVR, and calibration-targeted reinforcement learning
• Hot-path / cold-path partitioning in agent control loops

#System One Models#Non-Autoregressive#Calibration#AI Systems

Read Full Article

26 min read

Inference-Efficient LLMs: How Sparse Compute and Compressed Memory Work

September 10, 2026

Highlights

• 40-layer causal encoder-decoder split: 20 causal encoder layers plus 20 decoder layers
• Compressed Sparse Attention 2 with cross-layer KV reuse, index reuse, and hierarchical sparse indexing
• FP4 global KV cache at about 890 bytes per token and bounded reconstruction of short-lived SWA state
• 196B-parameter sparsely accessed Engram conditional memory
• Single-Pass manifold-constrained Hyper-Connections and DSpark five-token speculative drafting
• Native multimodal pre-training with a DeepSeek-ViT vision encoder

#DeepSeek#LLM Architecture#Mixture of Experts#Inference Systems

Read Full Article

14 min read

World Models: The Architecture Behind Spatial Intelligence

September 1, 2026

How world models actually work — latent dynamics, joint-embedding prediction, video simulators, and the 2026 shift to spatially grounded context where every token carries a 3D camera pose.

Highlights

• Recurrent state-space models: deterministic recurrence plus stochastic latents for dynamics
• Joint-embedding prediction in representation space instead of pixel space
• Autoregressive diffusion transformers over rectified flow in a compressed latent
• 3D Gaussian splatting, point clouds and depth maps as first-class model outputs
• Persistence and drift: why consistency is a representation problem, not a sampling problem

#World Models#Spatial Intelligence#3D Reconstruction#AI Architecture

Read Full Article

11 min read

Multi-Million-Token Context LLMs: How Long Context and Large Parameter Models Work

May 24, 2026

A practical explanation of context windows, model parameters, attention, KV cache memory, and why a million-token prompt is useful but not the same as perfect memory.

Highlights

• Tokenization and embeddings
• Self-attention, KV cache, positional encoding, and context caching
• Dense versus mixture-of-experts parameter usage
• Long-context limits and retrieval trade-offs

#LLM#Transformers#Long Context#AI Systems

Read Full Article