Inside DeepSeek: Reverse Engineering an AI Assistant by Interviewing Itself
Summary
- DeepSeek is analyzed via an interview-first methodology, then cross-checked against public papers to separate observed facts from inferences and guesses. - The article explores core concepts like context window, prompt pipeline, token generation, and the distinction between observation vs inference vs guess. - It explains hidden reasoning in two senses (latent activations vs reasoning tokens) and discusses tool calls, memory, hallucinations, and safety as multi-layered systems. - MoE and MLA are highlighted as key architectural techniques for long-context handling, with numbers verified against arXiv papers (e.g., 256 experts, 671B total parameters, 37B active per token). - Readers are urged to interview for behavior and verify architectural claims with papers; transcripts and interactive labs accompany the guide.