The insane engineering of Deepseek V4
Deepseek V4 explained. #ai #aitools #ainews #llm #agi #deepseek #claude #agi
Thanks to our sponsor Abacus AI. Try ChatLLM & DeepAgent today: http://chatllm.abacus.ai/?token=aisearch
Deepseek v4: https://api-docs.deepseek.com/news/news260424
LLMs explained: https://youtu.be/U2hZFMVNSE0
Residual connections: https://youtu.be/2IfAVV7ewO0
0:00 Deepseek V4 intro
1:00 Deepseek V4 specs
2:06 The challenge of 1M context
4:16 Hybrid attention
5:11 CSA & sparse selection
6:50 HCA
8:22 Sliding window attention
10:44 Insane efficiency gains
12:02 Signal explosion
13:00 Residual connections
13:52 mHC
14:17 ChatLLM
15:24 mHC continued
17:54 Muon
19:26 Infra challenges
22:31 Training challenges
24:09 Anticipatory routing
25:24 SOTA results
Newsletter: https://aisearch.substack.com/
Find AI tools & jobs: https://ai-search.io/
Support: https://ko-fi.com/aisearch
Here’s my equipment, in case you’re wondering:
Lenovo Thinkbook: https://amzn.to/4jWeKwH
Dell Precision 5690: https://www.dell.com/en-us/dt/ai-technologies/index.htm?utm_source=AISearchTools&utm_medium=youtube&utm_campaign=precisionai#tab0=0
GPU: Nvidia RTX 5000 Ada https://nvda.ws/3zfqGqS
Mic: Shure SM7B https://amzn.to/3DErjt1
Audio interface: Scarlett Solo https://amzn.to/3qELMeu
Bedsure Calming Dog Bed for Small Dogs - Donut Washable Small Puppy Bed, 23 inches Anti-Slip Round Fluffy Plush Faux Fur Large Cat Bed, Fits up to 25 lbs Pets, Camel
$27.00 (as of May 1, 2026 02:13 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)
Monster Energy Zero Ultra, Sugar Free Energy Drink, 16 Ounce | Pack of 15
$27.48 (as of May 1, 2026 02:13 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)
Amazon Fire TV Stick HD, free and live TV, Alexa Voice Remote, smart home controls, HD streaming
$34.99 (as of May 1, 2026 02:13 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)
50 Bags Hyssop Dried Herb Hyssop Tea Herbal (Hyssópus Officinális) - 50 Count 1.5 g Tea Bag Herb Hyssops Hissop Hysopp Hisopo
$14.99 (as of May 1, 2026 02:13 GMT +00:00 - More infoProduct prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on [relevant Amazon Site(s), as applicable] at the time of purchase will apply to the purchase of this product.)
Post Comment