tiny-llm and Practical PyTorch Learning Prerequisites
tiny-llm Exactly what I wished for. It also contains links to two existing PyTorch related courses to machine learning from Carnegie Mellon University.
Tag: papers
9 posts
Ultra-Scale LLM Training Playbook and Streaming DiLoCo
The Ultra-Scale Playbook: Training LLMs on GPU Clusters — Amazing, and finally we have a 100-page open-source online book on how models are trained with multiple GPUs.
Unsloth GRPO, S1-Style Scaling, and RL Learning Resources
Unsloth.ai’s GRPO — it seems that the Unsloth implementation of GRPO uses less GPU memory, and it supports QLoRA and LoRA.
AI Peer Review with LLMs and S1 Test-Time Scaling
DOGE: Make AI Conferences Great Again — Zeyuan (Allen) Zhu wrote a very interesting piece on using LLMs as arbitrators in the reviewer-author discussions.
GRPO on Apple MLX and Minimal-R1 Scaling Insights
GRPO will soon be added to Apple MLX — The PR now works, using about 32 GB of memory when training Qwen2.5-0.5B.
DeepSeek, Export Controls, and Open-Weight AI Debates
On DeepSeek and Export Controls — Dario Amodei, Anthropic’s CEO, wrote a fairly long editorial on DeepSeek.
Qwen 2.5 7B 1M Local Testing and RL Survey Notes
Qwen 2.5 7B 1M — I have just tried Qwen’s latest local model, the 7B 1M, locally in LM Studio 0.3.8 (Build 4). I loaded an entire PhD thesis into the model, and LM Studio gleefully chose inject-full-content as its content injection strategy.
Nvidia, DeepSeek, and RL Reasoning: Long-Form Analysis Notes
Although it’s quite long, The Short Case for Nvidia Stock is a fascinating read. Also, agents are not happening yet.
Open-R1 and TinyZero: Early DeepSeek R1 Reproductions
Open-R1 — Hugging Face started to reproduce DeepSeek R1 in the open, and discussed the R1 technical report in a recorded YouTube video.