Latest Articles

Dive deep into the world of Artificial Intelligence with our curated collection of articles, covering the latest breakthroughs and insights from leading researchers and engineers.

October 11, 2025Technology

Document Chunking for RAG: A Practical Guide

Boost your RAG system's performance with our guide to document chunking. Explore strategies from recursive to semantic chunking with Python & LangChain code.

Ai Yun Shu

document chunking RAG chunking chunking strategies+3 more

September 24, 2025Technology

LangChain vs. LangGraph vs. LangSmith: Which to Use?

Confused by the LangChain ecosystem? Learn the key differences between LangChain, LangGraph, and LangSmith to choose the right tool for your LLM stack.

Shen Du Xue Xi Shi Jue

LangChain vs LangGraph LangChain LangGraph+4 more

September 23, 2025Technology

How to Add Special Tokens to LLMs Safely

Learn how to add special tokens to LLMs during fine-tuning without causing catastrophic forgetting. Our guide covers smart initialization and PEFT/LoRA.

Bao Bao Suan Fa Bi Ji

add special tokens to LLM LLM fine-tuning catastrophic forgetting+1 more

September 20, 2025Technology

What is Context Engineering? Improve LLM Accuracy

Learn what Context Engineering is and how it improves LLM accuracy. Discover the 3 key stages in RAG systems for better, more reliable AI-generated outputs.

Quan Ge Tan Ai

Context Engineering Retrieval-Augmented Generation (RAG)Large Language Model (LLM)+1 more

September 19, 2025Technology

Top 10 Underground AI Tools of 2025

Discover the top 10 AI tools thriving in the underground economy. Based on real API data, we reveal the AI coding agents and role-playing apps developers use.

Shi Zi Lu Kou Crossing

underground AI economy top AI tools AI coding agents+1 more

September 18, 2025Technology

AI Programming Assistant: The Future of Coding

Explore the future of AI programming assistants. Learn about a local-first, secure AI coding tool that automates refactoring, testing, and deployment from your CLI.

Lao Yang Ai Gao Sheng Huo

AI programming assistant AI coding tools command line AI+2 more

September 18, 2025Technology

Top RAG Frameworks 2025: A Complete Guide

Explore the top RAG frameworks of 2025. Compare production-ready tools like Haystack & RAGFlow with cutting-edge research to build powerful AI applications.

Chen Jin Shi Xue Ai

RAG frameworks Retrieval-Augmented Generation RAG applications+1 more

September 17, 2025Technology

LLM Agents Explained: A Visual Guide to AI Agents

Explore the architecture of LLM agents. This visual guide covers memory, tools, planning, and multi-agent systems like AutoGen. Learn how AI agents work.

Lao Liu Shuo Nlp

LLM agents LLM agent architecture multi-agent systems+1 more

September 17, 2025Technical

RAG Evaluation 101: From Recall@K to Answer Faithfulness

# RAG Evaluation 101: From Recall@K to Answer Faithfulness Retrieval-Augmented Generation (RAG) systems combine an information retriever with a generative model to produce answers grounded in extern...

AI Author

RAG Evaluation Retrieval-Augmented Generation+3 more

September 16, 2025Technology

GRPO-RoC: Better Training for Tool-Augmented AI

Learn how outcome-based rewards teach AI models bad habits. Discover GRPO-RoC, a training method that improves AI reasoning by curating high-quality data.

Qing Ke Ai

GRPO-RoC tool-augmented models AI training+1 more

September 15, 2025Technology

30x Faster LLM RL Training: The Checkpoint-Engine Story

Discover how the Checkpoint-Engine achieves a 30x speedup in LLM RL training. Learn about its innovative approach to parameter updates for large-scale RL.

Qing Ke Ai

LLM RL training parameter update checkpoint-engine+1 more

September 13, 2025Technology

Multi-head Latent Attention (MLA) Explained

Learn about Multi-head Latent Attention (MLA) and how it improves on Multi-Query Attention (MQA). Discover Matrix Absorption and its impact on performance.

Chen Jin Shi Xue Ai

Multi-head Latent Attention MLA Matrix Absorption+1 more

September 12, 2025Technology

Build an iOS App with AI: A Vibe Coding Guide

Learn how to build an iOS app with AI. This Vibe Coding guide covers AI pair programming, using v0 for UI, and leveraging generative AI for faster development.

Shan Gui Er

Vibe Coding build iOS app with AI AI coding assistant+1 more

September 11, 2025Technology

Alibaba's Qwen3-Next: A Deep Dive into its MoE Arch

Explore the architecture of Alibaba's Qwen3-Next, a powerful large language model. Learn about its Mixture of Experts (MoE) design and performance.

Zheng Li

Qwen3-Next Alibaba Qwen3-Next large language model+1 more

September 10, 2025Technology

What is Agentic RAG? A Complete Guide

Learn what Agentic RAG is and how it improves on traditional RAG. This guide shows you how to build an adaptive system with LangGraph and the Qwen model.

Ning Si Ai

Agentic RAG LangGraph Retrieval-Augmented Generation+1 more

September 9, 2025Technology

Build a Llama-Style MoE Model From Scratch (Part 2)

Learn how to train a language model with this PyTorch training loop guide. Explore text generation, the AdamW optimizer, and Mixture of Experts models.

Zheng Li

train language model pytorch training loop text generation+1 more

September 8, 2025Technology

Build a Llama-Style MoE Model From Scratch (Part 1)

Learn how to build a Llama-style MoE language model from scratch. This guide covers the Mixture of Experts architecture, tokenization, and model setup.

Zheng Li

MoE language model Llama-style MoE

September 7, 2025Technology

Supervised Fine-Tuning: A Guide to LLM Reasoning

Learn the complete Supervised Fine-Tuning (SFT) pipeline to enhance LLM reasoning. This guide covers the DeepSeek R1 process, from SFT to knowledge distillation.

Ning Si Ai

Supervised Fine-Tuning SFT pipeline language model fine-tuning+1 more

September 5, 2025Technology

GRPO Training Pipeline: SFT to RL for Better Reasoning

Learn to implement a full GRPO training pipeline. This guide covers Supervised Fine-Tuning (SFT) with cold-start data, CoT prompting, and the GRPOTrainer.

Ning Si Ai

GRPO training pipeline Supervised Fine-Tuning (SFT)model reasoning+1 more

September 4, 2025Technology

DeepSeek-Coder-V2's Reward Model Explained

Explore the 5 core reward functions powering DeepSeek-Coder-V2. Learn how its modular reward model for accuracy, reasoning, and format shapes AI behavior.

Ning Si Ai

DeepSeek-Coder-V2 reward model reward function+1 more

September 3, 2025Technology

Replicate DeepSeek R1 with RL: A Guide

Learn to replicate the DeepSeek R1 training process. This guide covers building a reinforcement learning pipeline from scratch using GRPO for advanced LLM reasoning.

Ning Si Ai

DeepSeek R1 Reinforcement Learning Group Relative Policy Optimization+1 more

September 2, 2025Technology

Boost LLM Goodput: Prefill-Decode Separation

Learn how Prefill-Decode separation in LLM serving boosts goodput by 4.48x. Discover DistServe, a new architecture that optimizes latency and meets strict SLOs.

GiantPandaLLM

LLM serving goodput Prefill-Decode separation+1 more

September 1, 2025Technology

What is Knowledge Distillation in AI?

Learn how knowledge distillation and model temperature work to train smaller, more efficient AI models. A key technique for LLM model compression.

Chen Jin Shi Xue Ai

knowledge distillation model temperature ai model compression+1 more

August 20, 2025Technology

AI Infrastructure: The Real Engine Behind AI Agents

Struggling with AI projects? The problem isn't your models, it's your AI infrastructure. Learn why data silos & lag hold you back and how to build a better f...

Pingxingjilu

AI infrastructure AI agents data-to-AI pipeline+1 more

August 6, 2025Technology

LLaMA Factory: Easy LLM Fine-Tuning Guide

Learn to install and use LLaMA Factory to fine-tune hundreds of LLMs on your local machine. This guide covers CUDA setup, installation, and WebUI usage.

Number in the Moutain

LLaMA Factory LLM fine-tuning fine-tune LLM+1 more

August 4, 2025Technology

What Is a Transformer Model? An In-Depth Guide

Explore the Transformer model and its architecture. Learn about attention mechanisms and how Transformers power modern AI and Large Language Models (LLMs).

Quan Ge Tan Ai

Transformer model Transformer architecture Transformers AI+1 more

August 2, 2025Tools & Frameworks

Ollama Guide: Run LLMs Locally on Your Machine

A comprehensive guide to using Ollama for running large language models like Llama 3 and Mistral on your local machine. Learn installation, commands, and how to create custom models.

Jordan Lee

Ollama run LLMs locally Ollama guide+1 more

August 1, 2025Technology

What Are LLMs? A Guide to Generative AI

Discover what Large Language Models (LLMs) are and how they power Generative AI. This in-depth guide covers the Transformer architecture, prompt engineering, and more.

Quan Ge Tan Ai

Large Language Models (LLMs)Generative AI Transformer architecture+1 more

July 30, 2025Technology

Separated Architectures for LLM RL Post-Training

Explore the shift to separated architectures for RL post-training of LLMs. Learn how systems like AsyncFlow & TransferQueue solve data orchestration challenges.

Little Boji

RL post-training separated architecture LLM post-training+1 more

July 29, 2025Technology

LLM Inference on H800: A Disaggregated Architecture Guide

Explore LLM inference optimization on H800 SuperPods. Learn how a disaggregated architecture with SGLang tackles the prefill bottleneck to boost throughput.

yiakwy

LLM inference disaggregated architecture H800 SuperPod+2 more

July 28, 2025Technology

PyTorch Memory Snapshot: A Guide to GPU Usage Analysis

Monitoring **PyTorch GPU memory usage** during model training can be perplexing. To demystify this, we'll dive into the **PyTorch memory snapshot** tool, a powerful utility for detailed **GPU memory ...

Panda

PyTorch memory snapshot GPU memory analysis PyTorch memory usage+1 more

July 27, 2025Technology

SFT Flaw: A Learning Rate Tweak Unlocks LLM Potential

Discover a critical flaw in Supervised Fine-Tuning (SFT) that limits LLM performance. Learn how a simple learning rate tweak unifies SFT and DPO for a 25% gain.

Alex

Supervised Fine-Tuning (SFT)Direct Preference Optimization (DPO)LLM fine-tuning+1 more

July 26, 2025General

GraphRAG's Workflow

Unpack the powerful workflow behind GraphRAG. Learn how it transforms data into a network of nodes and edges, uses intelligent graph traversal for searching, and applies advanced metrics and metadata filters to deliver highly relevant, contextualized answers.

Technology AI Innovation+1 more

July 25, 2025Technology

First Principles of GPU Performance

This article delves into the core challenges of GPU performance, analyzing the differences between compute-bound and memory-bound operations and highlighting the issue of underutilized memory bandwidth. It further proposes strategies to maximize throughput and looks ahead to the collaborative future of CPUs and GPUs, as well as the evolution of GPU architecture, offering a first-principles perspective on understanding and optimizing GPU performance.

xiaodong gong

Technology AI Innovation+1 more

July 24, 2025Technology

Two Major Challenges in Reinforcement Learning Finally Solved by ICLR Papers

Traditional reinforcement learning models struggle with real-time applications due to "AI lag." Two ICLR 2025 papers from Mila introduce groundbreaking solutions to tackle inaction and delay regret, enabling large AI models to operate in high-frequency, dynamic environments without compromising speed or intelligence.

Alex

Technology AI Innovation+1 more

July 23, 2025Technology

Discuss the infrastructure requirements of Agentic AI.

The rise of Agentic AI places unprecedented demands on our infrastructure. This article explores the emerging software and hardware requirements, from specialized runtimes and memory services to zero-trust security models, dissecting AWS's new Bedrock AgentCore platform and discussing the future of AI infrastructure.

Alex

Technology AI Innovation+1 more

July 22, 2025Technology

From DeepSeek-V3 to Kimi K2：Eight Modern Large Language Model Architecture Designs

This article dissects the architectural evolution of modern large language models in 2025, moving beyond benchmarks to analyze the core design choices of flagship open-source models. We explore key innovations like DeepSeek-V3's Multi-Head Latent Attention (MLA) and Mixture of Experts (MoE), OLMo 2's unique normalization strategies, Gemma 3's use of sliding window attention, and Llama 4's take on MoE. By focusing on these architectural blueprints, we gain a clearer understanding of the engineering priorities shaping the future of LLMs.

Alex

Technology AI Innovation

July 21, 2025Technology

Deploying Kimi K2: Scalable MoE Model on 128 GPUs

Learn how to deploy Kimi K2, a state-of-the-art Mixture-of-Experts (MoE) model, on a massive 128 H200 GPU cluster. This guide covers the key challenges and solutions using OME and SGLang for scalable, high-performance inference, achieving 4800 tokens/second with low latency.

Alex

Kimi K2 deployment Mixture-of-Experts model OME+1 more

July 20, 2025Technology

How to Choose the Right ldmatrix in CUTLASS CuTe

Learn how to select the best ldmatrix operation in CUTLASS CuTe for high-performance GPU matrix multiplication. Optimize data movement and performance.

Alex

CUTLASS CuTe ldmatrix operation TiledMMA+1 more

July 20, 2025Technology

Optimizing TiledCopy for Memory Coalescing on NVIDIA GPUs

Unlock the full potential of your CUDA kernels by mastering memory coalescing with TiledCopy. This article dives deep into optimizing data transfers from Global to Shared Memory on NVIDIA GPUs, covering cp.async, row-major vs. column-major layouts, and cache line alignment to maximize memory bandwidth and accelerate your deep learning workloads.

Alex

TiledCopy memory coalescing cp.async+1 more

July 19, 2025Technology

Fine-Tune Qwen3 with Unsloth: Fast, Efficient AI Training

# Fine-Tuning Qwen3 with Unsloth: Step-by-Step Guide Qwen3, the latest generation of large language models, is redefining AI with advanced reasoning, instruction following, and robust multilingual s...

Alex

Qwen3 fine-tuning Unsloth LoRA+1 more

July 18, 2025Technology

Baidu ERNIE 4.5: Multimodal Model Training & Fine-Tuning

# Baidu ERNIE 4.5: Advancements in Multimodal Large Language Models Baidu's ERNIE 4.5 marks a major leap in artificial intelligence, especially in the development of **multimodal large language mode...

Alex

ERNIE 4.5 multimodal large language models Baidu+1 more

July 17, 2025Technology

MemOS: Persistent Memory for LLMs & Next-Gen AI Agents

# MemOS: Persistent Memory for LLMs & Next-Gen AI Agents ![MemOS hero image showing a brain with digital connections](/images/2025/07-july/2025-07-17-memos-persistent-memory-llms-next-gen-ai_102.jpg...

Alex

MemOS LLM memory management persistent memory for LLMs+1 more

July 16, 2025Technology

Supervised Fine-Tuning (SFT) for LLMs: A Practical Guide

Learn how Supervised Fine-Tuning (SFT) transforms LLMs from base models to chat assistants. Step-by-step guide to SFT workflow, datasets, and best practices.

Alex

Supervised Fine-Tuning LLM fine-tuning instruction-tuned model+1 more

July 15, 2025Technology

How Linear Layers Power Multi-Head Attention in Transformers

Discover how linear layers enable multi-head attention in Transformers, powering advanced NLP models with parallel processing and rich representations.

Alex

multi-head attention linear layers Transformer architecture+1 more

July 14, 2025Technology

Single vs Multi-Controller in veRL: Pathways to RL

Explore single-controller vs multi-controller in veRL, inspired by Google's Pathways, and learn their impact on distributed reinforcement learning systems.

Alex

single-controller multi-controller veRL+1 more

July 13, 2025Technology

Reinforcement Learning for LLM Reasoning: Trends & Insights

The field of artificial intelligence has seen rapid advancements in reinforcement learning for reasoning, particularly within large language models (LLMs). This article reviews influential research s...

Alex

reinforcement learning for reasoning RL-based reasoning in large language models GRPO+1 more

July 10, 2025Technology

Grok 4: xAI's Breakthrough AI Model Surpasses Benchmarks

Discover how xAI's Grok 4 sets new AI benchmarks, outperforms rivals, and introduces multi-agent systems in the race for next-gen artificial intelligence.

Alex

Grok 4 xAI large language model+1 more

July 10, 2025Technology

Qwen3 Training Pipeline: Pre-training, RL, and Distillation

## Qwen3 Training Pipeline: Pre-training, Reinforcement Learning, and Model Distillation ### Qwen3 Pre-training: Building a Robust Foundation Qwen3 training begins with a comprehensive three-stage ...

Alex

Qwen3 training Qwen3 pre-training Qwen3 reinforcement learning+1 more

July 9, 2025Technology

LLM API Market 2024: Google Leads as OpenAI Slips

## LLM API Market 2024: Key Trends and Model Leaderboard As we reach the midpoint of 2024, the competitive landscape for large language models (LLMs) is shifting rapidly. The so-called "LLM Wars" ar...

Alex

LLM API market large language models OpenRouter+1 more

July 9, 2025Technology

Training a 671B LLM with Reinforcement Learning: Key...

Discover the technical challenges and solutions in training a 671B parameter LLM with Reinforcement Learning, covering frameworks, memory, and efficiency.

Alex

671B parameter LLM Reinforcement Learning large language model training+1 more

July 8, 2025Technology

Bridging Traditional and AI Infrastructure: Key Differences

Discover how traditional infrastructure skills translate to AI infrastructure. Learn key concepts, differences, and engineering fundamentals for LLM systems.

Alex

AI infrastructure traditional infrastructure distributed systems+1 more

July 7, 2025Technology

SGLang Disaggregated LLM Inference: Architecture &...

With its impressive performance and elegant architecture, **SGLang** is rapidly establishing itself in the competitive world of **large language model (LLM) inference**. Could it be the next PyTorch,...

Alex

SGLang LLM inference disaggregated inference+2 more

July 6, 2025Technology

Direct Reinforcement Learning on Base LLMs: The Next Leap

### Why Direct Reinforcement Learning on Base Language Models is the Next Frontier Direct reinforcement learning (RL) on base language models is emerging as a transformative approach in LLM optimiza...

Alex

direct reinforcement learning base language models zero-RL+1 more

July 5, 2025Technology

How Andrew Ng Scopes Down AI Projects for Fast Progress

Learn Andrew Ng

Alex

AI project management scoping down projects developer productivity+1 more

July 3, 2025Technology

Reinforcement Learning for LLMs: An Intuitive Guide

Reinforcement learning for LLMs (large language models) is revolutionizing the field of artificial intelligence by enabling models to learn beyond the constraints of supervised learning. This article...

Alex

reinforcement learning for LLMs RL for large language models supervised learning+1 more

July 2, 2025Technology

Decoding Strategies for Large Language Models Explained

# Decoding Strategies for Large Language Models (LLMs) At the core of every large language model (LLM) is a sophisticated process for generating text. Instead of selecting words at random, the model...

yong qiang

large language model decoding strategies LLM sampling methods Greedy Search+1 more

June 30, 2025Technology

Kimi Researcher: End-to-End RL for Advanced AI Agents

## Kimi Researcher: Advancing AI Agents with End-to-End Reinforcement Learning Kimi Researcher is the flagship product of the Kimi Agent initiative, designed to revolutionize research automation thr...

Alex

Kimi Researcher end-to-end reinforcement learning AI agent+1 more

June 26, 2025Technology

Qwen3 QK-Norm: Improved On-Device AI Stability

## Qwen3 Model Family: QK-Norm and Enhanced Attention Mechanism The Qwen3 model family, Alibaba's latest large language model release, introduces a significant upgrade for on-device AI: the adoption...

Alex

Qwen3 QK-Norm attention mechanism+1 more