Kimi K2: A Trillion-Parameter Open-Source LLM
By Ji Zhi Liuin Large Language Models
Explore Kimi K2, the 1.04T parameter open-source MoE model. Our deep dive covers its MuonClip optimizer, agentic AI training, and benchmark performance.
Search Articles
Search results for “Model Training”
Found 13 articles
Best Vector Databases for RAG 2025: Milvus vs Pinecone vs Chroma (10M Vector Benchmarks & Pricing)
By Yi Yanin Technology
Benchmarked 4 vector databases with 10M vectors: Milvus wins on cost ($500/mo vs $1,200), Pinecone on ease (zero-ops). Real latency tests, throughput metrics, scalability comparisons. Includes migration guide & Python code.
Document Chunking for RAG: 9 Strategies Tested (70% Accuracy Boost 2025)
By Ai Yun Shuin Technology
Boost RAG accuracy by 70%: We tested 9 chunking strategies (fixed-size, recursive, semantic). Semantic chunking won. Optimal: 256-512 tokens, 10-20% overlap. Includes Python/LangChain code and production tips.
30x Faster LLM RL Training: The Checkpoint-Engine Story
By Qing Ke Aiin Technology
Discover how the Checkpoint-Engine achieves a 30x speedup in LLM RL training. Learn about its innovative approach to parameter updates for large-scale RL.
Build a Llama-Style MoE Model From Scratch (Part 2)
By Zheng Liin Technology
Learn how to train a language model with this PyTorch training loop guide. Explore text generation, the AdamW optimizer, and Mixture of Experts models.
GRPO Training Pipeline: SFT to RL for Better Reasoning
By Ning Si Aiin Technology
Learn to implement a full GRPO training pipeline. This guide covers Supervised Fine-Tuning (SFT) with cold-start data, CoT prompting, and the GRPOTrainer.
Knowledge Distillation: Shrink GPT-4 to 10x Smaller (95% Accuracy - 2025 Guide)
By Chen Jin Shi Xue Aiin Technology
Compress LLMs 10-100x smaller using knowledge distillation. Learn teacher-student training, temperature scaling (T=3-5), soft targets. DistilBERT case: 40% smaller, 60% faster, 97% accuracy. Complete tutorial.
AI Infrastructure: 6 Components That Make or Break Agents (40-70% Cost Cut - 2025)
By Pingxingjiluin Technology
AI agents fail without proper infrastructure: 80% effort should go to infra, not models. Build 6-component pipeline: data pipelines, vector DBs (Pinecone/Weaviate), model serving, orchestration, monitoring, security. Cut costs 40-70% with optimization.
No-Code LLaMA 3 Fine-Tuning: 3 Steps with LLaMA Factory (2025)
By Number in the Moutainin Technology
Fine-tune LLaMA 3 with zero coding in 3 steps using LLaMA Factory WebUI. Save 80% GPU memory with QLoRA on RTX 3090/4090. Beginner-friendly tutorial with CUDA setup. Supports 100+ models.
PyTorch Memory Snapshot: Debug OOM Errors (GPU Memory Guide - 2025)
By Pandain Technology
Monitoring **PyTorch GPU memory usage** during model training can be perplexing. To demystify this, we'll dive into the **PyTorch memory snapshot** tool, a powerful utility for detailed **GPU memory ...
Baidu ERNIE 4.5: Multimodal Model Training & Fine-Tuning
By Alexin Technology
# Baidu ERNIE 4.5: Advancements in Multimodal Large Language Models
Baidu's ERNIE 4.5 marks a major leap in artificial intelligence, especially in the development of **multimodal large language mode...
Train 671B DeepSeek V3: RLHF Guide (10x Faster with GRPO - 2025)
By Alexin Technology
Master training 671B parameter LLMs with RL. Solve 5 critical challenges: Megatron vs FSDP, memory offloading, weight conversion, 1000+ GPU scaling. Real DeepSeek V3 workflow with GRPO achieving 10x speedup.
AI vs Traditional Infrastructure: 5 Key Differences (2025 Migration Guide)
By Alexin Technology
Migrating from traditional to AI infrastructure? Master 5 critical differences: GPU vs CPU scaling, KV Cache vs web caching, 3D parallelism vs load balancing. Real migration strategies for LLM systems in 2025.