Algorithms — Topic Index

Below is an index of articles grouped by topic. Click a heading to jump to the section.

Algorithms

• Advanced Prompt Engineering Techniques for 2026
  • Learn context engineering, Chain-of-Symbol, DSPy 3.0, agentic prompting, and cost optimization. Master techniques used by professionals for superior LLM outputs.
• Advanced RAG Optimization: Production-Ready Retrieval Systems
  • Master advanced RAG optimization techniques including chunking strategies, reranking, query transformations, and hybrid search for production AI systems.
• Agentic RAG: Autonomous Retrieval-Augmented Generation with AI Agents
  • Agentic RAG enhances traditional RAG by adding autonomous agents that can plan, reason, and dynamically retrieve information. Learn how this paradigm shift enables more intelligent and accurate AI systems.
• Algorithm Design Principles: A Practical Guide
  • A practical guide to algorithm design principles — efficiency, data structure selection, recursion vs iteration, and the major design strategies with code examples.
• Algorithm Learning Resources: Complete Guide for 2026
  • A curated list of valuable resources for learning algorithms and data structures, from visualizations to competitive programming.
• Autoencoders and Variational Autoencoders: Unsupervised Learning Fundamentals
  • Comprehensive guide to Autoencoders and VAEs - neural network architectures for unsupervised learning, dimensionality reduction, and generative modeling in 2026.
• Barnes-Hut Algorithm: Efficient N-Body Simulation
  • Learn about the Barnes-Hut algorithm for efficient N-body simulations. Understand how quadtrees and octrees reduce force calculations from O(n²) to O(n log n) for physics simulations, galaxy modeling, and particle systems.
• Chain of Thought Distillation: Teaching Small Models to Reason
  • Explore how Chain of Thought distillation transfers reasoning capabilities from large language models to compact student models.
• Chain of Thought Reasoning: Advanced Techniques for LLM Reasoning
  • CoT prompting achieves up to 10% accuracy improvement. Learn entropy-guided CoT, latent visual CoT, cognitive CoT, and multi-level frameworks for enhanced reasoning.
• Chain of Verification CoVe: Reducing LLM Hallucinations Through Self-Verification
  • Chain of Verification (CoVe) enables LLMs to verify their own outputs against retrieved facts. Learn how this self-critique mechanism dramatically reduces hallucinations and improves reliability.
• Community Detection Algorithms: Graph Clustering Methods
  • Comprehensive guide to Community Detection Algorithms - methods for discovering communities in networks, including Louvain, Label Propagation, spectral clustering, and applications in 2026.
• Competitive Programming: Getting Started
  • Introduction to competitive programming including algorithms, data structures, contest platforms, and preparation strategies for technical interviews.
• Continual Learning Algorithms: Overcoming Catastrophic Forgetting
  • Master continual learning algorithms that enable AI systems to acquire new knowledge while retaining previously learned information without catastrophic forgetting.
• Contrastive Learning: Self-Supervised Representation Learning
  • Master contrastive learning algorithms that learn powerful representations by comparing positive and negative pairs, enabling deep learning without labeled data.
• Convolutional Neural Networks: The Foundation of Computer Vision
  • Comprehensive guide to CNNs covering convolutional layers, pooling, architectures like ResNet and EfficientNet, and their applications in computer vision
• Data Structures for Interviews: Essential Knowledge
  • Master essential data structures for technical interviews including arrays, linked lists, trees, graphs, and hash tables with implementation examples and common patterns.
• Differential Privacy in Machine Learning
  • Comprehensive guide to Differential Privacy in ML - mathematical foundations, privacy-preserving algorithms, DP-SGD, and practical implementation in 2026.
• Diffusion Models: The Mathematics of Generative AI
  • Comprehensive guide to diffusion models covering DDPM, stable diffusion, image generation, and the mathematical foundations behind AI art in 2026
• Direct Preference Optimization DPO: Simplifying LLM Alignment
  • Direct Preference Optimization eliminates the complexity of RLHF by directly optimizing against human preferences. Learn how DPO replaces PPO with a simple classification loss.
• Divide and Conquer: Breaking Down Problems to Build Elegant Solutions
  • A comprehensive guide to divide-and-conquer algorithms - learn how this powerful paradigm breaks complex problems into manageable pieces, with practical examples including merge sort, quicksort, and binary search.
• Efficient Long-Context LLM: Strategies for Million-Token Contexts
  • Learn efficient long-context techniques: sliding window attention, hierarchical methods, sparse attention, KV cache optimization, and dynamic sparse attention for on-device deployment.
• Energy-Based Models (EBMs): A Unified Framework for Learning and Inference
  • Explore energy-based models as a flexible alternative to probabilistic models for generative modeling, classification, and constraint satisfaction in modern AI systems.
• Fast Polynomial Multiplication with FFT: From O(n²) to O(n log n)
  • Learn how the Fast Fourier Transform revolutionizes polynomial multiplication, reducing complexity from O(n²) to O(n log n). Explore the math, algorithm, and practical applications of FFT in signal processing and beyond.
• Federated Learning: Privacy-Preserving Machine Learning
  • Comprehensive guide to Federated Learning - enabling machine learning models to train on distributed data without centralizing sensitive information in 2026.
• FlashAttention-3: Next-Generation Transformer Optimization
  • FlashAttention-3 achieves 75% FLOP utilization on NVIDIA H100 GPUs through asynchronous computation and low-precision techniques. Learn the revolutionary optimizations.
• Function Calling and Tool Use: Building Agentic LLM Systems
  • Learn ReAct patterns, function calling protocols, tool orchestration, and building AI agents that can reason, act, and observe in loops.
• Function Calling: Enabling LLMs to Use External Tools and APIs
  • Function Calling transforms LLMs from passive text generators into active problem solvers that can use external tools, APIs, and compute resources. Learn the mechanisms, implementations, and real-world applications.
• Gated Linear Attention: Efficient Transformers with Data-Dependent Gating
  • GLA combines linear attention efficiency with learned gating for expressivity. Learn how it achieves RNN-like inference with transformer-like training.
• Generative Adversarial Networks: The Game Theory of Deep Learning
  • Comprehensive guide to GANs covering adversarial training, generator/discriminator architectures, style transfer, and applications in generative AI
• Genetic Algorithms: Evolutionary Optimization
  • Comprehensive guide to Genetic Algorithms - evolutionary computation methods inspired by natural selection, including selection, crossover, mutation, and practical applications in 2026.
• Gradient Descent and Optimization Algorithms
  • Comprehensive guide to Gradient Descent optimization algorithms - from basic SGD to Adam, including learning rate scheduling, momentum, and adaptive methods in 2026.
• Graph Algorithms: Implementation and Applications
  • Comprehensive guide to graph algorithms including traversal, shortest path, matching, and their practical applications in software development.
• Graph Embedding Methods: node2vec, DeepWalk, and LINE
  • Comprehensive guide to Graph Embedding methods - transforming graph structures into dense vectors using DeepWalk, node2vec, LINE, and modern techniques in 2026.
• Graph Neural Networks: Deep Learning on Graph Structures in 2026
  • Comprehensive guide to Graph Neural Networks (GNNs), covering message passing, architectures like GCN and GAT, and applications in recommendation systems, molecular discovery, and social networks
• GraphRAG: Graph-Based Retrieval-Augmented Generation
  • Master GraphRAG algorithms that combine knowledge graphs with LLMs for improved retrieval, reasoning, and question answering over structured data.
• GraphRAG: Knowledge Graph Enhanced Retrieval-Augmented Generation
  • GraphRAG achieves 85%+ accuracy vs 70% for vector-only RAG. Learn knowledge graph construction, hybrid retrieval, entity extraction, and multi-hop reasoning for enterprise AI.
• GRPO: Group Relative Policy Optimization DeepSeek’s RL Breakthrough
  • GRPO eliminates the critic network from reinforcement learning, using group-based relative rewards. Learn how DeepSeek-R1 achieved reasoning breakthroughs with this efficient algorithm.
• How to Write Algorithms More Easily
  • A practical guide on how to approach algorithm design by thinking and planning on paper before writing any code, leading to more accurate and efficient solutions.
• Knapsack Problem in Ruby
  • 背包问题的Ruby实现
• Knowledge Distillation: LLM Compression and Efficient Transfer
  • Distill large LLMs into compact students. Learn teacher-student frameworks, distillation techniques, temporal adaptation, low-rank feature distillation, and deployment strategies.
• Knowledge Distillation: Model Compression Techniques
  • Master knowledge distillation algorithms that transfer knowledge from large teacher models to compact student models for efficient deployment.
• KV Cache Eviction Strategies for Long-Context LLM Inference
  • Efficient KV cache management is critical for long-context inference. Learn about eviction strategies, memory optimization techniques, and algorithms that enable processing millions of tokens.
• LLM Quantization: GPTQ, AWQ, and GGUF for Efficient Deployment
  • Quantization reduces LLM memory by 4-8x with minimal quality loss. Learn GPTQ, AWQ, GGUF formats, quantization levels, and deployment strategies for efficient inference.
• Long-Context Language Models: Scaling to Million-Token Contexts
  • Infini-attention enables infinite context with bounded memory. Learn context extension techniques, hierarchical methods, and infrastructure for million-token windows.
• Meta-Learning: Algorithms for Learning to Learn
  • Comprehensive guide to Meta-Learning and Few-Shot Learning - algorithms that enable AI systems to learn new tasks quickly with minimal examples in 2026.
• Mixture of Depths: Dynamic Computation in Transformers
  • MoD dynamically adjusts computation per token, enabling 2-4x speedup in long-sequence processing. Learn how DeepSeek uses this technique for efficient inference.
• Mixture of Experts (MoE): Scaling Large Language Models Efficiently
  • Master Mixture of Experts algorithms that enable massive model capacity through sparse activation, powering systems like GPT-4 with efficient computation.
• Model Quantization: LLM Compression Techniques
  • Master model quantization algorithms that compress large language models to 4-bit, 2-bit or lower while maintaining accuracy, enabling efficient deployment.
• Monte Carlo Tree Search: Decision-Making Through Simulation
  • Explore Monte Carlo Tree Search algorithm, its applications in game AI, and how it powers systems like AlphaGo.
• Multi-Agent AI Systems: Collaboration and Coordination Frameworks
  • 40% of enterprise apps will use AI agents by 2026. Learn agent protocols (MCP, A2A, ACP), orchestration patterns, CrewAI, LangGraph, and enterprise deployment strategies.
• Multi-Agent Systems: Collaborative AI Agent Architecture
  • Master multi-agent system algorithms that enable multiple AI agents to collaborate, compete, and solve complex problems through distributed intelligence.
• Multi-Head Latent Attention MLA: DeepSeek’s Memory Optimization
  • Multi-Head Latent Attention reduces KV cache by 93% while maintaining performance. Learn how DeepSeek revolutionized transformer memory efficiency with this innovative technique.
• Multi-Token Prediction MTP: Accelerating LLM Generation
  • Multi-Token Prediction enables large language models to predict multiple tokens simultaneously, dramatically improving inference speed. Learn how DeepSeek and Meta pioneered this technique.
• Neural Architecture Search: Automated Deep Learning Model Design
  • NAS automates neural network architecture discovery using RL, evolutionary algorithms, and differentiable methods. Learn how to reduce 80% of ML engineering effort.
• Neuromorphic Computing: Brain-Inspired Algorithms and Hardware
  • Exploring neuromorphic computing that mimics brain architecture, covering spiking neural networks, event-based processing, and the future of energy-efficient AI in 2026
• P vs NP: The Million-Dollar Problem That Could Change Everything
  • Explore the P vs NP problem, one of computer science’s greatest unsolved mysteries. Learn why this complexity theory question matters for cryptography, optimization, and the future of computing. Includes real-world examples, NP-completeness, and implications for security and AI.
• P vs. NP: Understanding the Most Important Problem in Computer Science
  • A beginner-friendly guide to P problems, NP problems, NP-complete, and NP-hard. Learn through real-world examples like Sudoku, Traveling Salesman, and why this million-dollar question matters.
• PagedAttention: Memory Optimization Revolution for LLM Inference
  • PagedAttention brings operating system concepts to AI memory management, enabling 24x better throughput for LLM serving. Learn how vLLM achieves this breakthrough.
• PageRank Algorithm: Link Analysis and Graph Ranking
  • Comprehensive guide to PageRank - Google’s foundational algorithm for ranking web pages and graph nodes, including implementation, variations, and applications in 2026.
• Particle Swarm Optimization: Swarm Intelligence in Practice
  • Comprehensive guide to Particle Swarm Optimization (PSO) - a swarm intelligence algorithm inspired by bird flocking, including variants, implementation, and applications in 2026.
• Prompt Caching for LLMs: Reducing Latency and Cost at Scale
  • Learn how prompt caching works in large language models, its implementation strategies, and how it reduces inference costs by up to 90%.
• Quad Tree (C++ Implementation)
• Raft Consensus Algorithm: Complete Implementation Guide 2026
  • Master the Raft consensus algorithm with comprehensive coverage of leader election, log replication, safety guarantees, and practical implementation patterns for distributed systems.
• Recurrent Neural Networks and LSTM: Processing Sequential Data
  • Comprehensive guide to RNNs, LSTM, and GRU covering sequence modeling, vanishing gradients, and applications in NLP and time series
• Reinforcement Learning Algorithms: From Q-Learning to Deep Q-Networks
  • A comprehensive guide to reinforcement learning algorithms covering policy gradients, DQN, Actor-Critic methods, and modern RL approaches for complex decision-making in 2026
• Reinforcement Learning from Human Feedback: Aligning AI with Human Preferences
  • RLHF aligns LLMs with human values through preference learning. Learn the 3-stage pipeline, reward modeling, PPO optimization, and how DPO simplifies alignment.
• Ring Attention and USP: Scaling Transformer Context to Millions of Tokens
  • Ring Attention and Unified Sequence Parallelism enable processing millions of tokens by distributing attention across multiple GPUs. Learn how these techniques overcome context length limitations.
• RWKV: Receptance Weighted Key Value for Efficient Language Modeling
  • RWKV combines transformer parallel training with RNN efficient inference. Learn how this architecture achieves linear scaling while matching transformer performance.
• S-Mamba: Scalable Selective State Space Models for Modern AI
  • S-Mamba extends the Mamba architecture with scalable selective state space models. Learn how this innovation enables efficient processing across language, vision, and time series applications.
• Self-Consistency Decoding: Ensemble Reasoning in LLMs
  • Learn how self-consistency decoding improves LLM reasoning by sampling multiple reasoning paths and selecting the most consistent answer.
• Self-Consistency in LLM Reasoning: Ensemble Methods for Reliable Outputs
  • Self-consistency improves reasoning by sampling multiple paths and voting. Learn confidence-aware methods, structured frameworks, and efficient aggregation for reliable LLM outputs.
• Self-Reflection in LLMs: Enabling Models to Critique and Improve Their Own Outputs
  • Self-Reflection enables LLMs to examine their own outputs, identify errors, and revise responses. Learn how this meta-cognitive capability is transforming AI reliability and reasoning.
• Simulated Annealing: Probabilistic Optimization
  • Comprehensive guide to Simulated Annealing - a probabilistic optimization algorithm inspired by metallurgy, including Metropolis criterion, cooling schedules, and applications in 2026.
• Soft Mixture of Experts SoftMoE: Beyond Hard Expert Selection
  • SoftMoE transforms sparse MoE by using differentiable soft assignments instead of hard routing. Learn how this approach achieves the best of both worlds: the efficiency of sparse computation with the training stability of dense models.
• Sparse Attention Algorithms: Efficient Transformers at Scale
  • Master sparse attention algorithms that reduce Transformers quadratic complexity to linear, enabling efficient processing of long sequences in modern AI systems.
• Sparse Mixture of Experts: Scaling Language Models Efficiently
  • SMoE activates only a subset of parameters per token, enabling massive model capacity with constant compute. Learn about routing mechanisms, load balancing, and deployment.
• Speculative Decoding: Lossless LLM Inference Acceleration
  • Master speculative decoding algorithms that accelerate LLM inference by 2-3x using draft verification, enabling faster text generation without quality loss.
• State Space Models (SSM) and Mamba: The Post-Transformer Architecture
  • Explore state space models and Mamba architecture—a linear-time sequence modeling approach that challenges Transformers with efficient long-range dependency handling.
• State Space Models: Mamba and the Post-Transformer Architecture
  • Mamba-3 achieves 4% better performance than Transformers with 7x faster inference. Learn SSM foundations, selective mechanisms, and hybrid architectures for efficient inference.
• Transformer Architecture: Attention Mechanisms Explained
  • Comprehensive guide to Transformer architecture, attention mechanisms, self-attention, and how they revolutionized natural language processing and beyond in 2026
• Tree of Thoughts (ToT): Advanced Reasoning Algorithms
  • Master Tree of Thoughts and related reasoning algorithms that enable LLMs to explore multiple reasoning paths, backtrack, and find optimal solutions.
• Two-Phase Commit (2PC) Protocol: Complete Guide for Distributed Systems
  • A comprehensive guide to the Two-Phase Commit (2PC) protocol, covering implementation, code examples, failure scenarios, and best practices for distributed transaction management.
• Understanding Big O Notation
  • Master Big O notation for analyzing algorithm efficiency including time and space complexity with examples and practical applications.
• Understanding Conflict-Driven Clause Learning (CDCL): The Algorithm That Revolutionized SAT Solving
  • Master CDCL algorithm with 20+ code examples, implementation patterns, and real-world applications. Learn conflict analysis, clause learning, and modern SAT solving techniques.

algorithms

• Backtracking Algorithms: Solving Complex Problems Systematically
  • A comprehensive guide to backtracking algorithms - understand the pattern and solve classic problems like N-Queens, Sudoku, and permutations
• Bit Manipulation Tricks: Essential Techniques for Interviews
  • A comprehensive guide to bit manipulation - understand bitwise operations, tricks, and how to solve problems efficiently
• Dynamic Programming Patterns: Complete Guide
  • Master dynamic programming with common patterns including fibonacci, knapsack, LIS, LCS, and more. Learn top-down vs bottom-up approaches with practical examples.
• Sorting Algorithms Deep Dive: Complete Guide
  • Master all major sorting algorithms with implementations, time complexities, and when to use each. Includes practical examples and interview tips.
• Trie Implementation: Prefix Trees for Efficient String Operations
  • A comprehensive guide to Trie data structure - understand implementation and solve problems like autocomplete and prefix matching

Data-Structures

• String Matching Algorithms: KMP, Rabin-Karp, and Beyond
  • Master efficient string matching algorithms for searching patterns in text, from basic approaches to advanced techniques.

MPI

• Understanding MPI: Message Passing Interface for Parallel Computing

If you find missing articles or inaccurate groupings, run ./scripts/update_index.py with appropriate flags.