Data-engineering — Topic Index

Below is an index of articles grouped by topic. Click a heading to jump to the section.

Data Engineering

• Data Governance: Catalog, Lineage, and Access Control
  • Learn how to build comprehensive data governance with catalogs, lineage tracking, and access control. Includes practical implementations using Apache Atlas, Amundsen, and modern cloud solutions.
• Data Lakehouse Architecture: Complete Guide
  • Master data lakehouse architecture in 2026. Learn how to combine data lake flexibility with data warehouse reliability. Covers Delta Lake, Apache Iceberg, implementation strategies, and best practices.
• Data Mesh Implementation: Building Domain-Owned Data Products
  • A practical guide to implementing data mesh architecture and creating domain-owned data products that scale across organizations.
• Data Pipeline Orchestration: Airflow vs Prefect vs Dagster
  • Comprehensive comparison of leading data pipeline orchestration tools. Learn when to use Apache Airflow, Prefect, or Dagster, with architecture patterns, code examples, and selection criteria.
• Data Pipeline Orchestration: Complete Guide
  • Master data pipeline orchestration with Airflow, Dagster, and Prefect. Learn to build scalable, reliable ETL pipelines, manage dependencies, and implement best practices for data workflows.
• Data Quality: Validation, Monitoring, and Observability
  • Learn how to build robust data quality systems with validation frameworks, monitoring solutions, and observability practices. Includes code examples using Great Expectations, dbt, and custom solutions.
• Data Warehouse Modernization: From Legacy Systems to Cloud-Native Architecture
  • A comprehensive guide to modernizing legacy data warehouse systems and transitioning to cloud-native architectures.
• ETL vs ELT: Modern Data Integration Patterns
  • Compare ETL and ELT approaches for modern data integration. Learn when to use each pattern, tool recommendations, and implementation strategies for cloud data warehouses.
• MLOps for Data Engineers: Machine Learning Pipeline Automation
  • Learn how to build MLOps pipelines for automating machine learning workflows. Covers model training, versioning, deployment, monitoring, and integration with data engineering systems.
• Real-time Analytics: ClickHouse, Druid, and Materialized Views
  • Learn how to build real-time analytics systems using ClickHouse, Apache Druid, and materialized views. Compare architectures, use cases, and implementation patterns.

data-engineering

• Apache Spark: Big Data Processing at Scale 2026
  • A comprehensive guide to Apache Spark for big data processing in 2026. Learn about RDDs, DataFrames, Spark SQL, optimization techniques, and building scalable data pipelines.
• Data Lakehouse: Combining Data Lake and Data Warehouse
  • A comprehensive guide to Data Lakehouse architecture, combining the flexibility of data lakes with the management features of data warehouses. Learn about Delta Lake, Apache Iceberg, Hudi, ACID transactions, and time travel.
• Data Mesh: Decentralized Data Architecture 2026
  • A comprehensive guide to Data Mesh architecture in 2026, a decentralized approach to data management that treats data as a product. Learn about domain ownership, data as a product, self-serve platform, and federated governance.
• Stream Processing with Kafka and Flink
  • A comprehensive guide to stream processing using Apache Kafka and Apache Flink. Learn about event streaming, exactly-once semantics, windowing, and building real-time data pipelines.

Data-Engineering

• Change Data Capture (CDC) Complete Guide
  • Master Change Data Capture (CDC) techniques for real-time data integration: Debezium, Kafka Connect, implementation patterns, and best practices.
• Data Catalog Implementation Guide
  • Build and implement a data catalog: metadata management, discovery, governance, and business glossary. Tools, architectures, and best practices for 2026.
• Data Mesh Implementation Complete Guide
  • Understanding data mesh architecture, implementing domain-oriented data ownership, and building federated data platforms.
• Data Quality Management Complete Guide
  • Building comprehensive data quality programs including validation frameworks, monitoring systems, and remediation processes.
• Data Science Career Guide: From Beginner to Professional
  • Complete roadmap for building a data science career including skills required, learning path, portfolio building, job search strategies, and salary expectations for 2026.
• Introduction to Natural Language Processing
  • Explore Natural Language Processing fundamentals including text preprocessing, sentiment analysis, transformers, and building NLP applications.
• Introduction to Time Series Analysis
  • Learn time series analysis fundamentals including forecasting methods, decomposition, stationarity, and building predictive models for temporal data.
• Machine Learning Operations: MLOps Fundamentals
  • Learn MLOps fundamentals including model deployment, versioning, monitoring, and building reliable ML pipelines in production.
• Privacy-Preserving Machine Learning: Techniques and Implementation
  • Learn privacy-preserving ML techniques including federated learning, differential privacy, secure multi-party computation, and homomorphic encryption.
• Understanding Big Data Technologies
  • Learn big data fundamentals including Hadoop, Spark, distributed computing, data lakes, and processing massive datasets at scale.
• Understanding Neural Networks and Deep Learning
  • Master neural networks and deep learning fundamentals including perceptrons, backpropagation, CNNs, RNNs, and building neural network applications.

If you find missing articles or inaccurate groupings, run ./scripts/update_index.py with appropriate flags.