Introduction
GitOps is revolutionizing how teams manage infrastructure and deployments. By using Git as the single source of truth, you get version control, code review, and audit trails for your entire system.
This guide covers GitOps best practices: from core concepts to implementation patterns.
What is GitOps?
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ GITOPS WORKFLOW โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ โ
โ โโโโโโโโโโโ โโโโโโโโโโโ โโโโโโโโโโโ โโโโโโโโโโโ โ
โ โ Developer โโโโโโถโ Git โโโโโโถโ CI/CD โโโโโโถโCluster โ โ
โ โ Commits โ โ Repositoryโ โ Pipeline โ โ โ โ
โ โโโโโโโโโโโ โโโโโโโโโโโ โโโโโโโโโโโ โโโโโโโโโโโ โ
โ โ
โ Git is the source of truth โ
โ Changes are declarative โ
โ Automated synchronization โ
โ Drift detection and correction โ
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
Core Principles
The Four Pillars
| Principle | Description |
|---|---|
| Declarative | Define desired state, not procedures |
| Versioned | All changes in Git history |
| Automated | Continuous reconciliation |
| Auditable | Code review for all changes |
Repository Structure
Monorepo vs Polyrepo
# Monorepo Structure
repository/
โโโ apps/
โ โโโ frontend/
โ โ โโโ base/
โ โ โ โโโ deployment.yaml
โ โ โ โโโ service.yaml
โ โ โโโ staging/
โ โ โ โโโ kustomization.yaml
โ โ โโโ production/
โ โ โโโ kustomization.yaml
โ โโโ backend/
โ โโโ ...
โโโ infrastructure/
โ โโโ cluster/
โ โ โโโ base/
โ โ โโโ addons/
โ โโโ networking/
โโโ config/
โโโ argocd/
โโโ sealed-secrets/
# Polyrepo Structure
# repositories:
# โโโ app-frontend/
# โ โโโ src/
# โ โโโ manifests/ (GitOps here)
# โโโ app-backend/
# โ โโโ src/
# โ โโโ manifests/ (GitOps here)
# โโโ platform-infra/
# โโโ terraform/
# โโโ manifests/ (GitOps here)
GitOps Tools
ArgoCD
# ArgoCD Application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-app
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/org/repo
targetRevision: HEAD
path: apps/my-app/overlays/production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
Flux
# Flux GitRepository
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: my-app
namespace: flux-system
spec:
url: https://github.com/org/repo
ref:
branch: main
---
# Flux Kustomization
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: my-app
namespace: flux-system
spec:
sourceRef:
kind: GitRepository
name: my-app
path: ./deploy/production
prune: true
selfHeal: true
Kustomize for Environment Management
Base Configuration
# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: myapp:latest
ports:
- containerPort: 8080
Environment Overlays
# staging/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
- patch: |-
- op: replace
path: /spec/replicas
value: 2
target:
kind: Deployment
configMapGenerator:
- name: app-config
literals:
- ENVIRONMENT=staging
- LOG_LEVEL=debug
# production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
patches:
- patch: |-
- op: replace
path: /spec/replicas
value: 5
target:
kind: Deployment
configMapGenerator:
- name: app-config
literals:
- ENVIRONMENT=production
- LOG_LEVEL=info
CI/CD Integration
GitHub Actions
# .github/workflows/deploy.yaml
name: Deploy to Cluster
on:
push:
branches: [main]
paths:
- 'apps/**'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Kustomize
uses: kymckay/action-kustomize@v1
- name: Login to Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and Push
run: |
docker build -t myapp:${{ github.sha }} .
docker push myapp:${{ github.sha }}
- name: Update Manifests
run: |
cd apps/my-app/overlays/production
kustomize edit set image myapp=myapp:${{ github.sha }}
- name: Commit Changes
run: |
git config --local user.email "[email protected]"
git config --local user.name "github-actions"
git add -A
git commit -m "Deploy: ${{ github.sha }}"
git push
Handling Secrets
Sealed Secrets
# Install sealed secrets
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: database-credentials
spec:
encryptedData:
username: AgBy... # kubeseal generated
password: AgBy... # kubeseal generated
template:
metadata:
labels:
app: myapp
# Generate sealed secret
kubeseal --format yaml --namespace production \
< secret.yaml > sealed-secret.yaml
External Secrets Operator
# External Secret
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: database-credentials
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secrets-manager
kind: SecretStore
target:
name: db-credentials
creationPolicy: Owner
data:
- secretKey: username
remoteRef:
key: production/db-credentials
property: username
- secretKey: password
remoteRef:
key: production/db-credentials
property: password
Drift Detection
Monitoring Drift
# Drift detection script
import subprocess
import json
def check_drift():
"""Check for drift between Git and cluster"""
# Get Git state
git_manifests = subprocess.run(
["kustomize", "build", "./apps/production"],
capture_output=True,
text=True
)
# Get cluster state
cluster_manifests = subprocess.run(
["kubectl", "get", "-o", "json", "-A", "all"],
capture_output=True,
text=True
)
# Compare (simplified)
if git_manifests.stdout != cluster_manifests.stdout:
print("โ ๏ธ Drift detected!")
return True
print("โ
No drift detected")
return False
Best Practices
Good: Small, Frequent Commits
# Good: Single change per commit
# Commit 1: Update replica count
# Commit 2: Change environment variable
# Commit 3: Add new environment
Bad: Large Commits
# Bad: Multiple changes at once
# Commit: Update everything
# - replicas
# - image
# - env vars
# - network policy
Good: Review Process
# Required reviewers for production
PROTECTED_BRANCHES = {
"main": {
"required_reviewers": 2,
"required_checks": ["lint", "test", "deploy-preview"]
},
"production": {
"required_reviewers": 3,
"required_checks": ["lint", "test", "security-scan", "deploy-preview"]
}
}
Rollback Strategies
ArgoCD Rollback
# Rollback via CLI
argocd app rollback my-app 2
# Rollback via UI
# 1. Go to Application
# 2. Click on revision
# 3. Click "Rollback"
Git Revert
# Revert production change
git revert abc123
git push origin main
# ArgoCD will sync the revert
Conclusion
GitOps transforms infrastructure management:
- Single source of truth: Git is the only source
- Declarative: Define desired state
- Automated reconciliation: Continuous sync
- Audit trail: Full history in Git
Comments