Introduction
Kubernetes security is a shared responsibility between cloud providers, cluster operators, and application teams. With containers powering most modern applications, understanding Kubernetes security is essential. This guide covers comprehensive security practices for production Kubernetes deployments.
Security Architecture
Defense in Depth
Layers of security:
- Cloud/Infra: VPC, IAM, network controls
- Cluster: API server, etcd, kubelet
- Container: Images, runtime, registry
- Application: RBAC, secrets, network policies
Security Principles
- Least privilege: Minimal permissions everywhere
- Defense in depth: Multiple security layers
- Assume breach: Plan for compromise
- Shift left: Security early in CI/CD
Cluster Security
API Server Security
Authentication
- Use strong auth: OIDC, LDAP, or RBAC
- Disable anonymous: Set –anonymous-auth=false
- Use service accounts: For workloads, not users
Authorization
# RBAC Example
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
etcd Security
- Enable encryption at rest
- Use TLS for all connections
- Restrict access to etcd nodes
- Regular backups
Kubelet Security
# kubelet configuration
authentication:
anonymous:
enabled: false
webhook:
enabled: true
cacheTTL: 2m0s
authorization:
mode: Webhook
Container Security
Container Images
Image Scanning
- Scan for vulnerabilities in CI/CD
- Block high/critical vulnerabilities
- Use minimal base images
- Regularly update dependencies
Best Practices
# Use specific versions
FROM node:20-alpine AS builder
# Run as non-root
USER 1001
# Use multi-stage builds
# No secrets in images
# Minimal packages installed
Runtime Security
Pod Security Standards
apiVersion: v1
kind: Pod
metadata:
name: secure-pod
spec:
securityContext:
runAsNonRoot: true
runAsUser: 10000
fsGroup: 20000
seccompProfile:
type: RuntimeDefault
containers:
- name: app
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
Resource Limits
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
Network Security
Network Policies
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: api-allow
namespace: production
spec:
podSelector:
matchLabels:
app: api
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- protocol: TCP
port: 8080
egress:
- to:
- podSelector:
matchLabels:
app: database
ports:
- protocol: TCP
port: 5432
Service Mesh
- Istio: Comprehensive security
- Linkerd: Simpler, lightweight
- Cilium: eBPF-based
Service mesh provides:
- mTLS between services
- Fine-grained traffic control
- Policy enforcement
Secrets Management
Built-in Secrets
apiVersion: v1
kind: Secret
metadata:
name: db-creds
type: Opaque
stringData:
username: admin
password: changeme
External Secrets
HashiCorp Vault
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: vault-secret
spec:
refreshInterval: 1h
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: db-password
data:
- secretKey: password
remoteRef:
key: database/prod
property: password
Best Practices
- Never commit secrets to Git
- Rotate secrets regularly
- Use encryption at rest
- Audit secret access
Workload Security
RBAC Configuration
# ClusterRole for namespace admin
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: namespace-admin
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
- nonResourceURLs: ["*"]
verbs: ["*"]
---
# RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: team-admin
namespace: team-a
subjects:
- kind: Group
name: team-a
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: namespace-admin
apiGroup: rbac.authorization.k8s.io
Pod Security
Security Context
securityContext:
runAsNonRoot: true
runAsUser: 10000
fsGroup: 20000
seccompProfile:
type: RuntimeDefault
Security Policies
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restricted
spec:
privileged: false
allowPrivilegeEscalation: false
requiredDropCapabilities:
- ALL
volumes:
- configMap
- emptyDir
- projected
- secret
- downwardAPI
- persistentVolumeClaim
hostNetwork: false
hostIPC: false
hostPID: false
runAsUser:
rule: MustRunAsNonRoot
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
fsGroup:
rule: RunAsAny
Monitoring and Auditing
Audit Logging
# Kubernetes audit policy
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
omitStages:
- RequestReceived
- level: RequestResponse
resources:
- group: ""
resources: ["pods"]
- level: None
users: ["system:kube-proxy"]
- level: None
verbs: ["watch"]
resources:
- group: ""
resources: ["endpoints"]
Monitoring
Key metrics to monitor:
- API request rate and errors
- Pod security violations
- Resource usage anomalies
- Authentication failures
- Network policy violations
Tools:
- Prometheus + Alertmanager
- Grafana for visualization
- Falco for runtime security
CI/CD Pipeline Security
Secure Pipeline
- Source: Sign commits, verify provenance
- Build: Scan images, no secrets in build
- Test: Security tests, dependency scanning
- Deploy: Immutable manifests, GitOps
Image Signing
# Sign with Cosign
cosign sign --key cosign.key image:tag
# Verify
cosign verify --key cosign.pub image:tag
Incident Response
Preparation
- Document runbooks
- Practice incident response
- Establish communication channels
- Set up forensics capability
Response Steps
- Identify and contain
- Investigate scope
- Eradicate threat
- Recover systems
- Post-mortem and improve
Conclusion
Kubernetes security requires attention at every layer. Implement defense in depth, follow least privilege, continuously monitor, and practice incident response. Security is an ongoing process, not a destination.
Comments