Introduction
Kubernetes Operators are a powerful extension mechanism that allows you to encode operational knowledge into software. They automate complex application deployment, configuration, and management tasks that would otherwise require manual intervention.
Understanding Operators
Before diving into the diagram, it helps to understand the problem the operator pattern solves. The built-in Kubernetes resources—Pods, Services, Deployments, and ConfigMaps—are deliberately generic. They know how to run containers and route traffic, but they have no concept of what your application actually needs to stay healthy. A database, for example, needs scheduled backups, ordered failover, version-safe upgrades, and storage that survives pod restarts. Encoding that knowledge directly into YAML quickly becomes unmanageable, because static manifests cannot react to failures or changing conditions.
The traditional workflow—author manifests, run kubectl apply, then babysit the cluster—works
acceptably for stateless web services. For stateful workloads it breaks down. Someone has to
notice when a replica is unhealthy, drain traffic before a rolling upgrade, or rotate
credentials after a backup job fails. Each of those manual steps is a source of toil, human
error, and configuration drift between environments.
An operator attacks this problem by teaching Kubernetes to understand your application. It does
so through two complementary mechanisms. First, a Custom Resource Definition (CRD) introduces a
new API type—in this case a Database object—so application state becomes first-class Kubernetes
data that can be stored, versioned, and queried through the API server. Second, a controller
watches those custom resources and continuously drives the cluster toward the declared state.
The diagram below summarizes this relationship: where kubectl apply once fired YAML at the
cluster and hoped for the best, the operator instead watches a Custom Resource and reconciles
the cluster to match.
┌─────────────────────────────────────────────────────────────────┐
│ Kubernetes Operator Concept │
├─────────────────────────────────────────────────────────────────┤
│ │
│ Traditional Deployment: │
│ │
│ ┌──────────────┐ │
│ │ YAML │ ──▶ kubectl apply ──▶ K8s Cluster │
│ │ Manifests │ │
│ └──────────────┘ │
│ │
│ Problem: Complex apps need: │
│ - Automated backup and restore │
│ - Failover and recovery │
│ - Configuration updates │
│ - Custom scaling logic │
│ - Monitoring and alerts │
│ │
│ Solution: Operator │
│ │
│ ┌──────────────────┐ │
│ │ Custom Resource │ ──▶ Operator ──▶ K8s Resources │
│ │ (MyApp) │ (Controller) (Pods, Services) │
│ └──────────────────┘ │
│ │
│ The Operator understands: │
│ - How to install the application │
│ - How to upgrade it │
│ - How to handle failures │
│ - How to scale based on metrics │
│ │
└─────────────────────────────────────────────────────────────────┘
Operator Architecture
Notice that the diagram contains two cooperating pieces rather than one monolith. On the left,
the Kubernetes API server exposes the built-in resources alongside your Custom Resource
Definition; on the right sits the operator itself. The operator is not a magic add-on—it is an
ordinary program running as a Deployment that reads and writes through the same authenticated
API you use with kubectl. The connection between the two is a watch, which means the
controller is notified of changes to Database objects instead of polling for them.
The reconciliation loop in the middle of the diagram is the heart of every operator, and it is worth understanding precisely. Each iteration follows the same seven steps: watch the custom resources, detect a change, read the current cluster state, calculate the desired state, apply the difference, write status back to the API server, then repeat. Because the loop is declarative and event-driven, it naturally self-heals. If a Pod is deleted behind your back, the next reconcile detects the drift and recreates it without any human involvement.
This design carries real trade-offs. The loop is always running, so an operator consumes CPU and API-server traffic even when nothing is wrong; naive implementations can hammer the API server with unnecessary reads. In practice you mitigate this with caching informers, rate limiting, and requeue backoff. Note also that status is written back to the API server at the end of the loop—that closes the feedback circuit. If the operator crashes mid-loop, the status stays stale, which is precisely why the operator’s own health and readiness endpoints matter as much as the resources it manages.
┌─────────────────────────────────────────────────────────────────┐
│ Operator Architecture │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Kubernetes API Server │ │
│ │ ┌─────────────────┐ ┌─────────────────────────────┐ │ │
│ │ │ Built-in │ │ Custom Resource Definition │ │ │
│ │ │ Resources │ │ (CRD): MyApp │ │ │
│ │ │ (Pod, Service) │ └─────────────────────────────┘ │ │
│ │ └─────────────────┘ │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ▲ │
│ │ Watch │
│ │ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Operator / Controller │ │
│ │ ┌─────────────────────────────────────────────────────┐ │ │
│ │ │ Reconciliation Loop │ │ │
│ │ │ │ │ │
│ │ │ 1. Watch Custom Resources │ │ │
│ │ │ 2. Detect changes │ │ │
│ │ │ 3. Read current state │ │ │
│ │ │ 4. Calculate desired state │ │ │
│ │ │ 5. Make changes to cluster │ │ │
│ │ │ 6. Update status │ │ │
│ │ │ 7. Repeat │ │ │
│ │ └─────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Kubernetes Resources Created │ │
│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────────┐ │ │
│ │ │ Pods │ │Services │ │ ConfigMaps│ │ Secrets │ │ │
│ │ └─────────┘ └─────────┘ └─────────┘ └─────────────┘ │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
Python Operator (kopf)
This section shows the same operator implemented in Python using kopf, a lightweight framework that lowers the barrier to writing controllers. kopf works by letting you attach Python functions to lifecycle events on your custom resources using decorators. Behind the scenes it manages the watch, the reconciliation loop, and the API client plumbing, so the code you write reads like ordinary application logic rather than low-level Kubernetes machinery.
The first design decision is visible at the top of the file: the DatabaseSpec and
DatabaseStatus dataclasses. These provide a typed view of the fields defined in the CRD.
While kopf hands handlers a plain spec dictionary, the dataclasses document the contract—which
fields exist, their defaults, and what they mean—and give you a natural place to add validation
or helper methods. Notice that handler functions receive body, meta, spec, and status
as keyword arguments; these correspond to the live object, its metadata, and the last stored
status.
The main handler, database_create_or_update, is registered for both create and update events.
Treating the two as one code path is a deliberate choice: it makes reconciliation idempotent,
because the same logic runs whether the object just appeared or changed. The handler
orchestrates the cluster resources in order—StatefulSet first, then Service, then backups if
enabled—and returns a status dictionary that kopf persists to the object. Errors are caught and
converted into a Failed status with a readable message, so a broken deployment is immediately
visible through kubectl get database rather than through silent partial setup.
Two smaller handlers round out the controller. The delete handler removes the child resources
and any backup artifacts when the custom resource goes away. The field handler on spec.size
watches for scaling requests and adjusts replica counts accordingly; kopf lets you trigger
logic on individual field changes instead of the whole object, which keeps the code focused.
The helper functions at the bottom all follow the same read-then-create pattern: they query the
API for the existing resource and only call create when it is missing, avoiding the
ApiException you would otherwise get from blindly applying. This is the idempotency pattern
in action, and it is the single most important habit to carry into any operator you write.
import kopf
import kubernetes.client
from kubernetes.client import CoreV1Api
from typing import Dict, List
import yaml
# Configuration
kopf.configure(debug=True)
# Custom Resource Definition will be created automatically
# apiVersion: example.com/v1
# kind: Database
@dataclass
class DatabaseSpec:
"""Database specification."""
version: str
size: str # small, medium, large
backup_enabled: bool = False
backup_schedule: str = "0 2 * * *" # Daily at 2 AM
@dataclass
class DatabaseStatus:
"""Database status."""
ready: bool = False
endpoint: str = ""
version: str = ""
phase: str = "Creating"
@kopf.on.create('databases.example.com')
@kopf.on.update('databases.example.com')
def database_create_or_update(body, meta, spec, status, **kwargs):
"""Handle Database resource creation or update."""
name = meta.name
namespace = meta.namespace
# Extract spec
version = spec.get('version', '14')
size = spec.get('size', 'small')
backup_enabled = spec.get('backup_enabled', False)
# Create or update resources
try:
# Create StatefulSet
_create_statefulset(name, namespace, version, size)
# Create Service
_create_service(name, namespace)
# Handle backups if enabled
if backup_enabled:
_setup_backup(name, namespace, spec.get('backup_schedule'))
# Update status
return {
'status': {
'ready': True,
'endpoint': f'{name}.{namespace}.svc.cluster.local',
'version': version,
'phase': 'Running'
}
}
except Exception as e:
return {
'status': {
'ready': False,
'phase': 'Failed',
'error': str(e)
}
}
@kopf.on.delete('databases.example.com')
def database_delete(body, meta, **kwargs):
"""Handle Database resource deletion."""
name = meta.name
namespace = meta.namespace
# Cleanup resources
_delete_statefulset(name, namespace)
_delete_service(name, namespace)
_cleanup_backups(name, namespace)
return {}
@kopf.on.field('databases.example.com', field='spec.size')
def database_scale(spec, old, new, **kwargs):
"""Handle size changes (scaling)."""
# Scale up or down based on size change
_update_replicas(new)
def _create_statefulset(name: str, namespace: str, version: str, size: str):
"""Create StatefulSet for database."""
replicas = {'small': 1, 'medium': 2, 'large': 3}.get(size, 1)
# Resource limits based on size
resources = {
'small': {'cpu': '500m', 'memory': '512Mi'},
'medium': {'cpu': '1', 'memory': '1Gi'},
'large': {'cpu': '2', 'memory': '2Gi'},
}.get(size, {'cpu': '500m', 'memory': '512Mi'})
statefulset = {
'apiVersion': 'apps/v1',
'kind': 'StatefulSet',
'metadata': {
'name': name,
'namespace': namespace
},
'spec': {
'serviceName': name,
'replicas': replicas,
'selector': {
'matchLabels': {'app': name}
},
'template': {
'metadata': {
'labels': {'app': name}
},
'spec': {
'containers': [{
'name': 'postgres',
'image': f'postgres:{version}',
'ports': [{'containerPort': 5432}],
'env': [
{'name': 'POSTGRES_DB', 'value': name},
{'name': 'POSTGRES_USER', 'value': name},
],
'resources': {
'requests': resources,
'limits': resources
}
}]
}
}
}
}
api = kubernetes.client.AppsV1Api()
try:
api.read_namespaced_stateful_set(name, namespace)
except kubernetes.client.exceptions.ApiException:
api.create_namespaced_stateful_set(namespace, statefulset)
def _create_service(name: str, namespace: str):
"""Create Service for database."""
service = {
'apiVersion': 'v1',
'kind': 'Service',
'metadata': {'name': name, 'namespace': namespace},
'spec': {
'selector': {'app': name},
'ports': [
{'name': 'postgres', 'port': 5432, 'targetPort': 5432}
],
'clusterIP': 'None' # Headless service for StatefulSet
}
}
api = kubernetes.client.CoreV1Api()
try:
api.read_namespaced_service(name, namespace)
except kubernetes.client.exceptions.ApiException:
api.create_namespaced_service(namespace, service)
def _setup_backup(name: str, namespace: str, schedule: str):
"""Setup automated backups using CronJob."""
# Implementation for backup CronJob
pass
def _delete_statefulset(name: str, namespace: str):
"""Delete StatefulSet."""
api = kubernetes.client.AppsV1Api()
try:
api.delete_namespaced_stateful_set(name, namespace)
except:
pass
def _delete_service(name: str, namespace: str):
"""Delete Service."""
api = kubernetes.client.CoreV1Api()
try:
api.delete_namespaced_service(name, namespace)
except:
pass
def _cleanup_backups(name: str, namespace: str):
"""Cleanup backups."""
pass
Before moving on, it is worth weighing the Python approach against what follows. kopf is an excellent choice when you want the fastest possible path from idea to working operator, when the operator logic is mostly glue code, or when your team is more comfortable in Python than Go. The trade-offs are equally real: a Python operator ships as an interpreted application with a larger runtime image, consumes more memory per instance, and relies on dynamic dispatch that can mask bugs until runtime. For large clusters or high-throughput reconciliation, that overhead matters.
Go and controller-runtime, shown next, are the ecosystem default for a reason. The code is compiled, statically typed, and produces a single small binary that starts quickly and reconciles efficiently. That comes at the cost of more boilerplate: you define structs that mirror the CRD, and the framework relies on code generation driven by kubebuilder markers to wire everything together. The example below mirrors the Python version so you can compare the two styles directly.
Go Operator (Controller Runtime)
The Go operator is built on controller-runtime, the library that powers the Operator SDK and
Kubebuilder scaffolds. The DatabaseReconciler struct embeds client.Client directly, which
is idiomatic controller-runtime style: by embedding the client interface, every method on the
reconciler gains read and write access to the cluster without extra wiring. A Scheme is kept
alongside it so the controller can convert between Go types and the JSON the API server stores.
The Reconcile method is called for every event Kubernetes delivers for Database objects and
their owned children. The first step is always to fetch the current object by namespaced name.
The handling of IsNotFound is important: when a Database is deleted, a reconcile is still
fired, and returning a nil result quietly tells the framework there is nothing left to do
rather than logging an error. After fetching, the reconciler computes the desired replica count
from the size field and calls controllerutil.CreateOrUpdate for the StatefulSet and Service.
That helper is the Go equivalent of the read-then-create pattern from the Python version—it
either creates the resource or updates it to match the desired state, and it is what makes the
loop idempotent. Finally the status subresource is updated, and the function returns a
ctrl.Result signalling whether to requeue.
Three parts of the file are scaffolding you would normally get from kubebuilder rather than
write by hand. The kubebuilder RBAC markers at the top are parsed during code generation to
produce ClusterRole manifests scoped to exactly the resources the operator touches—note how the
markers separate permissions on databases from permissions on the child StatefulSets and
Services. SetupWithManager declares the ownership relationships: the controller watches
Database objects and their owned StatefulSets and Services, so changes to children also trigger
reconciliation. The mutate functions describe the desired state and, critically, call
SetControllerReference. This establishes an owner reference, which means when the Database
object is deleted, the garbage collector deletes its children automatically. The main
function assembles the pieces and enables leader election so that if you run multiple replicas
for availability, only one acts as leader at a time—preventing two operators from fighting over
the same resources.
package main
import (
"context"
"fmt"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/util/intstr"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
"sigs.k8s.io/controller-runtime/pkg/log"
examplecomv1 "example.com/api/v1"
)
// DatabaseReconciler reconciles a Database object
type DatabaseReconciler struct {
client.Client
Scheme *runtime.Scheme
}
//+kubebuilder:rbac:groups=example.com,resources=databases,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=example.com,resources=databases/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups="",resources=services,verbs=get;list;watch;create;update;patch;delete
func (r *DatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
logger := log.FromContext(ctx)
// Fetch the Database instance
db := &examplecomv1.Database{}
err := r.Get(ctx, req.NamespacedName, db)
if err != nil {
if errors.IsNotFound(err) {
return ctrl.Result{}, nil
}
return ctrl.Result{}, err
}
// Get desired size
size := getReplicas(db.Spec.Size)
// Create or update StatefulSet
sts := &appsv1.StatefulSet{}
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, sts, func() error {
return r.mutateStatefulSet(db, sts)
})
if err != nil {
logger.Error(err, "Failed to create/update StatefulSet")
return ctrl.Result{}, err
}
// Create or update Service
svc := &corev1.Service{}
_, err = controllerutil.CreateOrUpdate(ctx, r.Client, svc, func() error {
return r.mutateService(db, svc)
})
if err != nil {
logger.Error(err, "Failed to create/update Service")
return ctrl.Result{}, err
}
// Update status
db.Status.Ready = true
db.Status.Endpoint = fmt.Sprintf("%s.%s.svc.cluster.local", db.Name, db.Namespace)
db.Status.Version = db.Spec.Version
db.Status.Replicas = size
if err := r.Status().Update(ctx, db); err != nil {
logger.Error(err, "Failed to update Database status")
return ctrl.Result{}, err
}
return ctrl.Result{}, nil
}
func (r *DatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&examplecomv1.Database{}).
Owns(&appsv1.StatefulSet{}).
Owns(&corev1.Service{}).
Complete(r)
}
func (r *DatabaseReconciler) mutateStatefulSet(db *examplecomv1.Database, sts *appsv1.StatefulSet) error {
sts.ObjectMeta = metav1.ObjectMeta{
Name: db.Name,
Namespace: db.Namespace,
}
replicas := getReplicas(db.Spec.Size)
sts.Spec = appsv1.StatefulSetSpec{
ServiceName: db.Name,
Replicas: &replicas,
Selector: &metav1.LabelSelector{
MatchLabels: map[string]string{"app": db.Name},
},
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{"app": db.Name},
},
Spec: corev1.PodSpec{
Containers: []corev1.Container{{
Name: "postgres",
Image: fmt.Sprintf("postgres:%s", db.Spec.Version),
Ports: []corev1.ContainerPort{{
Name: "postgres",
ContainerPort: 5432,
}},
Env: []corev1.EnvVar{
{Name: "POSTGRES_DB", Value: db.Name},
{Name: "POSTGRES_USER", Value: db.Name},
},
}},
},
},
}
// Set owner reference
controllerutil.SetControllerReference(db, sts, r.Scheme)
return nil
}
func (r *DatabaseReconciler) mutateService(db *examplecomv1.Database, svc *corev1.Service) error {
svc.ObjectMeta = metav1.ObjectMeta{
Name: db.Name,
Namespace: db.Namespace,
}
svc.Spec = corev1.ServiceSpec{
Selector: map[string]string{"app": db.Name},
Ports: []corev1.ServicePort{{
Name: "postgres",
Port: 5432,
TargetPort: intstr.FromInt(5432),
}},
ClusterIP: corev1.ClusterIPNone,
}
controllerutil.SetControllerReference(db, svc, r.Scheme)
return nil
}
func getReplicas(size string) int32 {
switch size {
case "small":
return 1
case "medium":
return 2
case "large":
return 3
default:
return 1
}
}
func main() {
ctrl.SetLogger(zap.New())
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
Scheme: scheme,
LeaderElection: true,
LeaderElectionID: "example.com",
})
if err != nil {
os.Exit(1)
}
if err = (&DatabaseReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
}).SetupWithManager(mgr); err != nil {
os.Exit(1)
}
if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
os.Exit(1)
}
}
Custom Resource Definition
The Go operator compiles against a generated Go package (example.com/api/v1) that assumes the
Database type exists in the cluster. That type is defined, out of band, by the Custom Resource
Definition below. Think of the CRD as the contract between your manifests and your controller:
the controller reads and writes this shape through the API server, and the CRD schema is what
the API server validates every request against.
The definition registers a new API group example.com with a namespaced resource named
databases and the short name db, so kubectl get db works as shorthand. The versions
block is where most production complexity lives. A version that has served: true responds to
API requests, while storage: true marks the version used to persist objects to etcd; a
typical rollout serves v1 and v2 during a migration window while only one version writes to
storage. The schema beneath it is written in OpenAPI v3 and does double duty as validation and
documentation. The required list makes version and size mandatory, the enum on size
rejects unknown tiers at admission time, and the default values fill in backup_enabled and
backup_schedule when a user omits them. Notice that spec and status are defined as
separate properties; the status block is what the operator writes back, and keeping it distinct
from spec prevents status updates from triggering spurious spec-based reconciles.
Because the schema is enforced by the API server rather than the operator, mistakes are caught at admission time with a clear message, before any controller code runs. This separation—schema on the API side, behavior in the controller—is what makes operators safe to hand to users who are not Kubernetes experts.
# crd.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: databases.example.com
spec:
group: example.com
names:
kind: Database
plural: databases
shortNames:
- db
scope: Namespaced
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
required:
- version
- size
properties:
version:
type: string
description: "PostgreSQL version"
example: "14"
size:
type: string
enum: [small, medium, large]
description: "Database size/tier"
backup_enabled:
type: boolean
default: false
backup_schedule:
type: string
default: "0 2 * * *"
status:
type: object
properties:
ready:
type: boolean
endpoint:
type: string
version:
type: string
replicas:
type: integer
Using the Operator
With the operator deployed and the CRD installed, using it feels identical to using any other
Kubernetes resource: you write a manifest, apply it, and let the controller do the rest. The
database.yaml below requests a production-grade database—PostgreSQL 14 at the large tier with
nightly backups enabled. The operator reads that spec, creates the StatefulSet with the replica
count and resource limits that map to the large size, creates the headless Service, and
installs the backup CronJob scheduled for 2 AM.
The lifecycle is fully declarative, which is the point. To scale, you never touch the StatefulSet directly; you patch the custom resource and let the operator translate the new size into the correct replica count. When the delete runs, the operator’s cleanup handlers and finalizer logic ensure the database and its backups are removed cleanly rather than leaving orphaned resources behind.
# database.yaml
apiVersion: example.com/v1
kind: Database
metadata:
name: my-production-db
namespace: production
spec:
version: "14"
size: large
backup_enabled: true
backup_schedule: "0 2 * * *"
Apply these manifests and observe how the controller reacts at each step:
# Deploy operator
kubectl apply -f operator.yaml
# Deploy database
kubectl apply -f database.yaml
# Check status
kubectl get database my-production-db
# Scale database
kubectl patch database my-production-db -p '{"spec":{"size":"medium"}}' --type=merge
# Delete database
kubectl delete database my-production-db
Common Operator Patterns
Production operators are rarely as simple as create-and-forget. Two patterns in particular separate a toy operator from one you can run in production, and both are shown in this code sample. The first is finalizers. When you delete a Kubernetes object, deletion does not happen immediately—the API server marks it for deletion and then waits until all finalizers listed on the object are removed. By adding a finalizer when the resource is created and removing it only after your cleanup work finishes, you guarantee that external resources (volumes, DNS entries, cloud databases) are torn down even if the operator crashes mid-cleanup. Without a finalizer, the custom resource can vanish while its backing infrastructure lives on, silently leaking cloud spend.
The second pattern is structured status conditions. Rather than a single free-form status
string, Kubernetes conventions favor a list of condition objects, each with a type, a boolean
status, a reason, and a lastTransitionTime. This machine-readable format is what tools,
dashboards, and other controllers expect, and the transition time lets operators reason about
how long something has been in a given state. The update_status helper implements this idiom
faithfully: it finds an existing condition of the requested type and updates it in place, or
appends a new one, stamping each entry with the current timestamp. Using conditions for every
meaningful state (Reconciling, Ready, Failed) makes the health of your custom resource as
easy to inspect as that of a built-in Deployment.
# Finalizers - Ensure cleanup
@kopf.on.create('myresources.example.com')
def add_finalizer(body, **kwargs):
if 'finalizers' not in body.metadata:
body.metadata['finalizers'] = []
if 'my-operator/finalizer' not in body.metadata['finalizers']:
body.metadata['finalizers'].append('my-operator/finalizer')
return {'metadata': {'finalizers': body.metadata['finalizers']}}
@kopf.on.delete('myresources.example.com')
def cleanup_on_delete(spec, **kwargs):
# Perform cleanup before deletion
_cleanup_resources(spec)
# Return finalizer to remove it
return {}
# Status Conditions
def update_status(conditions: list, condition_type: str, status: bool, reason: str):
"""Update status conditions."""
now = datetime.utcnow()
# Find existing condition
existing = next((c for c in conditions if c.get('type') == condition_type), None)
if existing:
existing['status'] = status
existing['reason'] = reason
existing['lastTransitionTime'] = now.isoformat()
else:
conditions.append({
'type': condition_type,
'status': status,
'reason': reason,
'lastTransitionTime': now.isoformat()
})
return conditions
Best Practices
Taken together, the patterns in this article boil down to the checklist below. Each item addresses a failure mode that has taken down real operators in production. Finalizers, as discussed, prevent orphaned external resources. Status conditions give you and your monitoring the visibility to know what the operator is doing. Graceful error handling matters because a reconcile that panics can wedge the whole watch pipeline; always return an error to the framework so it can retry with backoff rather than crash. Logging is your primary debugging tool inside a control loop that may run thousands of times a day, so log the object being reconciled, not just a message.
Idempotency is non-negotiable—your reconcile may run multiple times for the same state, and every operation must be safe to repeat. Admission webhooks validate user input where the CRD schema’s constraints are not the only defense, and they are the right place for cross-field rules that OpenAPI cannot express. Leader election prevents two operator replicas from reconciling the same objects simultaneously, which would otherwise corrupt state. Finally, remember that status updates and child-object writes are ordinary API calls; design with rate limits and watch volume in mind so your operator scales with your cluster rather than against it.
# Operator best practices
# 1. Use finalizers for cleanup
# 2. Implement status conditions
# 3. Handle reconcile errors gracefully
# 4. Add proper logging
# 5. Implement idempotency
# 6. Add webhooks for validation
# 7. Use leader election for HA
# 8. Implement proper error handling
Conclusion
Kubernetes Operators extend Kubernetes to manage complex applications automatically. By encoding operational knowledge into controllers, operators can handle deployment, scaling, backups, and recovery without manual intervention.
As you adopt operators, start with a framework that matches your team’s language, keep the reconcile loop idempotent from day one, and invest in finalizers and status conditions before you add more features—the operational habits you establish early are what keep a growing fleet of operators maintainable. The operator pattern is a long-term investment in your infrastructure: the initial authoring effort is repaid every time a database restores itself, a replica scales back after a traffic spike, or a failed rollout rolls back without a human being paged.
Key takeaways:
- Operators use Custom Resource Definitions to extend Kubernetes
- The reconciliation loop ensures desired state is maintained
- Use operator frameworks like Operator SDK or kopf for development
- Implement finalizers for proper cleanup
- Add status conditions for better observability
Comments