VPN Encryption Algorithms: AES, ChaCha20, and Key Exchange

Introduction

Every VPN protocol encrypts your traffic, but the specific algorithms used determine both security strength and performance. Choosing the wrong cipher suite can leave you vulnerable or cause unnecessary slowdowns. This guide explains the algorithms used in modern VPNs — what they do, how they compare, and which to choose.

Symmetric Encryption: Protecting the Data

Symmetric encryption uses the same key to encrypt and decrypt. It’s fast enough to encrypt all your traffic in real-time.

AES-256-GCM

AES (Advanced Encryption Standard) with 256-bit keys in GCM (Galois/Counter Mode) is the gold standard for VPN encryption.

Why AES-256-GCM:

• 256-bit key = 2²⁵⁶ possible keys — computationally infeasible to brute-force
• GCM mode provides authenticated encryption — detects tampering automatically
• Hardware acceleration via AES-NI instructions on modern CPUs (Intel, AMD, ARM)
• Used by: OpenVPN, IPsec/IKEv2, WireGuard (as an alternative)

# Test AES-NI support on your CPU
grep -m1 aes /proc/cpuinfo

# Benchmark AES-256-GCM
openssl speed -evp aes-256-gcm
# Typical result: ~3-5 GB/s on modern hardware with AES-NI

ChaCha20-Poly1305

ChaCha20-Poly1305 is WireGuard’s default cipher suite, designed by Daniel Bernstein.

Why ChaCha20-Poly1305:

• Performs well on devices without hardware AES acceleration (mobile, IoT, older CPUs)
• Resistant to timing attacks — constant-time implementation
• 256-bit security, same as AES-256
• Poly1305 provides authentication (like GCM)

# Benchmark ChaCha20-Poly1305
openssl speed -evp chacha20-poly1305
# Typically 2-4x faster than AES-256-GCM on devices without AES-NI
# Similar speed on devices with AES-NI

AES-128-GCM vs AES-256-GCM

AES-128-GCM: 128-bit key, ~10-15% faster, still extremely secure
AES-256-GCM: 256-bit key, slightly slower, extra margin against future attacks

Recommendation: Use AES-256-GCM unless you have severe performance constraints.
The performance difference is negligible on modern hardware.

Block Cipher Modes Comparison

Always use GCM. CBC without proper authentication is vulnerable to padding oracle attacks.

Key Exchange: Establishing the Shared Secret

Before encrypting data, both sides must agree on a shared key without transmitting it over the network. This is the key exchange problem.

Diffie-Hellman (DH) and ECDH

Diffie-Hellman allows two parties to derive a shared secret over an untrusted channel. An eavesdropper who sees all traffic cannot compute the shared secret.

Alice generates: private key a, public key A = g^a mod p
Bob generates:   private key b, public key B = g^b mod p

Alice sends A to Bob, Bob sends B to Alice.

Alice computes: B^a mod p = g^(ab) mod p
Bob computes:   A^b mod p = g^(ab) mod p

Both arrive at the same shared secret g^(ab) mod p.
An eavesdropper sees A and B but cannot compute g^(ab) without knowing a or b.

Curve25519 (ECDH)

Elliptic Curve Diffie-Hellman with Curve25519 is the modern standard:

• 128-bit security with only 32-byte keys (vs 2048-bit RSA for equivalent security)
• Designed to avoid implementation pitfalls — hard to implement incorrectly
• Used by: WireGuard (exclusively), OpenSSH, Signal, TLS 1.3
• ~10x faster than RSA-2048 key exchange

# Python example: Curve25519 key exchange
from cryptography.hazmat.primitives.asymmetric.x25519 import X25519PrivateKey

# Alice
alice_private = X25519PrivateKey.generate()
alice_public  = alice_private.public_key()

# Bob
bob_private = X25519PrivateKey.generate()
bob_public  = bob_private.public_key()

# Exchange public keys, compute shared secret
alice_shared = alice_private.exchange(bob_public)
bob_shared   = bob_private.exchange(alice_public)

assert alice_shared == bob_shared  # Same shared secret!

Key Exchange Comparison

Recommendation: Use Curve25519 or ECDH P-256. Avoid classical DH with parameters below 3072 bits.

Perfect Forward Secrecy (PFS)

PFS ensures that if your long-term private key is compromised, past sessions cannot be decrypted.

Without PFS:

Attacker records encrypted traffic today.
Later compromises your server's private key.
Decrypts all recorded traffic retroactively.

With PFS:

Each session uses ephemeral (temporary) keys.
Session keys are discarded after use.
Compromising the server key doesn't help decrypt past sessions.

How PFS Works in Practice

Session 1: Generate ephemeral key pair → derive session key → encrypt traffic → discard ephemeral key
Session 2: Generate NEW ephemeral key pair → derive NEW session key → encrypt traffic → discard
Session 3: Generate NEW ephemeral key pair → ...

WireGuard implements PFS automatically — it rotates session keys every 3 minutes and after 1GB of data.

# Verify PFS in OpenVPN config
grep -i "tls-cipher\|dh\|ecdh" /etc/openvpn/server.conf

# Good: uses ECDHE (ephemeral)
tls-cipher TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384

# Bad: uses static DH (no PFS)
tls-cipher TLS-RSA-WITH-AES-256-GCM-SHA384

Authentication: Verifying Identity

Encryption protects data in transit, but you also need to verify you’re talking to the right server.

Certificate-Based Authentication (PKI)

# Generate CA and server certificate (OpenVPN/IPsec)
# Using easy-rsa
./easyrsa init-pki
./easyrsa build-ca
./easyrsa gen-req server nopass
./easyrsa sign-req server server

# Verify certificate
openssl verify -CAfile ca.crt server.crt
openssl x509 -in server.crt -text -noout | grep -E "Subject:|Issuer:|Not After"

Pre-Shared Keys (WireGuard)

WireGuard uses Curve25519 public keys for authentication — no PKI needed:

# /etc/wireguard/wg0.conf
[Interface]
PrivateKey = <server-private-key>
Address = 10.0.0.1/24

[Peer]
PublicKey = <client-public-key>
AllowedIPs = 10.0.0.2/32
# Optional: additional pre-shared key for post-quantum resistance
PresharedKey = <optional-psk>

HMAC Authentication

Many VPN protocols use HMAC (Hash-based Message Authentication Code) to verify packet integrity:

HMAC-SHA256: 256-bit authentication tag, widely supported
HMAC-SHA512: 512-bit tag, stronger but slower
Poly1305:    Used with ChaCha20, very fast, 128-bit security

Cipher Suite Selection Guide

WireGuard (Fixed — No Choice Needed)

WireGuard uses a fixed cipher suite: ChaCha20-Poly1305 + Curve25519 + BLAKE2s. No configuration needed — it’s always secure.

OpenVPN

# /etc/openvpn/server.conf

# Modern cipher suite (OpenVPN 2.5+)
cipher AES-256-GCM
auth SHA256
tls-version-min 1.2
tls-cipher TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384:TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384

# Enable PFS
dh none  # use ECDH instead of static DH
ecdh-curve prime256v1

# Verify cipher in use
openvpn --show-tls

IPsec/IKEv2 (strongSwan)

# /etc/ipsec.conf
conn myvpn
    ike=aes256gcm16-sha256-ecp256!   # IKE phase 1
    esp=aes256gcm16-sha256!           # ESP phase 2 (data encryption)
    keyexchange=ikev2
    dpdaction=clear
    dpddelay=300s
    rekey=yes

Quick Reference: What to Use

Avoid: 3DES, RC4, DES, MD5, SHA1, DH < 2048-bit, RSA < 2048-bit.

Testing Your VPN’s Encryption

# Check what cipher OpenVPN negotiated
grep "Data Channel" /var/log/openvpn.log
# Should show: AES-256-GCM

# Test TLS cipher for OpenVPN management interface
openssl s_client -connect vpn.example.com:443 2>/dev/null | grep "Cipher is"

# WireGuard — always ChaCha20-Poly1305, verify with:
wg show wg0

# Check IPsec SAs (Security Associations)
ip xfrm state
# Look for: enc cbc(aes) or gcm(aes) in the output

Resources

• NIST Cryptographic Standards
• RFC 8439: ChaCha20-Poly1305
• Curve25519 by D.J. Bernstein
• OpenVPN Hardening Guide