Background
In TCP/IP, when a TCP connection is established, both client and server can send a FIN request to end the connection. Client wants to keep the ESTABLISHED state the long the better, for there are so many files are needed to get, such as HTML/JS/CSS/Image, etc.
However, In the interest of the server, the ESTABLISHED state the short the better, because the there are many other clients are waiting for service and the resources of a server is limited.
So, generally, after all the files have been downloaded, the browser will not close the connection actively, and the server often close the connection actively.
How to close a connection after a certain amount of time in Nginx ? the keepalive_timeout directive kicks in.
Syntax: keepalive_timeout timeout [header_timeout];
Default:
keepalive_timeout 75s;
Context: http, server, location
The first parameter sets a timeout during which a keep-alive client connection will stay open on the server side. The zero value disables keep-alive client connections. The optional second parameter sets a value in the “Keep-Alive: timeout=time” response header field. Two parameters may differ.
The “Keep-Alive: timeout=time” header field is recognized by Mozilla and Konqueror. MSIE closes keep-alive connections by itself in about 60 seconds.
By default, when a browser initiates a request to nginx, the HTTP request header sets connection: keep-alive. This connection can be used to transfer multiple files, such as JS, HTML, CSS, JPG, etc., avoiding the overhead of repeatedly establishing new connections. Therefore, the browser will not actively close the connection after communication is complete. Even if the client sets keep-alive, the server can also actively close the connection (not discussed here for now).
This creates a problem: in high-concurrency scenarios, a large number of such connections are generated in a short period, each occupying a certain amount of server resources (mainly memory and open file counts; if proxying, consider connections nginx makes to backend services like PHP — these should not be too numerous). This is not ideal — the server hopes that once a client finishes its visit, it will release the connection as soon as possible to avoid prolonged resource consumption.
So finding an appropriate value is important for service efficiency — on one hand, avoiding the overhead of repeatedly establishing connections; on the other hand, preventing clients from occupying connection resources indefinitely.
About the usage of keepalive_timeout in nginx:
Context: http, server, location
Usage format: keepalive_timeout arg1 [arg2]
keepalive_timeout accepts 2 parameters:
arg1 (required): timeout for nginx to initiate connection close to the client
arg2 (optional): nginx sets Keep-Alive:timeout=arg2 in the response header, asking the client to close the connection actively.
In some cases, the browser may ignore this setting. The following experiment shows that setting arg2 had no effect.
This directive defines the duration of keep-alive, which keeps the client-to-server connection valid for a certain period. Within this time, the client does not need to establish or re-establish a connection to access the server.
Experiment Environment
Client IP：192.168.0.16
Server IP：192.168.0.21
Setup a Nginx server, which serves a html page, we are going to focus on TCP protocol.
# file: nginx.conf
http {
keepalive_timeout 15;
# the unit is seconds
}
Using tcpdump on Nginx Server
nginx start at port 80，using tcpdump
$ tcpdump tcp port 80
Experiment 1：keepalive_timout: 15s
The following package shows that client sents ESTABLISH request to the server at 11:53:12, after 15s, the server at 11:53:27, send FIN to clinet.
The client initiates the request at 12s, and at 27s nginx sends FIN to the client, requesting to close the connection.
11:53:12.013881 IP 192.168.0.16.52119 > 192.168.0.21.http: Flags [S], seq 1647038819, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
11:53:12.013953 IP 192.168.0.21.http > 192.168.0.16.52119: Flags [S.], seq 2440006518, ack 1647038820, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
11:53:12.014227 IP 192.168.0.16.52119 > 192.168.0.21.http: Flags [.], ack 1, win 16425, length 0
11:53:12.015674 IP 192.168.0.16.52119 > 192.168.0.21.http: Flags [P.], seq 1:534, ack 1, win 16425, length 533
11:53:12.015720 IP 192.168.0.21.http > 192.168.0.16.52119: Flags [.], ack 534, win 123, length 0
11:53:12.015908 IP 192.168.0.21.http > 192.168.0.16.52119: Flags [P.], seq 1:173, ack 534, win 123, length 172
11:53:12.213230 IP 192.168.0.16.52119 > 192.168.0.21.http: Flags [.], ack 173, win 16382, length 0
11:53:27.031171 IP 192.168.0.21.http > 192.168.0.16.52119: Flags [F.], seq 173, ack 534, win 123, length 0
11:53:27.031388 IP 192.168.0.16.52119 > 192.168.0.21.http: Flags [.], ack 174, win 16382, length 0
11:54:12.036632 IP 192.168.0.16.52119 > 192.168.0.21.http: Flags [.], seq 533:534, ack 174, win 16382, length 1
11:54:12.036678 IP 192.168.0.21.http > 192.168.0.16.52119: Flags [.], ack 534, win 123, length 0
11:54:57.028996 IP 192.168.0.16.52119 > 192.168.0.21.http: Flags [.], seq 533:534, ack 174, win 16382, length 1
11:54:57.029037 IP 192.168.0.21.http > 192.168.0.16.52119: Flags [R], seq 2440006692, win 0, length 0
Exp 2：keepalive_timout:6s
The following package shows that client sents ESTABLISH request to the server at 16:13:07, after 6s, the server at 16:13:13, send FIN to clinet.
The client initiates the request at 07s, and at 13s nginx sends FIN to the client, requesting to close the connection.
16:13:07.986580 IP 192.168.0.16.53518 > 192.168.0.21.http: Flags [S], seq 141352550, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:13:07.986653 IP 192.168.0.21.http > 192.168.0.16.53518: Flags [S.], seq 3684334522, ack 141352551, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:13:07.986853 IP 192.168.0.16.53518 > 192.168.0.21.http: Flags [.], ack 1, win 16425, length 0
16:13:07.988210 IP 192.168.0.16.53518 > 192.168.0.21.http: Flags [P.], seq 1:534, ack 1, win 16425, length 533
16:13:07.988239 IP 192.168.0.21.http > 192.168.0.16.53518: Flags [.], ack 534, win 123, length 0
16:13:07.988911 IP 192.168.0.21.http > 192.168.0.16.53518: Flags [P.], seq 1:173, ack 534, win 123, length 172
16:13:08.186352 IP 192.168.0.16.53518 > 192.168.0.21.http: Flags [.], ack 173, win 16382, length 0
16:13:13.995178 IP 192.168.0.21.http > 192.168.0.16.53518: Flags [F.], seq 173, ack 534, win 123, length 0
16:13:13.995424 IP 192.168.0.16.53518 > 192.168.0.21.http: Flags [.], ack 174, win 16382, length 0
16:13:55.937133 IP 192.168.0.16.53518 > 192.168.0.21.http: Flags [F.], seq 534, ack 174, win 16382, length 0
16:13:55.937168 IP 192.168.0.21.http > 192.168.0.16.53518: Flags [.], ack 535, win 123, length 0
Exp 3：keepalive_timout: 0s
The connection will be closed by the server as soon as the data transfer was completed.
When set to 0, the long connection is closed immediately after data transfer completes, returning the header Connection: close.
16:36:33.526022 IP 192.168.0.16.53988 > 192.168.0.21.http: Flags [S], seq 453414335, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:36:33.526097 IP 192.168.0.21.http > 192.168.0.16.53988: Flags [S.], seq 1445206998, ack 453414336, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:36:33.526391 IP 192.168.0.16.53988 > 192.168.0.21.http: Flags [.], ack 1, win 16425, length 0
16:36:33.528002 IP 192.168.0.16.53988 > 192.168.0.21.http: Flags [P.], seq 1:534, ack 1, win 16425, length 533
16:36:33.528032 IP 192.168.0.21.http > 192.168.0.16.53988: Flags [.], ack 534, win 123, length 0
16:36:33.528402 IP 192.168.0.21.http > 192.168.0.16.53988: Flags [P.], seq 1:168, ack 534, win 123, length 167
16:36:33.528497 IP 192.168.0.21.http > 192.168.0.16.53988: Flags [F.], seq 168, ack 534, win 123, length 0
16:36:33.528748 IP 192.168.0.16.53988 > 192.168.0.21.http: Flags [.], ack 169, win 16383, length 0
16:36:33.529182 IP 192.168.0.16.53988 > 192.168.0.21.http: Flags [F.], seq 534, ack 169, win 16383, length 0
16:36:33.529200 IP 192.168.0.21.http > 192.168.0.16.53988: Flags [.], ack 535, win 123, length 0
Exp 4：keepalive_timeout 30s, Keep-Alive:timeout=5;
keepalive_timeout 30 5;
16:42:32.941249 IP 192.168.0.16.54128 > 192.168.0.21.http: Flags [S], seq 1359017244, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
16:42:32.941316 IP 192.168.0.21.http > 192.168.0.16.54128: Flags [S.], seq 2951328040, ack 1359017245, win 14600, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
16:42:32.941524 IP 192.168.0.16.54128 > 192.168.0.21.http: Flags [.], ack 1, win 16425, length 0
16:42:32.947169 IP 192.168.0.16.54128 > 192.168.0.21.http: Flags [P.], seq 1:534, ack 1, win 16425, length 533
16:42:32.947218 IP 192.168.0.21.http > 192.168.0.16.54128: Flags [.], ack 534, win 123, length 0
16:42:32.947747 IP 192.168.0.21.http > 192.168.0.16.54128: Flags [P.], seq 1:196, ack 534, win 123, length 195
16:42:33.148722 IP 192.168.0.21.http > 192.168.0.16.54128: Flags [P.], seq 1:196, ack 534, win 123, length 195
16:42:33.148886 IP 192.168.0.16.54128 > 192.168.0.21.http: Flags [.], ack 196, win 16376, options [nop,nop,sack 1 {1:196}], length 0
16:43:02.979017 IP 192.168.0.21.http > 192.168.0.16.54128: Flags [F.], seq 196, ack 534, win 123, length 0
16:43:02.979265 IP 192.168.0.16.54128 > 192.168.0.21.http: Flags [.], ack 197, win 16376, length 0
16:43:21.010604 IP 192.168.0.16.54128 > 192.168.0.21.http: Flags [F.], seq 534, ack 197, win 16376, length 0
16:43:21.010644 IP 192.168.0.21.http > 192.168.0.16.54128: Flags [.], ack 535, win 123, length 0
Here the second parameter did not cause the client to initiate the close. In other words, it had no effect — needs further investigation.
Packets captured simultaneously on the client using Wireshark on Windows
Note: The Wireshark capture here is not as detailed as tcpdump and didn’t show the full picture. The filter parameter used was: ip.addr == 192.168.0.21 && tcp.port == 80
keepalive_timout set to 15s
keepalive_timout set to 6s
Summary of the C/S Communication Process
- Clinet send syn to server
- Data transfer
- keepalive_time time is up，nginx send fin to clinet，the connection closed
After the connection is closed, the ESTABLISHED status will disappear，using netstat -antp | grep nginx
tcp 0 0 192.168.0.21:80 192.168.0.16:53424 ESTABLISHED 23034/nginx
When does keepalive_timeout trigger execution?
keepalive_timeout value = the time of `FIN` sent by server - the time of `SYN` start from client
If the connection is closed, the client will re-request a new connection.
Several Questions
- Wireshark did not show the bidirectional communication process; unsure how to set the parameters.
- Wireshark eventually showed a RST packet; the reason is unknown, but tcpdump did not have this issue. It’s recommended to rely on the tcpdump command.
- If keepalive_timout is set to 0, the response header returns
Connection: closeand the connection closes immediately. - The second parameter did not take effect.
Comments