3 nghttpx - HTTP/2 proxy - HOW-TO
4 ===============================
6 :doc:`nghttpx.1` is a proxy translating protocols between HTTP/2 and
7 other protocols (e.g., HTTP/1). It operates in several modes and each
8 mode may require additional programs to work with. This article
9 describes each operation mode and explains the intended use-cases. It
10 also covers some useful options later.
15 If nghttpx is invoked without :option:`--http2-proxy`, it operates in
16 default mode. In this mode, it works as reverse proxy (gateway) for
17 HTTP/3, HTTP/2 and HTTP/1 clients to backend servers. This is also
18 known as "HTTP/2 router".
20 By default, frontend connection is encrypted using SSL/TLS. So
21 server's private key and certificate must be supplied to the command
22 line (or through configuration file). In this case, the frontend
23 protocol selection will be done via ALPN or NPN.
25 To turn off encryption on frontend connection, use ``no-tls`` keyword
26 in :option:`--frontend` option. HTTP/2 and HTTP/1 are available on
27 the frontend, and an HTTP/1 connection can be upgraded to HTTP/2 using
28 HTTP Upgrade. Starting HTTP/2 connection by sending HTTP/2 connection
29 preface is also supported.
31 In order to receive HTTP/3 traffic, use ``quic`` parameter in
32 :option:`--frontend` option (.e.g, ``--frontend='*,443;quic'``)
34 nghttpx can listen on multiple frontend addresses. This is achieved
35 by using multiple :option:`--frontend` options. For each frontend
36 address, TLS can be enabled or disabled.
38 By default, backend connections are not encrypted. To enable TLS
39 encryption on backend connections, use ``tls`` keyword in
40 :option:`--backend` option. Using patterns and ``proto`` keyword in
41 :option:`--backend` option, backend application protocol can be
42 specified per host/request path pattern. It means that you can use
43 both HTTP/2 and HTTP/1 in backend connections at the same time. Note
44 that default backend protocol is HTTP/1.1. To use HTTP/2 in backend,
45 you have to specify ``h2`` in ``proto`` keyword in :option:`--backend`
48 The backend is supposed to be a Web server. For example, to make
49 nghttpx listen to encrypted HTTP/2 requests at port 8443, and a
50 backend Web server is configured to listen to HTTP requests at port
51 8080 on the same host, run nghttpx command-line like this:
55 $ nghttpx -f0.0.0.0,8443 -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
57 Then an HTTP/2 enabled client can access the nghttpx server using HTTP/2. For
58 example, you can send a GET request using nghttp:
62 $ nghttp -nv https://localhost:8443/
67 If nghttpx is invoked with :option:`--http2-proxy` (or its shorthand
68 :option:`-s`) option, it operates in HTTP/2 proxy mode. The supported
69 protocols in frontend and backend connections are the same as in `default
70 mode`_. The difference is that this mode acts like a forward proxy and
71 assumes the backend is an HTTP proxy server (e.g., Squid, Apache Traffic
72 Server). HTTP/1 requests must include an absolute URI in request line.
74 By default, the frontend connection is encrypted. So this mode is
75 also called secure proxy.
77 To turn off encryption on the frontend connection, use ``no-tls`` keyword
78 in :option:`--frontend` option.
80 The backend must be an HTTP proxy server. nghttpx supports multiple
81 backend server addresses. It translates incoming requests to HTTP
82 request to backend server. The backend server performs real proxy
83 work for each request, for example, dispatching requests to the origin
84 server and caching contents.
86 The backend connection is not encrypted by default. To enable
87 encryption, use ``tls`` keyword in :option:`--backend` option. The
88 default backend protocol is HTTP/1.1. To use HTTP/2 in backend
89 connection, use :option:`--backend` option, and specify ``h2`` in
90 ``proto`` keyword explicitly.
92 For example, to make nghttpx listen to encrypted HTTP/2 requests at
93 port 8443, and a backend HTTP proxy server is configured to listen to
94 HTTP/1 requests at port 8080 on the same host, run nghttpx command-line
99 $ nghttpx -s -f'*,8443' -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
101 At the time of this writing, Firefox 41 and Chromium v46 can use
102 nghttpx as HTTP/2 proxy.
104 To make Firefox or Chromium use nghttpx as HTTP/2 proxy, user has to
105 create proxy.pac script file like this:
107 .. code-block:: javascript
109 function FindProxyForURL(url, host) {
110 return "HTTPS SERVERADDR:PORT";
113 ``SERVERADDR`` and ``PORT`` is the hostname/address and port of the
114 machine nghttpx is running. Please note that both Firefox and
115 Chromium require valid certificate for secure proxy.
117 For Firefox, open Preference window and select Advanced then click
118 Network tab. Clicking Connection Settings button will show the
119 dialog. Select "Automatic proxy configuration URL" and enter the path
120 to proxy.pac file, something like this:
124 file:///path/to/proxy.pac
126 For Chromium, use following command-line:
130 $ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
132 As HTTP/1 proxy server, Squid may work as out-of-box. Traffic server
133 requires to be configured as forward proxy. Here is the minimum
134 configuration items to edit:
138 CONFIG proxy.config.reverse_proxy.enabled INT 0
139 CONFIG proxy.config.url_remap.remap_required INT 0
141 Consult Traffic server `documentation
142 <http://trafficserver.readthedocs.org/en/latest/admin-guide/configuration/transparent-forward-proxying.en.html>`_
143 to know how to configure traffic server as forward proxy and its
144 security implications.
149 ALPN support requires OpenSSL >= 1.0.2.
151 Disable frontend SSL/TLS
152 ------------------------
154 The frontend connections are encrypted with SSL/TLS by default. To
155 turn off SSL/TLS, use ``no-tls`` keyword in :option:`--frontend`
156 option. If this option is used, the private key and certificate are
157 not required to run nghttpx.
159 Enable backend SSL/TLS
160 ----------------------
162 The backend connections are not encrypted by default. To enable
163 SSL/TLS encryption, use ``tls`` keyword in :option:`--backend` option.
165 Enable SSL/TLS on memcached connection
166 --------------------------------------
168 By default, memcached connection is not encrypted. To enable
169 encryption, use ``tls`` keyword in
170 :option:`--tls-ticket-key-memcached` for TLS ticket key, and
171 :option:`--tls-session-cache-memcached` for TLS session cache.
173 Specifying additional server certificates
174 -----------------------------------------
176 nghttpx accepts additional server private key and certificate pairs
177 using :option:`--subcert` option. It can be used multiple times.
179 Specifying additional CA certificate
180 ------------------------------------
182 By default, nghttpx tries to read CA certificate from system. But
183 depending on the system you use, this may fail or is not supported.
184 To specify CA certificate manually, use :option:`--cacert` option.
185 The specified file must be PEM format and can contain multiple
188 By default, nghttpx validates server's certificate. If you want to
189 turn off this validation, knowing this is really insecure and what you
190 are doing, you can use :option:`--insecure` option to disable
191 certificate validation.
193 Read/write rate limit
194 ---------------------
196 nghttpx supports transfer rate limiting on frontend connections. You
197 can do rate limit per frontend connection for reading and writing
200 To perform rate limit for reading, use :option:`--read-rate` and
201 :option:`--read-burst` options. For writing, use
202 :option:`--write-rate` and :option:`--write-burst`.
204 Please note that rate limit is performed on top of TCP and nothing to
205 do with HTTP/2 flow control.
207 Rewriting location header field
208 -------------------------------
210 nghttpx automatically rewrites location response header field if the
211 following all conditions satisfy:
213 * In the default mode (:option:`--http2-proxy` is not used)
214 * :option:`--no-location-rewrite` is not used
215 * URI in location header field is an absolute URI
216 * URI in location header field includes non empty host component.
217 * host (without port) in URI in location header field must match the
218 host appearing in ``:authority`` or ``host`` header field.
220 When rewrite happens, URI scheme is replaced with the ones used in
221 frontend, and authority is replaced with which appears in
222 ``:authority``, or ``host`` request header field. ``:authority``
223 header field has precedence over ``host``.
228 nghttpx supports hot swapping using signals. The hot swapping in
229 nghttpx is multi step process. First send USR2 signal to nghttpx
230 process. It will do fork and execute new executable, using same
231 command-line arguments and environment variables.
233 As of nghttpx version 1.20.0, that is all you have to do. The new
234 main process sends QUIT signal to the original process, when it is
235 ready to serve requests, to shut it down gracefully.
237 For earlier versions of nghttpx, you have to do one more thing. At
238 this point, both current and new processes can accept requests. To
239 gracefully shutdown current process, send QUIT signal to current
240 nghttpx process. When all existing frontend connections are done, the
241 current process will exit. At this point, only new nghttpx process
242 exists and serves incoming requests.
244 If you want to just reload configuration file without executing new
245 binary, send SIGHUP to nghttpx main process.
250 When rotating log files, it is desirable to re-open log files after
251 log rotation daemon renamed existing log files. To tell nghttpx to
252 re-open log files, send USR1 signal to nghttpx process. It will
253 re-open files specified by :option:`--accesslog-file` and
254 :option:`--errorlog-file` options.
256 Multiple frontend addresses
257 ---------------------------
259 nghttpx can listen on multiple frontend addresses. To specify them,
260 just use :option:`--frontend` (or its shorthand :option:`-f`) option
261 repeatedly. TLS can be enabled or disabled per frontend address
262 basis. For example, to listen on port 443 with TLS enabled, and on
271 Multiple backend addresses
272 --------------------------
274 nghttpx supports multiple backend addresses. To specify them, just
275 use :option:`--backend` (or its shorthand :option:`-b`) option
276 repeatedly. For example, to use ``192.168.0.10:8080`` and
277 ``192.168.0.11:8080``, use command-line like this:
278 ``-b192.168.0.10,8080 -b192.168.0.11,8080``. In configuration file,
283 backend=192.168.0.10,8080
284 backend=192.168.0.11,8008
286 nghttpx can route request to different backend according to request
287 host and path. For example, to route request destined to host
288 ``doc.example.com`` to backend server ``docserv:3000``, you can write
293 backend=docserv,3000;doc.example.com/
295 When you write this option in command-line, you should enclose
296 argument with single or double quotes, since the character ``;`` has a
297 special meaning in shell.
299 To route, request to request path ``/foo`` to backend server
300 ``[::1]:8080``, you can write like so:
304 backend=::1,8080;/foo
306 If the last character of path pattern is ``/``, all request paths
307 which start with that pattern match:
311 backend=::1,8080;/bar/
313 The request path ``/bar/buzz`` matches the ``/bar/``.
315 You can use ``*`` at the end of the path pattern to make it wildcard
316 pattern. ``*`` must match at least one character:
320 backend=::1,8080;/sample*
322 The request path ``/sample1/foo`` matches the ``/sample*`` pattern.
324 Of course, you can specify both host and request path at the same
329 backend=192.168.0.10,8080;example.com/foo
331 We can use ``*`` in the left most position of host to achieve wildcard
332 suffix match. If ``*`` is the left most character, then the remaining
333 string should match the request host suffix. ``*`` must match at
334 least one character. For example, ``*.example.com`` matches
335 ``www.example.com`` and ``dev.example.com``, and does not match
336 ``example.com`` and ``nghttp2.org``. The exact match (without ``*``)
337 always takes precedence over wildcard match.
339 One important thing you have to remember is that we have to specify
340 default routing pattern for so called "catch all" pattern. To write
341 "catch all" pattern, just specify backend server address, without
344 Usually, host is the value of ``Host`` header field. In HTTP/2, the
345 value of ``:authority`` pseudo header field is used.
347 When you write multiple backend addresses sharing the same routing
348 pattern, they are used as load balancing. For example, to use 2
349 servers ``serv1:3000`` and ``serv2:3000`` for request host
350 ``example.com`` and path ``/myservice``, you can write like so:
354 backend=serv1,3000;example.com/myservice
355 backend=serv2,3000;example.com/myservice
357 You can also specify backend application protocol in
358 :option:`--backend` option using ``proto`` keyword after pattern.
359 Utilizing this allows ngttpx to route certain request to HTTP/2, other
360 requests to HTTP/1. For example, to route requests to ``/ws/`` in
361 backend HTTP/1.1 connection, and use backend HTTP/2 for other
366 backend=serv1,3000;/;proto=h2
367 backend=serv1,3000;/ws/;proto=http/1.1
369 The default backend protocol is HTTP/1.1.
371 TLS can be enabled per pattern basis:
375 backend=serv1,8443;/;proto=h2;tls
376 backend=serv2,8080;/ws/;proto=http/1.1
378 In the above case, connection to serv1 will be encrypted by TLS. On
379 the other hand, connection to serv2 will not be encrypted by TLS.
381 Dynamic hostname lookup
382 -----------------------
384 By default, nghttpx performs backend hostname lookup at start up, or
385 configuration reload, and keeps using them in its entire session. To
386 make nghttpx perform hostname lookup dynamically, use ``dns``
387 parameter in :option:`--backend` option, like so:
391 backend=foo.example.com,80;;dns
393 nghttpx will cache resolved addresses for certain period of time. To
394 change this cache period, use :option:`--dns-cache-timeout`.
396 Enable PROXY protocol
397 ---------------------
399 PROXY protocol can be enabled per frontend. In order to enable PROXY
400 protocol, use ``proxyproto`` parameter in :option:`--frontend` option,
405 frontend=*,443;proxyproto
407 nghttpx supports both PROXY protocol v1 and v2. AF_UNIX in PROXY
408 protocol version 2 is ignored.
413 Two kinds of session affinity are available: client IP, and HTTP
416 To enable client IP based affinity, specify ``affinity=ip`` parameter
417 in :option:`--backend` option. If PROXY protocol is enabled, then an
418 address obtained from PROXY protocol is taken into consideration.
420 To enable HTTP Cookie based affinity, specify ``affinity=cookie``
421 parameter, and specify a name of cookie in ``affinity-cookie-name``
422 parameter. Optionally, a Path attribute can be specified in
423 ``affinity-cookie-path`` parameter:
427 backend=127.0.0.1,3000;;affinity=cookie;affinity-cookie-name=nghttpxlb;affinity-cookie-path=/
429 Secure attribute of cookie is set if client connection is protected by
435 nghttpx supports pre-shared key (PSK) cipher suites for both frontend
436 and backend TLS connections. For frontend connection, use
437 :option:`--psk-secrets` option to specify a file which contains PSK
438 identity and secrets. The format of the file is
439 ``<identity>:<hex-secret>``, where ``<identity>`` is PSK identity, and
440 ``<hex-secret>`` is PSK secret in hex, like so:
444 client1:9567800e065e078085c241d54a01c6c3f24b3bab71a606600f4c6ad2c134f3b9
445 client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99
447 nghttpx server accepts any of the identity and secret pairs in the
448 file. The default cipher suite list does not contain PSK cipher
449 suites. In order to use PSK, PSK cipher suite must be enabled by
450 using :option:`--ciphers` option. The desired PSK cipher suite may be
451 listed in `HTTP/2 cipher block list
452 <https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use
453 such PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list by
454 using :option:`--no-http2-cipher-block-list` option. But you should
455 understand its implications.
457 At the time of writing, even if only PSK cipher suites are specified
458 in :option:`--ciphers` option, certificate and private key are still
461 For backend connection, use :option:`--client-psk-secrets` option to
462 specify a file which contains single PSK identity and secret. The
463 format is the same as the file used by :option:`--psk-secrets`
464 described above, but only first identity and secret pair is solely
469 client2:b1376c3f8f6dcf7c886c5bdcceecd1e6f1d708622b6ddd21bda26ebd0c0bca99
471 The default cipher suite list does not contain PSK cipher suites. In
472 order to use PSK, PSK cipher suite must be enabled by using
473 :option:`--client-ciphers` option. The desired PSK cipher suite may
474 be listed in `HTTP/2 cipher block list
475 <https://tools.ietf.org/html/rfc7540#appendix-A>`_. In order to use
476 such PSK cipher suite with HTTP/2, disable HTTP/2 cipher block list by
477 using :option:`--client-no-http2-cipher-block-list` option. But you
478 should understand its implications.
483 As of nghttpx v1.34.0, if it is built with OpenSSL 1.1.1 or later, it
484 supports TLSv1.3. 0-RTT data is supported, but by default its
485 processing is postponed until TLS handshake completes to mitigate
486 replay attack. This costs extra round trip and reduces effectiveness
487 of 0-RTT data. :option:`--tls-no-postpone-early-data` makes nghttpx
488 not wait for handshake to complete before forwarding request included
489 in 0-RTT to get full potential of 0-RTT data. In this case, nghttpx
490 adds ``Early-Data: 1`` header field when forwarding a request to a
491 backend server. All backend servers should recognize this header
492 field and understand that there is a risk for replay attack. See `RFC
493 8470 <https://tools.ietf.org/html/rfc8470>`_ for ``Early-Data`` header
496 nghttpx disables anti replay protection provided by OpenSSL. The anti
497 replay protection of OpenSSL requires that a resumed request must hit
498 the same server which generates the session ticket. Therefore it
499 might not work nicely in a deployment where there are multiple nghttpx
500 instances sharing ticket encryption keys via memcached.
502 Because TLSv1.3 completely changes the semantics of cipher suite
503 naming scheme and structure, nghttpx provides the new option
504 :option:`--tls13-ciphers` and :option:`--tls13-client-ciphers` to
505 change preferred cipher list for TLSv1.3.
507 WebSockets over HTTP/2
508 ----------------------
510 nghttpx supports `RFC 8441 <https://tools.ietf.org/html/rfc8441>`_
511 Bootstrapping WebSockets with HTTP/2 for both frontend and backend
512 connections. This feature is enabled by default and no configuration
515 WebSockets over HTTP/3 is also supported.
520 nghttpx supports HTTP/3 if it is built with HTTP/3 support enabled.
521 HTTP/3 support is experimental.
523 In order to listen UDP port to receive HTTP/3 traffic,
524 :option:`--frontend` option must have ``quic`` parameter:
530 The above example makes nghttpx receive HTTP/3 traffic on UDP
533 nghttpx does not support HTTP/3 on backend connection.
535 Hot swapping (SIGUSR2) or configuration reload (SIGHUP) require eBPF
536 program. Without eBPF, old worker processes keep getting HTTP/3
537 traffic and do not work as intended. The QUIC keying material to
538 encrypt Connection ID must be set with
539 :option:`--frontend-quic-secret-file` and must provide the existing
540 keys in order to keep the existing connections alive during reload.
542 The construction of Connection ID closely follows Block Cipher CID
543 Algorithm described in `QUIC-LB draft
544 <https://datatracker.ietf.org/doc/html/draft-ietf-quic-load-balancers>`_.
545 A Connection ID that nghttpx generates is always 20 bytes long. It
546 uses first 2 bits as a configuration ID. The remaining bits in the
547 first byte are reserved and random. The next 4 bytes are server ID.
548 The next 4 bytes are used to route UDP datagram to a correct
549 ``SO_REUSEPORT`` socket. The remaining bytes are randomly generated.
550 The server ID and the next 12 bytes are encrypted with AES-ECB. The
551 key is derived from the keying materials stored in a file specified by
552 :option:`--frontend-quic-secret-file`. The first 2 bits of keying
553 material in the file is used as a configuration ID. The remaining
554 bits and following 3 bytes are reserved and unused. The next 32 bytes
555 are used as an initial secret. The remaining 32 bytes are used as a
556 salt. The encryption key is generated by `HKDF
557 <https://datatracker.ietf.org/doc/html/rfc5869>`_ with SHA256 and
558 these keying materials and ``connection id encryption key`` as info.
560 In order announce that HTTP/3 endpoint is available, you should
561 specify alt-svc header field. For example, the following options send
562 alt-svc header field in HTTP/1.1 and HTTP/2 response:
566 altsvc=h3,443,,,ma=3600
567 http2-altsvc=h3,443,,,ma=3600
569 Migration from nghttpx v1.18.x or earlier
570 -----------------------------------------
572 As of nghttpx v1.19.0, :option:`--ciphers` option only changes cipher
573 list for frontend TLS connection. In order to change cipher list for
574 backend connection, use :option:`--client-ciphers` option.
576 Similarly, :option:`--no-http2-cipher-block-list` option only disables
577 HTTP/2 cipher block list for frontend connection. In order to disable
578 HTTP/2 cipher block list for backend connection, use
579 :option:`--client-no-http2-cipher-block-list` option.
581 ``--accept-proxy-protocol`` option was deprecated. Instead, use
582 ``proxyproto`` parameter in :option:`--frontend` option to enable
583 PROXY protocol support per frontend.
585 Migration from nghttpx v1.8.0 or earlier
586 ----------------------------------------
588 As of nghttpx 1.9.0, ``--frontend-no-tls`` and ``--backend-no-tls``
591 To disable encryption on frontend connection, use ``no-tls`` keyword
592 in :option:`--frontend` potion:
596 frontend=*,3000;no-tls
598 The TLS encryption is now disabled on backend connection in all modes
599 by default. To enable encryption on backend connection, use ``tls``
600 keyword in :option:`--backend` option:
604 backend=127.0.0.1,8080;tls
606 As of nghttpx 1.9.0, ``--http2-bridge``, ``--client`` and
607 ``--client-proxy`` options have been removed. These functionality can
608 be used using combinations of options.
610 Use following option instead of ``--http2-bridge``:
614 backend=<ADDR>,<PORT>;;proto=h2;tls
616 Use following options instead of ``--client``:
620 frontend=<ADDR>,<PORT>;no-tls
621 backend=<ADDR>,<PORT>;;proto=h2;tls
623 Use following options instead of ``--client-proxy``:
628 frontend=<ADDR>,<PORT>;no-tls
629 backend=<ADDR>,<PORT>;;proto=h2;tls
631 We also removed ``--backend-http2-connections-per-worker`` option. It
632 was present because previously the number of backend h2 connection was
633 statically configured, and defaulted to 1. Now the number of backend
634 h2 connection is increased on demand. We know the maximum number of
635 concurrent streams per connection. When we push as many request as
636 the maximum concurrency to the one connection, we create another new
637 connection so that we can distribute load and avoid delay the request
638 processing. This is done automatically without any configuration.