1 Notes about coding with lws
2 ===========================
4 @section era Old lws and lws v2.0
6 Originally lws only supported the "manual" method of handling everything in the
7 user callback found in test-server.c / test-server-http.c.
9 Since v2.0, the need for most or all of this manual boilerplate has been
10 eliminated: the protocols[0] http stuff is provided by a generic lib export
11 `lws_callback_http_dummy()`. You can serve parts of your filesystem at part of
12 the URL space using mounts, the dummy http callback will do the right thing.
14 It's much preferred to use the "automated" v2.0 type scheme, because it's less
15 code and it's easier to support.
17 The minimal examples all use the modern, recommended way.
19 If you just need generic serving capability, without the need to integrate lws
20 to some other app, consider not writing any server code at all, and instead use
21 the generic server `lwsws`, and writing your special user code in a standalone
22 "plugin". The server is configured for mounts etc using JSON, see
23 ./READMEs/README.lwsws.md.
25 Although the "plugins" are dynamically loaded if you use lwsws or lws built
26 with libuv, actually they may perfectly well be statically included if that
27 suits your situation better, eg, ESP32 test server, where the platform does
28 not support processes or dynamic loading, just #includes the plugins
29 one after the other and gets the same benefit from the same code.
31 Isolating and collating the protocol code in one place also makes it very easy
32 to maintain and understand.
34 So it if highly recommended you put your protocol-specific code into the
35 form of a "plugin" at the source level, even if you have no immediate plan to
36 use it dynamically-loaded.
38 @section writeable Only send data when socket writeable
40 You should only send data on a websocket connection from the user callback
41 `LWS_CALLBACK_SERVER_WRITEABLE` (or `LWS_CALLBACK_CLIENT_WRITEABLE` for
44 If you want to send something, do NOT just send it but request a callback
45 when the socket is writeable using
47 - `lws_callback_on_writable(wsi)` for a specific `wsi`, or
49 - `lws_callback_on_writable_all_protocol(protocol)` for all connections
50 using that protocol to get a callback when next writeable.
52 Usually you will get called back immediately next time around the service
53 loop, but if your peer is slow or temporarily inactive the callback will be
54 delayed accordingly. Generating what to write and sending it should be done
55 in the ...WRITEABLE callback.
57 See the test server code for an example of how to do this.
59 Otherwise evolved libs like libuv get this wrong, they will allow you to "send"
60 anything you want but it only uses up your local memory (and costs you
61 memcpys) until the socket can actually accept it. It is much better to regulate
62 your send action by the downstream peer readiness to take new data in the first
63 place, avoiding all the wasted buffering.
65 Libwebsockets' concept is that the downstream peer is truly the boss, if he,
66 or our connection to him, cannot handle anything new, we should not generate
67 anything new for him. This is how unix shell piping works, you may have
68 `cat a.txt | grep xyz > remote", but actually that does not cat anything from
69 a.txt while remote cannot accept anything new.
71 @section oneper Only one lws_write per WRITEABLE callback
73 From v2.5, lws strictly enforces only one lws_write() per WRITEABLE callback.
75 You will receive a message about "Illegal back-to-back write of ... detected"
76 if there is a second lws_write() before returning to the event loop.
78 This is because with http/2, the state of the network connection carrying a
79 wsi is unrelated to any state of the wsi. The situation on http/1 where a
80 new request implied a new tcp connection and new SSL buffer, so you could
81 assume some window for writes is no longer true. Any lws_write() can fail
82 and be buffered for completion by lws; it will be auto-completed by the
85 Note that if you are handling your own http responses, writing the headers
86 needs to be done with a separate lws_write() from writing any payload. That
87 means after writing the headers you must call `lws_callback_on_writable(wsi)`
88 and send any payload from the writable callback.
90 @section otherwr Do not rely on only your own WRITEABLE requests appearing
92 Libwebsockets may generate additional `LWS_CALLBACK_CLIENT_WRITEABLE` events
93 if it met network conditions where it had to buffer your send data internally.
95 So your code for `LWS_CALLBACK_CLIENT_WRITEABLE` needs to own the decision
96 about what to send, it can't assume that just because the writeable callback
97 came something is ready to send.
99 It's quite possible you get an 'extra' writeable callback at any time and
100 just need to `return 0` and wait for the expected callback later.
102 @section dae Daemonization
104 There's a helper api `lws_daemonize` built by default that does everything you
105 need to daemonize well, including creating a lock file. If you're making
106 what's basically a daemon, just call this early in your init to fork to a
107 headless background process and exit the starting process.
109 Notice stdout, stderr, stdin are all redirected to /dev/null to enforce your
110 daemon is headless, so you'll need to sort out alternative logging, by, eg,
111 syslog via `lws_set_log_level(..., lwsl_emit_syslog)`.
113 @section conns Maximum number of connections
115 The maximum number of connections the library can deal with is decided when
116 it starts by querying the OS to find out how many file descriptors it is
117 allowed to open (1024 on Fedora for example). It then allocates arrays that
118 allow up to that many connections, minus whatever other file descriptors are
119 in use by the user code.
121 If you want to restrict that allocation, or increase it, you can use ulimit or
122 similar to change the available number of file descriptors, and when restarted
123 **libwebsockets** will adapt accordingly.
125 @section peer_limits optional LWS_WITH_PEER_LIMITS
127 If you select `LWS_WITH_PEER_LIMITS` at cmake, then lws will track peer IPs
128 and monitor how many connections and ah resources they are trying to use
129 at one time. You can choose to limit these at context creation time, using
130 `info.ip_limit_ah` and `info.ip_limit_wsi`.
132 Note that although the ah limit is 'soft', ie, the connection will just wait
133 until the IP is under the ah limit again before attaching a new ah, the
134 wsi limit is 'hard', lws will drop any additional connections from the
135 IP until it's under the limit again.
137 If you use these limits, you should consider multiple clients may simultaneously
138 try to access the site through NAT, etc. So the limits should err on the side
139 of being generous, while still making it impossible for one IP to exhaust
140 all the server resources.
142 @section evtloop Libwebsockets is singlethreaded
144 Libwebsockets works in a serialized event loop, in a single thread. It supports
145 the default poll() backend, and libuv, libev, and libevent event loop
146 libraries that also take this locking-free, nonblocking event loop approach that
147 is not threadsafe. There are several advantages to this technique, but one
148 disadvantage, it doesn't integrate easily if there are multiple threads that
149 want to use libwebsockets.
151 However integration to multithreaded apps is possible if you follow some guidelines.
153 1) Aside from two APIs, directly calling lws apis from other threads is not allowed.
155 2) If you want to keep a list of live wsi, you need to use lifecycle callbacks on
156 the protocol in the service thread to manage the list, with your own locking.
157 Typically you use an ESTABLISHED callback to add ws wsi to your list and a CLOSED
158 callback to remove them.
160 3) LWS regulates your write activity by being able to let you know when you may
161 write more on a connection. That reflects the reality that you cannot succeed to
162 send data to a peer that has no room for it, so you should not generate or buffer
163 write data until you know the peer connection can take more.
165 Other libraries pretend that the guy doing the writing is the boss who decides
166 what happens, and absorb as much as you want to write to local buffering. That does
167 not scale to a lot of connections, because it will exhaust your memory and waste
168 time copying data around in memory needlessly.
170 The truth is the receiver, along with the network between you, is the boss who
171 decides what will happen. If he stops accepting data, no data will move. LWS is
172 designed to reflect that.
174 If you have something to send, you call `lws_callback_on_writable()` on the
175 connection, and when it is writeable, you will get a `LWS_CALLBACK_SERVER_WRITEABLE`
176 callback, where you should generate the data to send and send it with `lws_write()`.
178 You cannot send data using `lws_write()` outside of the WRITEABLE callback.
180 4) For multithreaded apps, this corresponds to a need to be able to provoke the
181 `lws_callback_on_writable()` action and to wake the service thread from its event
182 loop wait (sleeping in `poll()` or `epoll()` or whatever). The rules above
183 mean directly sending data on the connection from another thread is out of the
186 Therefore the two apis mentioned above that may be used from another thread are
188 - For LWS using the default poll() event loop, `lws_callback_on_writable()`
190 - For LWS using libuv/libev/libevent event loop, `lws_cancel_service()`
192 If you are using the default poll() event loop, one "foreign thread" at a time may
193 call `lws_callback_on_writable()` directly for a wsi. You need to use your own
194 locking around that to serialize multiple thread access to it.
196 If you implement LWS_CALLBACK_GET_THREAD_ID in protocols[0], then LWS will detect
197 when it has been called from a foreign thread and automatically use
198 `lws_cancel_service()` to additionally wake the service loop from its wait.
200 For libuv/libev/libevent event loop, they cannot handle being called from other
201 threads. So there is a slightly different scheme, you may call `lws_cancel_service()`
202 to force the event loop to end immediately. This then broadcasts a callback (in the
203 service thread context) `LWS_CALLBACK_EVENT_WAIT_CANCELLED`, to all protocols on all
204 vhosts, where you can perform your own locking and walk a list of wsi that need
205 `lws_callback_on_writable()` calling on them.
207 `lws_cancel_service()` is very cheap to call.
209 5) The obverse of this truism about the receiver being the boss is the case where
210 we are receiving. If we get into a situation we actually can't usefully
211 receive any more, perhaps because we are passing the data on and the guy we want
212 to send to can't receive any more, then we should "turn off RX" by using the
213 RX flow control API, `lws_rx_flow_control(wsi, 0)`. When something happens where we
214 can accept more RX, (eg, we learn our onward connection is writeable) we can call
215 it again to re-enable it on the incoming wsi.
217 LWS stops calling back about RX immediately you use flow control to disable RX, it
218 buffers the data internally if necessary. So you will only see RX when you can
219 handle it. When flow control is disabled, LWS stops taking new data in... this makes
220 the situation known to the sender by TCP "backpressure", the tx window fills and the
221 sender finds he cannot write any more to the connection.
223 See the mirror protocol implementations for example code.
225 If you need to service other socket or file descriptors as well as the
226 websocket ones, you can combine them together with the websocket ones
227 in one poll loop, see "External Polling Loop support" below, and
228 still do it all in one thread / process context. If the need is less
229 architectural, you can also create RAW mode client and serving sockets; this
230 is how the lws plugin for the ssh server works.
232 @section anonprot Working without a protocol name
234 Websockets allows connections to negotiate without a protocol name...
235 in that case by default it will bind to the first protocol in your
236 vhost protocols[] array.
238 You can tell the vhost to use a different protocol by attaching a
239 pvo (per-vhost option) to the
243 * this sets a per-vhost, per-protocol option name:value pair
244 * the effect is to set this protocol to be the default one for the vhost,
245 * ie, selected if no Protocol: header is sent with the ws upgrade.
248 static const struct lws_protocol_vhost_options pvo_opt = {
255 static const struct lws_protocol_vhost_options pvo = {
264 context_info.pvo = &pvo;
269 Will select "my-protocol" from your protocol list (even if it came
270 in by plugin) as being the target of client connections that don't
273 @section closing Closing connections from the user side
275 When you want to close a connection, you do it by returning `-1` from a
276 callback for that connection.
278 You can provoke a callback by calling `lws_callback_on_writable` on
279 the wsi, then notice in the callback you want to close it and just return -1.
280 But usually, the decision to close is made in a callback already and returning
283 If the socket knows the connection is dead, because the peer closed or there
284 was an affirmitive network error like a FIN coming, then **libwebsockets** will
285 take care of closing the connection automatically.
287 If you have a silently dead connection, it's possible to enter a state where
288 the send pipe on the connection is choked but no ack will ever come, so the
289 dead connection will never become writeable. To cover that, you can use TCP
290 keepalives (see later in this document) or pings.
292 @section gzip Serving from inside a zip file
294 Lws now supports serving gzipped files from inside a zip container. Thanks to
295 Per Bothner for contributing the code.
297 This has the advtantage that if the client can accept GZIP encoding, lws can
298 simply send the gzip-compressed file from inside the zip file with no further
299 processing, saving time and bandwidth.
301 In the case the client can't understand gzip compression, lws automatically
302 decompressed the file and sends it normally.
304 Clients with limited storage and RAM will find this useful; the memory needed
305 for the inflate case is constrained so that only one input buffer at a time
308 To use this feature, ensure LWS_WITH_ZIP_FOPS is enabled at CMake.
310 `libwebsockets-test-server-v2.0` includes a mount using this technology
311 already, run that test server and navigate to http://localhost:7681/ziptest/candide.html
313 This will serve the book Candide in html, together with two jpgs, all from
314 inside a .zip file in /usr/[local/]share-libwebsockets-test-server/candide.zip
316 Usage is otherwise automatic, if you arrange a mount that points to the zipfile,
317 eg, "/ziptest" -> "mypath/test.zip", then URLs like `/ziptest/index.html` will be
318 servied from `index.html` inside `mypath/test.zip`
320 @section frags Fragmented messages
322 To support fragmented messages you need to check for the final
323 frame of a message with `lws_is_final_fragment`. This
324 check can be combined with `libwebsockets_remaining_packet_payload`
325 to gather the whole contents of a message, eg:
328 case LWS_CALLBACK_RECEIVE:
330 Client * const client = (Client *)user;
331 const size_t remaining = lws_remaining_packet_payload(wsi);
333 if (!remaining && lws_is_final_fragment(wsi)) {
334 if (client->HasFragments()) {
335 client->AppendMessageFragment(in, len, 0);
336 in = (void *)client->GetMessage();
337 len = client->GetMessageLength();
340 client->ProcessMessage((char *)in, len, wsi);
341 client->ResetMessage();
343 client->AppendMessageFragment(in, len, remaining);
348 The test app libwebsockets-test-fraggle sources also show how to
349 deal with fragmented messages.
352 @section debuglog Debug Logging
354 Also using `lws_set_log_level` api you may provide a custom callback to actually
355 emit the log string. By default, this points to an internal emit function
356 that sends to stderr. Setting it to `NULL` leaves it as it is instead.
358 A helper function `lwsl_emit_syslog()` is exported from the library to simplify
359 logging to syslog. You still need to use `setlogmask`, `openlog` and `closelog`
362 The logging apis are made available for user code.
370 The difference between notice and info is that notice will be logged by default
371 whereas info is ignored by default.
373 If you are not building with _DEBUG defined, ie, without this
376 $ cmake .. -DCMAKE_BUILD_TYPE=DEBUG
379 then log levels below notice do not actually get compiled in.
381 @section asan Building with ASAN
383 Under GCC you can select for the build to be instrumented with the Address
384 Sanitizer, using `cmake .. -DCMAKE_BUILD_TYPE=DEBUG -DLWS_WITH_ASAN=1`. LWS is routinely run during development with valgrind, but ASAN is capable of finding different issues at runtime, like operations which are not strictly defined in the C
385 standard and depend on platform behaviours.
387 Run your application like this
390 $ sudo ASAN_OPTIONS=verbosity=2:halt_on_error=1 /usr/local/bin/lwsws
393 and attach gdb to catch the place it halts.
395 @section extpoll External Polling Loop support
397 **libwebsockets** maintains an internal `poll()` array for all of its
398 sockets, but you can instead integrate the sockets into an
399 external polling array. That's needed if **libwebsockets** will
400 cooperate with an existing poll array maintained by another
403 Three callbacks `LWS_CALLBACK_ADD_POLL_FD`, `LWS_CALLBACK_DEL_POLL_FD`
404 and `LWS_CALLBACK_CHANGE_MODE_POLL_FD` appear in the callback for protocol 0
405 and allow interface code to manage socket descriptors in other poll loops.
407 You can pass all pollfds that need service to `lws_service_fd()`, even
408 if the socket or file does not belong to **libwebsockets** it is safe.
410 If **libwebsocket** handled it, it zeros the pollfd `revents` field before returning.
411 So you can let **libwebsockets** try and if `pollfd->revents` is nonzero on return,
412 you know it needs handling by your code.
414 Also note that when integrating a foreign event loop like libev or libuv where
415 it doesn't natively use poll() semantics, and you must return a fake pollfd
416 reflecting the real event:
418 - be sure you set .events to .revents value as well in the synthesized pollfd
420 - check the built-in support for the event loop if possible (eg, ./lib/libuv.c)
421 to see how it interfaces to lws
423 - use LWS_POLLHUP / LWS_POLLIN / LWS_POLLOUT from libwebsockets.h to avoid
424 losing windows compatibility
426 You also need to take care about "forced service" somehow... these are cases
427 where the network event was consumed, incoming data was all read, for example,
428 but the work arising from it was not completed. There will not be any more
429 network event to trigger the remaining work, Eg, we read compressed data, but
430 we did not use up all the decompressed data before returning to the event loop
431 because we had to write some of it.
433 Lws provides an API to determine if anyone is waiting for forced service,
434 `lws_service_adjust_timeout(context, 1, tsi)`, normally tsi is 0. If it returns
435 0, then at least one connection has pending work you can get done by calling
436 `lws_service_tsi(context, -1, tsi)`, again normally tsi is 0.
438 For eg, the default poll() event loop, or libuv/ev/event, lws does this
439 checking for you and handles it automatically. But in the external polling
440 loop case, you must do it explicitly. Handling it after every normal service
441 triggered by the external poll fd should be enough, since the situations needing
442 it are initially triggered by actual network events.
444 An example of handling it is shown in the test-server code specific to
447 @section cpp Using with in c++ apps
449 The library is ready for use by C++ apps. You can get started quickly by
450 copying the test server
453 $ cp test-apps/test-server.c test.cpp
456 and building it in C++ like this
459 $ g++ -DINSTALL_DATADIR=\"/usr/share\" -ocpptest test.cpp -lwebsockets
462 `INSTALL_DATADIR` is only needed because the test server uses it as shipped, if
463 you remove the references to it in your app you don't need to define it on
467 @section headerinfo Availability of header information
469 HTTP Header information is managed by a pool of "ah" structs. These are a
470 limited resource so there is pressure to free the headers and return the ah to
473 For that reason header information on HTTP connections that get upgraded to
474 websockets is lost after the ESTABLISHED callback. Anything important that
475 isn't processed by user code before then should be copied out for later.
477 For HTTP connections that don't upgrade, header info remains available the
480 @section http2compat Code Requirements for HTTP/2 compatibility
482 Websocket connections only work over http/1, so there is nothing special to do
483 when you want to enable -DLWS_WITH_HTTP2=1.
485 The internal http apis already follow these requirements and are compatible with
486 http/2 already. So if you use stuff like mounts and serve stuff out of the
487 filesystem, there's also nothing special to do.
489 However if you are getting your hands dirty with writing response headers, or
490 writing bulk data over http/2, you need to observe these rules so that it will
491 work over both http/1.x and http/2 the same.
493 1) LWS_PRE requirement applies on ALL lws_write(). For http/1, you don't have
494 to take care of LWS_PRE for http data, since it is just sent straight out.
495 For http/2, it will write up to LWS_PRE bytes behind the buffer start to create
496 the http/2 frame header.
498 This has implications if you treated the input buffer to lws_write() as const...
499 it isn't any more with http/2, up to 9 bytes behind the buffer will be trashed.
501 2) Headers are encoded using a sophisticated scheme in http/2. The existing
502 header access apis are already made compatible for incoming headers,
503 for outgoing headers you must:
505 - observe the LWS_PRE buffer requirement mentioned above
507 - Use `lws_add_http_header_status()` to add the transaction status (200 etc)
509 - use lws apis `lws_add_http_header_by_name()` and `lws_add_http_header_by_token()`
510 to put the headers into the buffer (these will translate what is actually
511 written to the buffer depending on if the connection is in http/2 mode or not)
513 - use the `lws api lws_finalize_http_header()` api after adding the last
516 - write the header using lws_write(..., `LWS_WRITE_HTTP_HEADERS`);
518 3) http/2 introduces per-stream transmit credit... how much more you can send
519 on a stream is decided by the peer. You start off with some amount, as the
520 stream sends stuff lws will reduce your credit accordingly, when it reaches
521 zero, you must not send anything further until lws receives "more credit" for
522 that stream the peer. Lws will suppress writable callbacks if you hit 0 until
523 more credit for the stream appears, and lws built-in file serving (via mounts
524 etc) already takes care of observing the tx credit restrictions. However if
525 you write your own code that wants to send http data, you must consult the
526 `lws_get_peer_write_allowance()` api to find out the state of your tx credit.
527 For http/1, it will always return (size_t)-1, ie, no limit.
529 This is orthogonal to the question of how much space your local side's kernel
530 will make to buffer your send data on that connection. So although the result
531 from `lws_get_peer_write_allowance()` is "how much you can send" logically,
532 and may be megabytes if the peer allows it, you should restrict what you send
533 at one time to whatever your machine will generally accept in one go, and
534 further reduce that amount if `lws_get_peer_write_allowance()` returns
535 something smaller. If it returns 0, you should not consume or send anything
536 and return having asked for callback on writable, it will only come back when
537 more tx credit has arrived for your stream.
539 4) Header names with captital letters are illegal in http/2. Header names in
540 http/1 are case insensitive. So if you generate headers by name, change all
541 your header name strings to lower-case to be compatible both ways.
543 5) Chunked Transfer-encoding is illegal in http/2, http/2 peers will actively
544 reject it. Lws takes care of removing the header and converting CGIs that
545 emit chunked into unchunked automatically for http/2 connections.
547 If you follow these rules, your code will automatically work with both http/1.x
550 @section ka TCP Keepalive
552 It is possible for a connection which is not being used to send to die
553 silently somewhere between the peer and the side not sending. In this case
554 by default TCP will just not report anything and you will never get any more
555 incoming data or sign the link is dead until you try to send.
557 To deal with getting a notification of that situation, you can choose to
558 enable TCP keepalives on all **libwebsockets** sockets, when you create the
561 To enable keepalive, set the ka_time member of the context creation parameter
562 struct to a nonzero value (in seconds) at context creation time. You should
563 also fill ka_probes and ka_interval in that case.
565 With keepalive enabled, the TCP layer will send control packets that should
566 stimulate a response from the peer without affecting link traffic. If the
567 response is not coming, the socket will announce an error at `poll()` forcing
570 Note that BSDs don't support keepalive time / probes / interval per-socket
571 like Linux does. On those systems you can enable keepalive by a nonzero
572 value in `ka_time`, but the systemwide kernel settings for the time / probes/
573 interval are used, regardless of what nonzero value is in `ka_time`.
576 @section sslopt Optimizing SSL connections
578 There's a member `ssl_cipher_list` in the `lws_context_creation_info` struct
579 which allows the user code to restrict the possible cipher selection at
580 context-creation time.
582 You might want to look into that to stop the ssl peers selecting a cipher which
583 is too computationally expensive. To use it, point it to a string like
585 `"RC4-MD5:RC4-SHA:AES128-SHA:AES256-SHA:HIGH:!DSS:!aNULL"`
587 if left `NULL`, then the "DEFAULT" set of ciphers are all possible to select.
589 You can also set it to `"ALL"` to allow everything (including insecure ciphers).
592 @section sslcerts Passing your own cert information direct to SSL_CTX
594 For most users it's enough to pass the SSL certificate and key information by
595 giving filepaths to the info.ssl_cert_filepath and info.ssl_private_key_filepath
596 members when creating the vhost.
598 If you want to control that from your own code instead, you can do so by leaving
599 the related info members NULL, and setting the info.options flag
600 LWS_SERVER_OPTION_CREATE_VHOST_SSL_CTX at vhost creation time. That will create
601 the vhost SSL_CTX without any certificate, and allow you to use the callback
602 LWS_CALLBACK_OPENSSL_LOAD_EXTRA_SERVER_VERIFY_CERTS to add your certificate to
603 the SSL_CTX directly. The vhost SSL_CTX * is in the user parameter in that
606 @section clientasync Async nature of client connections
608 When you call `lws_client_connect_info(..)` and get a `wsi` back, it does not
609 mean your connection is active. It just means it started trying to connect.
611 Your client connection is actually active only when you receive
612 `LWS_CALLBACK_CLIENT_ESTABLISHED` for it.
614 There's a 5 second timeout for the connection, and it may give up or die for
615 other reasons, if any of that happens you'll get a
616 `LWS_CALLBACK_CLIENT_CONNECTION_ERROR` callback on protocol 0 instead for the
619 After attempting the connection and getting back a non-`NULL` `wsi` you should
620 loop calling `lws_service()` until one of the above callbacks occurs.
622 As usual, see [test-client.c](../test-apps/test-client.c) for example code.
624 Notice that the client connection api tries to progress the connection
625 somewhat before returning. That means it's possible to get callbacks like
626 CONNECTION_ERROR on the new connection before your user code had a chance to
627 get the wsi returned to identify it (in fact if the connection did fail early,
628 NULL will be returned instead of the wsi anyway).
630 To avoid that problem, you can fill in `pwsi` in the client connection info
631 struct to point to a struct lws that get filled in early by the client
632 connection api with the related wsi. You can then check for that in the
633 callback to confirm the identity of the failing client connection.
636 @section fileapi Lws platform-independent file access apis
638 lws now exposes his internal platform file abstraction in a way that can be
639 both used by user code to make it platform-agnostic, and be overridden or
640 subclassed by user code. This allows things like handling the URI "directory
641 space" as a virtual filesystem that may or may not be backed by a regular
642 filesystem. One example use is serving files from inside large compressed
643 archive storage without having to unpack anything except the file being
646 The test server shows how to use it, basically the platform-specific part of
647 lws prepares a file operations structure that lives in the lws context.
649 The user code can get a pointer to the file operations struct
652 LWS_VISIBLE LWS_EXTERN struct lws_plat_file_ops *
653 `lws_get_fops`(struct lws_context *context);
656 and then can use helpers to also leverage these platform-independent
661 `lws_plat_file_open`(struct lws_plat_file_ops *fops, const char *filename,
662 lws_fop_flags_t *flags)
664 `lws_plat_file_close`(lws_fop_fd_t fop_fd)
667 `lws_plat_file_seek_cur`(lws_fop_fd_t fop_fd, lws_fileofs_t offset)
670 `lws_plat_file_read`(lws_fop_fd_t fop_fd, lws_filepos_t *amount,
671 uint8_t *buf, lws_filepos_t len)
674 `lws_plat_file_write`(lws_fop_fd_t fop_fd, lws_filepos_t *amount,
675 uint8_t *buf, lws_filepos_t len )
678 Generic helpers are provided which provide access to generic fops information or
679 call through to the above fops
683 lws_vfs_tell(lws_fop_fd_t fop_fd);
686 lws_vfs_get_length(lws_fop_fd_t fop_fd);
689 lws_vfs_get_mod_time(lws_fop_fd_t fop_fd);
692 lws_vfs_file_seek_set(lws_fop_fd_t fop_fd, lws_fileofs_t offset);
695 lws_vfs_file_seek_end(lws_fop_fd_t fop_fd, lws_fileofs_t offset);
699 The user code can also override or subclass the file operations, to either
700 wrap or replace them. An example is shown in test server.
702 ### Changes from v2.1 and before fops
704 There are several changes:
706 1) Pre-2.2 fops directly used platform file descriptors. Current fops returns and accepts a wrapper type lws_fop_fd_t which is a pointer to a malloc'd struct containing information specific to the filesystem implementation.
708 2) Pre-2.2 fops bound the fops to a wsi. This is completely removed, you just give a pointer to the fops struct that applies to this file when you open it. Afterwards, the operations in the fops just need the lws_fop_fd_t returned from the open.
710 3) Everything is wrapped in typedefs. See lws-plat-unix.c for examples of how to implement.
712 4) Position in the file, File Length, and a copy of Flags left after open are now generically held in the fop_fd.
713 VFS implementation must set and manage this generic information now. See the implementations in lws-plat-unix.c for
716 5) The file length is no longer set at a pointer provided by the open() fop. The api `lws_vfs_get_length()` is provided to
717 get the file length after open.
719 6) If your file namespace is virtual, ie, is not reachable by platform fops directly, you must set LWS_FOP_FLAG_VIRTUAL
720 on the flags during open.
722 7) There is an optional `mod_time` uint32_t member in the generic fop_fd. If you are able to set it during open, you
723 should indicate it by setting `LWS_FOP_FLAG_MOD_TIME_VALID` on the flags.
725 @section rawfd RAW file descriptor polling
727 LWS allows you to include generic platform file descriptors in the lws service / poll / event loop.
729 Open your fd normally and then
732 lws_sock_file_fd_type u;
734 u.filefd = your_open_file_fd;
736 if (!lws_adopt_descriptor_vhost(vhost, 0, u,
737 "protocol-name-to-bind-to",
738 optional_wsi_parent_or_NULL)) {
745 A wsi is created for the file fd that acts like other wsi, you will get these
746 callbacks on the named protocol
749 LWS_CALLBACK_RAW_ADOPT_FILE
750 LWS_CALLBACK_RAW_RX_FILE
751 LWS_CALLBACK_RAW_WRITEABLE_FILE
752 LWS_CALLBACK_RAW_CLOSE_FILE
755 starting with LWS_CALLBACK_RAW_ADOPT_FILE.
757 The minimal example `raw/minimal-raw-file` demonstrates how to use it.
759 `protocol-lws-raw-test` plugin also provides a method for testing this with
760 `libwebsockets-test-server-v2.0`:
762 The plugin creates a FIFO on your system called "/tmp/lws-test-raw"
764 You can feed it data through the FIFO like this
767 $ sudo sh -c "echo hello > /tmp/lws-test-raw"
770 This plugin simply prints the data. But it does it through the lws event
773 @section rawsrvsocket RAW server socket descriptor polling
775 You can also enable your vhost to accept RAW socket connections, in addition to
776 HTTP[s] and WS[s]. If the first bytes written on the connection are not a
777 valid HTTP method, then the connection switches to RAW mode.
779 This is disabled by default, you enable it by setting the `.options` flag
780 LWS_SERVER_OPTION_FALLBACK_TO_APPLY_LISTEN_ACCEPT_CONFIG, and setting
781 `.listen_accept_role` to `"raw-skt"` when creating the vhost.
783 RAW mode socket connections receive the following callbacks
786 LWS_CALLBACK_RAW_ADOPT
788 LWS_CALLBACK_RAW_WRITEABLE
789 LWS_CALLBACK_RAW_CLOSE
792 You can control which protocol on your vhost handles these RAW mode
793 incoming connections by setting the vhost info struct's `.listen_accept_protocol`
794 to the vhost protocol name to use.
796 `protocol-lws-raw-test` plugin provides a method for testing this with
797 `libwebsockets-test-server-v2.0`:
799 Run libwebsockets-test-server-v2.0 and connect to it by telnet, eg
802 $ telnet 127.0.0.1 7681
805 type something that isn't a valid HTTP method and enter, before the
806 connection times out. The connection will switch to RAW mode using this
807 protocol, and pass the unused rx as a raw RX callback.
809 The test protocol echos back what was typed on telnet to telnet.
811 @section rawclientsocket RAW client socket descriptor polling
813 You can now also open RAW socket connections in client mode.
815 Follow the usual method for creating a client connection, but set the
816 `info.method` to "RAW". When the connection is made, the wsi will be
817 converted to RAW mode and operate using the same callbacks as the
818 server RAW sockets described above.
820 The libwebsockets-test-client supports this using raw:// URLS. To
821 test, open a netcat listener in one window
827 and in another window, connect to it using the test client
830 $ libwebsockets-test-client raw://127.0.0.1:9999
833 The connection should succeed, and text typed in the netcat window (including a CRLF)
834 will be received in the client.
836 @section rawudp RAW UDP socket integration
838 Lws provides an api to create, optionally bind, and adopt a RAW UDP
839 socket (RAW here means an uninterpreted normal UDP socket, not a
843 LWS_VISIBLE LWS_EXTERN struct lws *
844 lws_create_adopt_udp(struct lws_vhost *vhost, int port, int flags,
845 const char *protocol_name, struct lws *parent_wsi);
848 `flags` should be `LWS_CAUDP_BIND` if the socket will receive packets.
850 The callbacks `LWS_CALLBACK_RAW_ADOPT`, `LWS_CALLBACK_RAW_CLOSE`,
851 `LWS_CALLBACK_RAW_RX` and `LWS_CALLBACK_RAW_WRITEABLE` apply to the
852 wsi. But UDP is different than TCP in some fundamental ways.
854 For receiving on a UDP connection, data becomes available at
855 `LWS_CALLBACK_RAW_RX` as usual, but because there is no specific
856 connection with UDP, it is necessary to also get the source address of
857 the data separately, using `struct lws_udp * lws_get_udp(wsi)`.
858 You should take a copy of the `struct lws_udp` itself (not the
859 pointer) and save it for when you want to write back to that peer.
861 Writing is also a bit different for UDP. By default, the system has no
862 idea about the receiver state and so asking for a `callback_on_writable()`
863 always believes that the socket is writeable... the callback will
864 happen next time around the event loop.
866 With UDP, there is no single "connection". You need to write with sendto() and
867 direct the packets to a specific destination. To return packets to a
868 peer who sent something earlier and you copied his `struct lws_udp`, you
869 use the .sa and .salen members as the last two parameters of the sendto().
871 The kernel may not accept to buffer / write everything you wanted to send.
872 So you are responsible to watch the result of sendto() and resend the
873 unsent part next time (which may involve adding new protocol headers to
874 the remainder depending on what you are doing).
876 @section ecdh ECDH Support
878 ECDH Certs are now supported. Enable the CMake option
880 cmake .. -DLWS_SSL_SERVER_WITH_ECDH_CERT=1
882 **and** the info->options flag
884 LWS_SERVER_OPTION_SSL_ECDH
886 to build in support and select it at runtime.
888 @section sslinfo SSL info callbacks
890 OpenSSL allows you to receive callbacks for various events defined in a
891 bitmask in openssl/ssl.h. The events include stuff like TLS Alerts.
893 By default, lws doesn't register for these callbacks.
895 However if you set the info.ssl_info_event_mask to nonzero (ie, set some
896 of the bits in it like `SSL_CB_ALERT` at vhost creation time, then
897 connections to that vhost will call back using LWS_CALLBACK_SSL_INFO
898 for the wsi, and the `in` parameter will be pointing to a struct of
902 struct lws_ssl_info {
908 The default callback handler in lws has a handler for LWS_CALLBACK_SSL_INFO
909 which prints the related information, You can test it using the switch
910 -S -s on `libwebsockets-test-server-v2.0`.
912 Returning nonzero from the callback will close the wsi.
914 @section smp SMP / Multithreaded service
916 SMP support is integrated into LWS without any internal threading. It's
917 very simple to use, libwebsockets-test-server-pthread shows how to do it,
918 use -j n argument there to control the number of service threads up to 32.
920 Two new members are added to the info struct
922 unsigned int count_threads;
923 unsigned int fd_limit_per_thread;
925 leave them at the default 0 to get the normal singlethreaded service loop.
927 Set count_threads to n to tell lws you will have n simultaneous service threads
928 operating on the context.
930 There is still a single listen socket on one port, no matter how many
933 When a connection is made, it is accepted by the service thread with the least
934 connections active to perform load balancing.
936 The user code is responsible for spawning n threads running the service loop
937 associated to a specific tsi (Thread Service Index, 0 .. n - 1). See
938 the libwebsockets-test-server-pthread for how to do.
940 If you leave fd_limit_per_thread at 0, then the process limit of fds is shared
941 between the service threads; if you process was allowed 1024 fds overall then
942 each thread is limited to 1024 / n.
944 You can set fd_limit_per_thread to a nonzero number to control this manually, eg
945 the overall supported fd limit is less than the process allowance.
947 You can control the context basic data allocation for multithreading from Cmake
948 using -DLWS_MAX_SMP=, if not given it's set to 1. The serv_buf allocation
949 for the threads (currently 4096) is made at runtime only for active threads.
951 Because lws will limit the requested number of actual threads supported
952 according to LWS_MAX_SMP, there is an api lws_get_count_threads(context) to
953 discover how many threads were actually allowed when the context was created.
955 See the test-server-pthreads.c sample for how to use.
957 @section smplocking SMP Locking Helpers
959 Lws provide a set of pthread mutex helpers that reduce to no code or
960 variable footprint in the case that LWS_MAX_SMP == 1.
962 Define your user mutex like this
965 lws_pthread_mutex(name);
968 If LWS_MAX_SMP > 1, this produces `pthread_mutex_t name;`. In the case
969 LWS_MAX_SMP == 1, it produces nothing.
971 Likewise these helpers for init, destroy, lock and unlock
975 void lws_pthread_mutex_init(pthread_mutex_t *lock)
976 void lws_pthread_mutex_destroy(pthread_mutex_t *lock)
977 void lws_pthread_mutex_lock(pthread_mutex_t *lock)
978 void lws_pthread_mutex_unlock(pthread_mutex_t *lock)
981 resolve to nothing if LWS_MAX_SMP == 1, otherwise produce the equivalent
984 pthreads is required in lws only if LWS_MAX_SMP > 1.
987 @section libevuv libev / libuv / libevent support
989 You can select either or both
993 -DLWS_WITH_LIBEVENT=1
995 at cmake configure-time. The user application may use one of the
996 context init options flags
998 LWS_SERVER_OPTION_LIBEV
999 LWS_SERVER_OPTION_LIBUV
1000 LWS_SERVER_OPTION_LIBEVENT
1002 to indicate it will use one of the event libraries at runtime.
1004 libev has some problems, its headers conflict with libevent, they both define
1005 critical constants like EV_READ to different values. Attempts
1006 to discuss clearing that up with libevent and libev did not get anywhere useful.
1008 In addition building anything with libev using gcc spews warnings, the
1009 maintainer is aware of this for many years, and blames gcc. We worked
1010 around this by disabling -Werror on the parts of lws that use libev.
1012 For these reasons and the response I got trying to raise these issues with
1013 them, if you have a choice about event loop, I would gently encourage you
1014 to avoid libev. Where lws uses an event loop itself, eg in lwsws, we use
1017 @section extopts Extension option control from user code
1019 User code may set per-connection extension options now, using a new api
1020 `lws_set_extension_option()`.
1022 This should be called from the ESTABLISHED callback like this
1024 lws_set_extension_option(wsi, "permessage-deflate",
1025 "rx_buf_size", "12"); /* 1 << 12 */
1028 If the extension is not active (missing or not negotiated for the
1029 connection, or extensions are disabled on the library) the call is
1030 just returns -1. Otherwise the connection's extension has its
1031 named option changed.
1033 The extension may decide to alter or disallow the change, in the
1034 example above permessage-deflate restricts the size of his rx
1035 output buffer also considering the protocol's rx_buf_size member.
1038 @section httpsclient Client connections as HTTP[S] rather than WS[S]
1040 You may open a generic http client connection using the same
1041 struct lws_client_connect_info used to create client ws[s]
1044 To stay in http[s], set the optional info member "method" to
1045 point to the string "GET" instead of the default NULL.
1047 After the server headers are processed, when payload from the
1048 server is available the callback LWS_CALLBACK_RECEIVE_CLIENT_HTTP
1051 You can choose whether to process the data immediately, or
1052 queue a callback when an outgoing socket is writeable to provide
1053 flow control, and process the data in the writable callback.
1055 Either way you use the api `lws_http_client_read()` to access the
1059 case LWS_CALLBACK_RECEIVE_CLIENT_HTTP:
1061 char buffer[1024 + LWS_PRE];
1062 char *px = buffer + LWS_PRE;
1063 int lenx = sizeof(buffer) - LWS_PRE;
1065 lwsl_notice("LWS_CALLBACK_RECEIVE_CLIENT_HTTP\n");
1068 * Often you need to flow control this by something
1069 * else being writable. In that case call the api
1070 * to get a callback when writable here, and do the
1071 * pending client read in the writeable callback of
1074 if (lws_http_client_read(wsi, &px, &lenx) < 0)
1082 Notice that if you will use SSL client connections on a vhost, you must
1083 prepare the client SSL context for the vhost after creating the vhost, since
1084 this is not normally done if the vhost was set up to listen / serve. Call
1085 the api lws_init_vhost_client_ssl() to also allow client SSL on the vhost.
1087 @section clipipe Pipelining Client Requests to same host
1089 If you are opening more client requests to the same host and port, you
1090 can give the flag LCCSCF_PIPELINE on `info.ssl_connection` to indicate
1091 you wish to pipeline them.
1093 Without the flag, the client connections will occur concurrently using a
1094 socket and tls wrapper if requested for each connection individually.
1095 That is fast, but resource-intensive.
1097 With the flag, lws will queue subsequent client connections on the first
1098 connection to the same host and port. When it has confirmed from the
1099 first connection that pipelining / keep-alive is supported by the server,
1100 it lets the queued client pipeline connections send their headers ahead
1101 of time to create a pipeline of requests on the server side.
1103 In this way only one tcp connection and tls wrapper is required to transfer
1104 all the transactions sequentially. It takes a little longer but it
1105 can make a significant difference to resources on both sides.
1107 If lws learns from the first response header that keepalive is not possible,
1108 then it marks itself with that information and detaches any queued clients
1109 to make their own individual connections as a fallback.
1111 Lws can also intelligently combine multiple ongoing client connections to
1112 the same host and port into a single http/2 connection with multiple
1113 streams if the server supports it.
1115 Unlike http/1 pipelining, with http/2 the client connections all occur
1116 simultaneously using h2 stream multiplexing inside the one tcp + tls
1119 You can turn off the h2 client support either by not building lws with
1120 `-DLWS_WITH_HTTP2=1` or giving the `LCCSCF_NOT_H2` flag in the client
1121 connection info struct `ssl_connection` member.
1123 @section vhosts Using lws vhosts
1125 If you set LWS_SERVER_OPTION_EXPLICIT_VHOSTS options flag when you create
1126 your context, it won't create a default vhost using the info struct
1127 members for compatibility. Instead you can call lws_create_vhost()
1128 afterwards to attach one or more vhosts manually.
1131 LWS_VISIBLE struct lws_vhost *
1132 lws_create_vhost(struct lws_context *context,
1133 struct lws_context_creation_info *info);
1136 lws_create_vhost() uses the same info struct as lws_create_context(),
1137 it ignores members related to context and uses the ones meaningful
1138 for vhost (marked with VH in libwebsockets.h).
1141 struct lws_context_creation_info {
1143 const char *iface; /* VH */
1144 const struct lws_protocols *protocols; /* VH */
1145 const struct lws_extension *extensions; /* VH */
1149 When you attach the vhost, if the vhost's port already has a listen socket
1150 then both vhosts share it and use SNI (is SSL in use) or the Host: header
1151 from the client to select the right one. Or if no other vhost already
1152 listening the a new listen socket is created.
1154 There are some new members but mainly it's stuff you used to set at
1155 context creation time.
1158 @section sni How lws matches hostname or SNI to a vhost
1160 LWS first strips any trailing :port number.
1162 Then it tries to find an exact name match for a vhost listening on the correct
1163 port, ie, if SNI or the Host: header provided abc.com:1234, it will match on a
1164 vhost named abc.com that is listening on port 1234.
1166 If there is no exact match, lws will consider wildcard matches, for example
1167 if cats.abc.com:1234 is provided by the client by SNI or Host: header, it will
1168 accept a vhost "abc.com" listening on port 1234. If there was a better, exact,
1169 match, it will have been chosen in preference to this.
1171 Connections with SSL will still have the client go on to check the
1172 certificate allows wildcards and error out if not.
1176 @section mounts Using lws mounts on a vhost
1178 The last argument to lws_create_vhost() lets you associate a linked
1179 list of lws_http_mount structures with that vhost's URL 'namespace', in
1180 a similar way that unix lets you mount filesystems into areas of your /
1181 filesystem how you like and deal with the contents transparently.
1184 struct lws_http_mount {
1185 struct lws_http_mount *mount_next;
1186 const char *mountpoint; /* mountpoint in http pathspace, eg, "/" */
1187 const char *origin; /* path to be mounted, eg, "/var/www/warmcat.com" */
1188 const char *def; /* default target, eg, "index.html" */
1190 struct lws_protocol_vhost_options *cgienv;
1195 unsigned int cache_reusable:1;
1196 unsigned int cache_revalidate:1;
1197 unsigned int cache_intermediaries:1;
1199 unsigned char origin_protocol;
1200 unsigned char mountpoint_len;
1204 The last mount structure should have a NULL mount_next, otherwise it should
1205 point to the 'next' mount structure in your list.
1207 Both the mount structures and the strings must persist until the context is
1208 destroyed, since they are not copied but used in place.
1210 `.origin_protocol` should be one of
1219 LWSMPRO_REDIR_HTTPS,
1224 - LWSMPRO_FILE is used for mapping url namespace to a filesystem directory and
1225 serve it automatically.
1227 - LWSMPRO_CGI associates the url namespace with the given CGI executable, which
1228 runs when the URL is accessed and the output provided to the client.
1230 - LWSMPRO_REDIR_HTTP and LWSMPRO_REDIR_HTTPS auto-redirect clients to the given
1233 - LWSMPRO_CALLBACK causes the http connection to attach to the callback
1234 associated with the named protocol (which may be a plugin).
1237 @section mountcallback Operation of LWSMPRO_CALLBACK mounts
1239 The feature provided by CALLBACK type mounts is binding a part of the URL
1240 namespace to a named protocol callback handler.
1242 This allows protocol plugins to handle areas of the URL namespace. For example
1243 in test-server-v2.0.c, the URL area "/formtest" is associated with the plugin
1244 providing "protocol-post-demo" like this
1247 static const struct lws_http_mount mount_post = {
1248 NULL, /* linked-list pointer to next*/
1249 "/formtest", /* mountpoint in URL namespace on this vhost */
1250 "protocol-post-demo", /* handler */
1251 NULL, /* default filename if none given */
1258 LWSMPRO_CALLBACK, /* origin points to a callback */
1259 9, /* strlen("/formtest"), ie length of the mountpoint */
1263 Client access to /formtest[anything] will be passed to the callback registered
1264 with the named protocol, which in this case is provided by a protocol plugin.
1266 Access by all methods, eg, GET and POST are handled by the callback.
1268 protocol-post-demo deals with accepting and responding to the html form that
1269 is in the test server HTML.
1271 When a connection accesses a URL related to a CALLBACK type mount, the
1272 connection protocol is changed until the next access on the connection to a
1273 URL outside the same CALLBACK mount area. User space on the connection is
1274 arranged to be the size of the new protocol user space allocation as given in
1275 the protocol struct.
1277 This allocation is only deleted / replaced when the connection accesses a
1278 URL region with a different protocol (or the default protocols[0] if no
1279 CALLBACK area matches it).
1281 This "binding connection to a protocol" lifecycle in managed by
1282 `LWS_CALLBACK_HTTP_BIND_PROTOCOL` and `LWS_CALLBACK_HTTP_DROP_PROTOCOL`.
1283 Because of HTTP/1.1 connection pipelining, one connection may perform
1284 many transactions, each of which may map to different URLs and need
1285 binding to different protocols. So these messages are used to
1286 create the binding of the wsi to your protocol including any
1287 allocations, and to destroy the binding, at which point you should
1288 destroy any related allocations.
1290 @section BINDTODEV SO_BIND_TO_DEVICE
1292 The .bind_iface flag in the context / vhost creation struct lets you
1293 declare that you want all traffic for listen and transport on that
1294 vhost to be strictly bound to the network interface named in .iface.
1296 This Linux-only feature requires SO_BIND_TO_DEVICE, which in turn
1297 requires CAP_NET_RAW capability... root has this capability.
1299 However this feature needs to apply the binding also to accepted
1300 sockets during normal operation, which implies the server must run
1301 the whole time as root.
1303 You can avoid this by using the Linux capabilities feature to have
1304 the unprivileged user inherit just the CAP_NET_RAW capability.
1306 You can confirm this with the test server
1310 $ sudo /usr/local/bin/libwebsockets-test-server -u agreen -i eno1 -k
1313 The part that ensures the capability is inherited by the unprivileged
1317 #if defined(LWS_HAVE_SYS_CAPABILITY_H) && defined(LWS_HAVE_LIBCAP)
1318 info.caps[0] = CAP_NET_RAW;
1319 info.count_caps = 1;
1324 @section dim Dimming webpage when connection lost
1326 The lws test plugins' html provides useful feedback on the webpage about if it
1327 is still connected to the server, by greying out the page if not. You can
1328 also add this to your own html easily
1330 - include lws-common.js from your HEAD section
1332 \<script src="/lws-common.js">\</script>
1334 - dim the page during initialization, in a script section on your page
1336 lws_gray_out(true,{'zindex':'499'});
1338 - in your ws onOpen(), remove the dimming
1340 lws_gray_out(false);
1342 - in your ws onClose(), reapply the dimming
1344 lws_gray_out(true,{'zindex':'499'});
1346 @section errstyle Styling http error pages
1348 In the code, http errors should be handled by `lws_return_http_status()`.
1350 There are basically two ways... the vhost can be told to redirect to an "error
1351 page" URL in response to specifically a 404... this is controlled by the
1352 context / vhost info struct (`struct lws_context_creation_info`) member
1353 `.error_document_404`... if non-null the client is redirected to this string.
1355 If it wasn't redirected, then the response code html is synthesized containing
1356 the user-selected text message and attempts to pull in `/error.css` for styling.
1358 If this file exists, it can be used to style the error page. See
1359 https://libwebsockets.org/git/badrepo for an example of what can be done (
1360 and https://libwebsockets.org/error.css for the corresponding css).