1 Notes about coding with lws
2 ===========================
4 @section dae Daemonization
6 There's a helper api `lws_daemonize` built by default that does everything you
7 need to daemonize well, including creating a lock file. If you're making
8 what's basically a daemon, just call this early in your init to fork to a
9 headless background process and exit the starting process.
11 Notice stdout, stderr, stdin are all redirected to /dev/null to enforce your
12 daemon is headless, so you'll need to sort out alternative logging, by, eg,
16 @section conns Maximum number of connections
18 The maximum number of connections the library can deal with is decided when
19 it starts by querying the OS to find out how many file descriptors it is
20 allowed to open (1024 on Fedora for example). It then allocates arrays that
21 allow up to that many connections, minus whatever other file descriptors are
22 in use by the user code.
24 If you want to restrict that allocation, or increase it, you can use ulimit or
25 similar to change the available number of file descriptors, and when restarted
26 **libwebsockets** will adapt accordingly.
29 @section evtloop Libwebsockets is singlethreaded
31 Libwebsockets works in a serialized event loop, in a single thread.
33 Directly performing websocket actions from other threads is not allowed.
34 Aside from the internal data being inconsistent in `forked()` processes,
35 the scope of a `wsi` (`struct websocket`) can end at any time during service
36 with the socket closing and the `wsi` freed.
38 Websocket write activities should only take place in the
39 `LWS_CALLBACK_SERVER_WRITEABLE` callback as described below.
41 [This network-programming necessity to link the issue of new data to
42 the peer taking the previous data is not obvious to all users so let's
43 repeat that in other words:
45 ***ONLY DO LWS_WRITE FROM THE WRITEABLE CALLBACK***
47 There is another network-programming truism that surprises some people which
48 is if the sink for the data cannot accept more:
50 ***YOU MUST PERFORM RX FLOW CONTROL***
52 See the mirror protocol implementations for example code.
54 Only live connections appear in the user callbacks, so this removes any
55 possibility of trying to used closed and freed wsis.
57 If you need to service other socket or file descriptors as well as the
58 websocket ones, you can combine them together with the websocket ones
59 in one poll loop, see "External Polling Loop support" below, and
60 still do it all in one thread / process context.
62 If you insist on trying to use it from multiple threads, take special care if
63 you might simultaneously create more than one context from different threads.
65 SSL_library_init() is called from the context create api and it also is not
66 reentrant. So at least create the contexts sequentially.
69 @section writeable Only send data when socket writeable
71 You should only send data on a websocket connection from the user callback
72 `LWS_CALLBACK_SERVER_WRITEABLE` (or `LWS_CALLBACK_CLIENT_WRITEABLE` for
75 If you want to send something, do not just send it but request a callback
76 when the socket is writeable using
78 - `lws_callback_on_writable(context, wsi)` for a specific `wsi`, or
80 - `lws_callback_on_writable_all_protocol(protocol)` for all connections
81 using that protocol to get a callback when next writeable.
83 Usually you will get called back immediately next time around the service
84 loop, but if your peer is slow or temporarily inactive the callback will be
85 delayed accordingly. Generating what to write and sending it should be done
86 in the ...WRITEABLE callback.
88 See the test server code for an example of how to do this.
91 @section otherwr Do not rely on only your own WRITEABLE requests appearing
93 Libwebsockets may generate additional `LWS_CALLBACK_CLIENT_WRITEABLE` events
94 if it met network conditions where it had to buffer your send data internally.
96 So your code for `LWS_CALLBACK_CLIENT_WRITEABLE` needs to own the decision
97 about what to send, it can't assume that just because the writeable callback
98 came it really is time to send something.
100 It's quite possible you get an 'extra' writeable callback at any time and
101 just need to `return 0` and wait for the expected callback later.
104 @section closing Closing connections from the user side
106 When you want to close a connection, you do it by returning `-1` from a
107 callback for that connection.
109 You can provoke a callback by calling `lws_callback_on_writable` on
110 the wsi, then notice in the callback you want to close it and just return -1.
111 But usually, the decision to close is made in a callback already and returning
114 If the socket knows the connection is dead, because the peer closed or there
115 was an affirmitive network error like a FIN coming, then **libwebsockets** will
116 take care of closing the connection automatically.
118 If you have a silently dead connection, it's possible to enter a state where
119 the send pipe on the connection is choked but no ack will ever come, so the
120 dead connection will never become writeable. To cover that, you can use TCP
121 keepalives (see later in this document) or pings.
124 @section frags Fragmented messages
126 To support fragmented messages you need to check for the final
127 frame of a message with `lws_is_final_fragment`. This
128 check can be combined with `libwebsockets_remaining_packet_payload`
129 to gather the whole contents of a message, eg:
132 case LWS_CALLBACK_RECEIVE:
134 Client * const client = (Client *)user;
135 const size_t remaining = lws_remaining_packet_payload(wsi);
137 if (!remaining && lws_is_final_fragment(wsi)) {
138 if (client->HasFragments()) {
139 client->AppendMessageFragment(in, len, 0);
140 in = (void *)client->GetMessage();
141 len = client->GetMessageLength();
144 client->ProcessMessage((char *)in, len, wsi);
145 client->ResetMessage();
147 client->AppendMessageFragment(in, len, remaining);
152 The test app libwebsockets-test-fraggle sources also show how to
153 deal with fragmented messages.
156 @section debuglog Debug Logging
158 Also using `lws_set_log_level` api you may provide a custom callback to actually
159 emit the log string. By default, this points to an internal emit function
160 that sends to stderr. Setting it to `NULL` leaves it as it is instead.
162 A helper function `lwsl_emit_syslog()` is exported from the library to simplify
163 logging to syslog. You still need to use `setlogmask`, `openlog` and `closelog`
166 The logging apis are made available for user code.
174 The difference between notice and info is that notice will be logged by default
175 whereas info is ignored by default.
177 If you are not building with _DEBUG defined, ie, without this
180 $ cmake .. -DCMAKE_BUILD_TYPE=DEBUG
183 then log levels below notice do not actually get compiled in.
187 @section extpoll External Polling Loop support
189 **libwebsockets** maintains an internal `poll()` array for all of its
190 sockets, but you can instead integrate the sockets into an
191 external polling array. That's needed if **libwebsockets** will
192 cooperate with an existing poll array maintained by another
195 Four callbacks `LWS_CALLBACK_ADD_POLL_FD`, `LWS_CALLBACK_DEL_POLL_FD`,
196 `LWS_CALLBACK_SET_MODE_POLL_FD` and `LWS_CALLBACK_CLEAR_MODE_POLL_FD`
197 appear in the callback for protocol 0 and allow interface code to
198 manage socket descriptors in other poll loops.
200 You can pass all pollfds that need service to `lws_service_fd()`, even
201 if the socket or file does not belong to **libwebsockets** it is safe.
203 If **libwebsocket** handled it, it zeros the pollfd `revents` field before returning.
204 So you can let **libwebsockets** try and if `pollfd->revents` is nonzero on return,
205 you know it needs handling by your code.
207 Also note that when integrating a foreign event loop like libev or libuv where
208 it doesn't natively use poll() semantics, and you must return a fake pollfd
209 reflecting the real event:
211 - be sure you set .events to .revents value as well in the synthesized pollfd
213 - check the built-in support for the event loop if possible (eg, ./lib/libuv.c)
214 to see how it interfaces to lws
216 - use LWS_POLLHUP / LWS_POLLIN / LWS_POLLOUT from libwebsockets.h to avoid
217 losing windows compatibility
220 @section cpp Using with in c++ apps
222 The library is ready for use by C++ apps. You can get started quickly by
223 copying the test server
226 $ cp test-server/test-server.c test.cpp
229 and building it in C++ like this
232 $ g++ -DINSTALL_DATADIR=\"/usr/share\" -ocpptest test.cpp -lwebsockets
235 `INSTALL_DATADIR` is only needed because the test server uses it as shipped, if
236 you remove the references to it in your app you don't need to define it on
240 @section headerinfo Availability of header information
242 HTTP Header information is managed by a pool of "ah" structs. These are a
243 limited resource so there is pressure to free the headers and return the ah to
246 For that reason header information on HTTP connections that get upgraded to
247 websockets is lost after the ESTABLISHED callback. Anything important that
248 isn't processed by user code before then should be copied out for later.
250 For HTTP connections that don't upgrade, header info remains available the
254 @section ka TCP Keepalive
256 It is possible for a connection which is not being used to send to die
257 silently somewhere between the peer and the side not sending. In this case
258 by default TCP will just not report anything and you will never get any more
259 incoming data or sign the link is dead until you try to send.
261 To deal with getting a notification of that situation, you can choose to
262 enable TCP keepalives on all **libwebsockets** sockets, when you create the
265 To enable keepalive, set the ka_time member of the context creation parameter
266 struct to a nonzero value (in seconds) at context creation time. You should
267 also fill ka_probes and ka_interval in that case.
269 With keepalive enabled, the TCP layer will send control packets that should
270 stimulate a response from the peer without affecting link traffic. If the
271 response is not coming, the socket will announce an error at `poll()` forcing
274 Note that BSDs don't support keepalive time / probes / interval per-socket
275 like Linux does. On those systems you can enable keepalive by a nonzero
276 value in `ka_time`, but the systemwide kernel settings for the time / probes/
277 interval are used, regardless of what nonzero value is in `ka_time`.
280 @section sslopt Optimizing SSL connections
282 There's a member `ssl_cipher_list` in the `lws_context_creation_info` struct
283 which allows the user code to restrict the possible cipher selection at
284 context-creation time.
286 You might want to look into that to stop the ssl peers selecting a cipher which
287 is too computationally expensive. To use it, point it to a string like
289 `"RC4-MD5:RC4-SHA:AES128-SHA:AES256-SHA:HIGH:!DSS:!aNULL"`
291 if left `NULL`, then the "DEFAULT" set of ciphers are all possible to select.
293 You can also set it to `"ALL"` to allow everything (including insecure ciphers).
296 @section clientasync Async nature of client connections
298 When you call `lws_client_connect_info(..)` and get a `wsi` back, it does not
299 mean your connection is active. It just means it started trying to connect.
301 Your client connection is actually active only when you receive
302 `LWS_CALLBACK_CLIENT_ESTABLISHED` for it.
304 There's a 5 second timeout for the connection, and it may give up or die for
305 other reasons, if any of that happens you'll get a
306 `LWS_CALLBACK_CLIENT_CONNECTION_ERROR` callback on protocol 0 instead for the
309 After attempting the connection and getting back a non-`NULL` `wsi` you should
310 loop calling `lws_service()` until one of the above callbacks occurs.
312 As usual, see [test-client.c](test-server/test-client.c) for example code.
314 Notice that the client connection api tries to progress the connection
315 somewhat before returning. That means it's possible to get callbacks like
316 CONNECTION_ERROR on the new connection before your user code had a chance to
317 get the wsi returned to identify it (in fact if the connection did fail early,
318 NULL will be returned instead of the wsi anyway).
320 To avoid that problem, you can fill in `pwsi` in the client connection info
321 struct to point to a struct lws that get filled in early by the client
322 connection api with the related wsi. You can then check for that in the
323 callback to confirm the identity of the failing client connection.
326 @section fileapi Lws platform-independent file access apis
328 lws now exposes his internal platform file abstraction in a way that can be
329 both used by user code to make it platform-agnostic, and be overridden or
330 subclassed by user code. This allows things like handling the URI "directory
331 space" as a virtual filesystem that may or may not be backed by a regular
332 filesystem. One example use is serving files from inside large compressed
333 archive storage without having to unpack anything except the file being
336 The test server shows how to use it, basically the platform-specific part of
337 lws prepares a file operations structure that lives in the lws context.
339 The user code can get a pointer to the file operations struct
342 LWS_VISIBLE LWS_EXTERN struct lws_plat_file_ops *
343 `lws_get_fops`(struct lws_context *context);
346 and then can use helpers to also leverage these platform-independent
350 static inline lws_fop_fd_t
351 `lws_plat_file_open`(struct lws_plat_file_ops *fops, const char *filename,
352 lws_filepos_t *filelen, lws_fop_flags_t *flags)
354 `lws_plat_file_close`(lws_fop_fd_t fop_fd)
356 static inline unsigned long
357 `lws_plat_file_seek_cur`(lws_fop_fd_t fop_fd, lws_fileofs_t offset)
360 `lws_plat_file_read`(lws_fop_fd_t fop_fd, lws_filepos_t *amount,
361 uint8_t *buf, lws_filepos_t len)
364 `lws_plat_file_write`(lws_fop_fd_t fop_fd, lws_filepos_t *amount,
365 uint8_t *buf, lws_filepos_t len )
368 The user code can also override or subclass the file operations, to either
369 wrap or replace them. An example is shown in test server.
371 ### Changes from v2.1 and before fops
373 There are three changes:
375 1) Pre-2.2 fops directly used platform file descriptors. Current fops returns and accepts a wrapper type lws_fop_fd_t which is a pointer to a malloc'd struct containing information specific to the filesystem implementation.
377 2) Pre-2.2 fops bound the fops to a wsi. This is completely removed, you just give a pointer to the fops struct that applies to this file when you open it. Afterwards, the operations in the fops just need the lws_fop_fd_t returned from the open.
379 3) Everything is wrapped in typedefs. See lws-plat-unix.c for examples of how to implement.
381 @section ecdh ECDH Support
383 ECDH Certs are now supported. Enable the CMake option
385 cmake .. -DLWS_SSL_SERVER_WITH_ECDH_CERT=1
387 **and** the info->options flag
389 LWS_SERVER_OPTION_SSL_ECDH
391 to build in support and select it at runtime.
393 @section smp SMP / Multithreaded service
395 SMP support is integrated into LWS without any internal threading. It's
396 very simple to use, libwebsockets-test-server-pthread shows how to do it,
397 use -j <n> argument there to control the number of service threads up to 32.
399 Two new members are added to the info struct
401 unsigned int count_threads;
402 unsigned int fd_limit_per_thread;
404 leave them at the default 0 to get the normal singlethreaded service loop.
406 Set count_threads to n to tell lws you will have n simultaneous service threads
407 operating on the context.
409 There is still a single listen socket on one port, no matter how many
412 When a connection is made, it is accepted by the service thread with the least
413 connections active to perform load balancing.
415 The user code is responsible for spawning n threads running the service loop
416 associated to a specific tsi (Thread Service Index, 0 .. n - 1). See
417 the libwebsockets-test-server-pthread for how to do.
419 If you leave fd_limit_per_thread at 0, then the process limit of fds is shared
420 between the service threads; if you process was allowed 1024 fds overall then
421 each thread is limited to 1024 / n.
423 You can set fd_limit_per_thread to a nonzero number to control this manually, eg
424 the overall supported fd limit is less than the process allowance.
426 You can control the context basic data allocation for multithreading from Cmake
427 using -DLWS_MAX_SMP=, if not given it's set to 32. The serv_buf allocation
428 for the threads (currently 4096) is made at runtime only for active threads.
430 Because lws will limit the requested number of actual threads supported
431 according to LWS_MAX_SMP, there is an api lws_get_count_threads(context) to
432 discover how many threads were actually allowed when the context was created.
434 It's required to implement locking in the user code in the same way that
435 libwebsockets-test-server-pthread does it, for the FD locking callbacks.
437 There is no knowledge or dependency in lws itself about pthreads. How the
438 locking is implemented is entirely up to the user code.
441 @section libevuv Libev / Libuv support
443 You can select either or both
448 at cmake configure-time. The user application may use one of the
449 context init options flags
451 LWS_SERVER_OPTION_LIBEV
452 LWS_SERVER_OPTION_LIBUV
454 to indicate it will use either of the event libraries.
457 @section extopts Extension option control from user code
459 User code may set per-connection extension options now, using a new api
460 `lws_set_extension_option()`.
462 This should be called from the ESTABLISHED callback like this
464 lws_set_extension_option(wsi, "permessage-deflate",
465 "rx_buf_size", "12"); /* 1 << 12 */
468 If the extension is not active (missing or not negotiated for the
469 connection, or extensions are disabled on the library) the call is
470 just returns -1. Otherwise the connection's extension has its
471 named option changed.
473 The extension may decide to alter or disallow the change, in the
474 example above permessage-deflate restricts the size of his rx
475 output buffer also considering the protocol's rx_buf_size member.
478 @section httpsclient Client connections as HTTP[S] rather than WS[S]
480 You may open a generic http client connection using the same
481 struct lws_client_connect_info used to create client ws[s]
484 To stay in http[s], set the optional info member "method" to
485 point to the string "GET" instead of the default NULL.
487 After the server headers are processed, when payload from the
488 server is available the callback LWS_CALLBACK_RECEIVE_CLIENT_HTTP
491 You can choose whether to process the data immediately, or
492 queue a callback when an outgoing socket is writeable to provide
493 flow control, and process the data in the writable callback.
495 Either way you use the api `lws_http_client_read()` to access the
499 case LWS_CALLBACK_RECEIVE_CLIENT_HTTP:
501 char buffer[1024 + LWS_PRE];
502 char *px = buffer + LWS_PRE;
503 int lenx = sizeof(buffer) - LWS_PRE;
505 lwsl_notice("LWS_CALLBACK_RECEIVE_CLIENT_HTTP\n");
508 * Often you need to flow control this by something
509 * else being writable. In that case call the api
510 * to get a callback when writable here, and do the
511 * pending client read in the writeable callback of
514 if (lws_http_client_read(wsi, &px, &lenx) < 0)
522 Notice that if you will use SSL client connections on a vhost, you must
523 prepare the client SSL context for the vhost after creating the vhost, since
524 this is not normally done if the vhost was set up to listen / serve. Call
525 the api lws_init_vhost_client_ssl() to also allow client SSL on the vhost.
529 @section vhosts Using lws vhosts
531 If you set LWS_SERVER_OPTION_EXPLICIT_VHOSTS options flag when you create
532 your context, it won't create a default vhost using the info struct
533 members for compatibility. Instead you can call lws_create_vhost()
534 afterwards to attach one or more vhosts manually.
537 LWS_VISIBLE struct lws_vhost *
538 lws_create_vhost(struct lws_context *context,
539 struct lws_context_creation_info *info);
542 lws_create_vhost() uses the same info struct as lws_create_context(),
543 it ignores members related to context and uses the ones meaningful
544 for vhost (marked with VH in libwebsockets.h).
547 struct lws_context_creation_info {
549 const char *iface; /* VH */
550 const struct lws_protocols *protocols; /* VH */
551 const struct lws_extension *extensions; /* VH */
555 When you attach the vhost, if the vhost's port already has a listen socket
556 then both vhosts share it and use SNI (is SSL in use) or the Host: header
557 from the client to select the right one. Or if no other vhost already
558 listening the a new listen socket is created.
560 There are some new members but mainly it's stuff you used to set at
561 context creation time.
564 @section sni How lws matches hostname or SNI to a vhost
566 LWS first strips any trailing :port number.
568 Then it tries to find an exact name match for a vhost listening on the correct
569 port, ie, if SNI or the Host: header provided abc.com:1234, it will match on a
570 vhost named abc.com that is listening on port 1234.
572 If there is no exact match, lws will consider wildcard matches, for example
573 if cats.abc.com:1234 is provided by the client by SNI or Host: header, it will
574 accept a vhost "abc.com" listening on port 1234. If there was a better, exact,
575 match, it will have been chosen in preference to this.
577 Connections with SSL will still have the client go on to check the
578 certificate allows wildcards and error out if not.
582 @section mounts Using lws mounts on a vhost
584 The last argument to lws_create_vhost() lets you associate a linked
585 list of lws_http_mount structures with that vhost's URL 'namespace', in
586 a similar way that unix lets you mount filesystems into areas of your /
587 filesystem how you like and deal with the contents transparently.
590 struct lws_http_mount {
591 struct lws_http_mount *mount_next;
592 const char *mountpoint; /* mountpoint in http pathspace, eg, "/" */
593 const char *origin; /* path to be mounted, eg, "/var/www/warmcat.com" */
594 const char *def; /* default target, eg, "index.html" */
596 struct lws_protocol_vhost_options *cgienv;
601 unsigned int cache_reusable:1;
602 unsigned int cache_revalidate:1;
603 unsigned int cache_intermediaries:1;
605 unsigned char origin_protocol;
606 unsigned char mountpoint_len;
610 The last mount structure should have a NULL mount_next, otherwise it should
611 point to the 'next' mount structure in your list.
613 Both the mount structures and the strings must persist until the context is
614 destroyed, since they are not copied but used in place.
616 `.origin_protocol` should be one of
630 - LWSMPRO_FILE is used for mapping url namespace to a filesystem directory and
631 serve it automatically.
633 - LWSMPRO_CGI associates the url namespace with the given CGI executable, which
634 runs when the URL is accessed and the output provided to the client.
636 - LWSMPRO_REDIR_HTTP and LWSMPRO_REDIR_HTTPS auto-redirect clients to the given
639 - LWSMPRO_CALLBACK causes the http connection to attach to the callback
640 associated with the named protocol (which may be a plugin).
643 @section mountcallback Operation of LWSMPRO_CALLBACK mounts
645 The feature provided by CALLBACK type mounts is binding a part of the URL
646 namespace to a named protocol callback handler.
648 This allows protocol plugins to handle areas of the URL namespace. For example
649 in test-server-v2.0.c, the URL area "/formtest" is associated with the plugin
650 providing "protocol-post-demo" like this
653 static const struct lws_http_mount mount_post = {
654 NULL, /* linked-list pointer to next*/
655 "/formtest", /* mountpoint in URL namespace on this vhost */
656 "protocol-post-demo", /* handler */
657 NULL, /* default filename if none given */
664 LWSMPRO_CALLBACK, /* origin points to a callback */
665 9, /* strlen("/formtest"), ie length of the mountpoint */
669 Client access to /formtest[anything] will be passed to the callback registered
670 with the named protocol, which in this case is provided by a protocol plugin.
672 Access by all methods, eg, GET and POST are handled by the callback.
674 protocol-post-demo deals with accepting and responding to the html form that
675 is in the test server HTML.
677 When a connection accesses a URL related to a CALLBACK type mount, the
678 connection protocol is changed until the next access on the connection to a
679 URL outside the same CALLBACK mount area. User space on the connection is
680 arranged to be the size of the new protocol user space allocation as given in
683 This allocation is only deleted / replaced when the connection accesses a
684 URL region with a different protocol (or the default protocols[0] if no
685 CALLBACK area matches it).
687 @section dim Dimming webpage when connection lost
689 The lws test plugins' html provides useful feedback on the webpage about if it
690 is still connected to the server, by greying out the page if not. You can
691 also add this to your own html easily
693 - include lws-common.js from your HEAD section
695 <script src="/lws-common.js"></script>
697 - dim the page during initialization, in a script section on your page
699 lws_gray_out(true,{'zindex':'499'});
701 - in your ws onOpen(), remove the dimming
705 - in your ws onClose(), reapply the dimming
707 lws_gray_out(true,{'zindex':'499'});