add new context arg to libwebsockets_serve_http_file

[profile/ivi/libwebsockets.git] / README-test-server

Using test-server as a quickstart
---------------------------------

You need to regenerate the autotools and libtoolize stuff for your system

$ ./autogen.sh

Then for a Fedora x86_86 box, the following config line was
needed:

 ./configure --prefix=/usr --libdir=/usr/lib64 --enable-openssl

For Apple systems, Christopher Baker reported that this is needed
(and I was told separately enabling openssl makes trouble somehow)

./configure CC="gcc -arch i386 -arch x86_64" CXX="g++ -arch i386 -arch
x86_64" CPP="gcc -E" CXXCPP="g++ -E" --enable-nofork

For mingw build, I did the following to get working build, ping test is
disabled when building this way

1) install mingw64_w32 compiler packages from Fedora
2) additionally install mingw64-zlib package
3) ./configure --prefix=/usr --enable-mingw --host=x86_64-w64-mingw32
4) make

For uClibc, you will likely need --enable-builtin-getifaddrs

otherwise if /usr/local/... and /usr/local/lib are OK then...

$ ./configure
$ make clean
$ make
$ sudo make install
$ libwebsockets-test-server

should be enough to get a test server listening on port 7861.


Configure script options
------------------------

There are several other possible configure options

--enable-nofork         disables the fork into the background API
                        and removes all references to fork() and
                        pr_ctl() from the sources.  Use it if your
                        platform doesn't support forking.

--enable-libcrypto      by default libwebsockets uses its own
                        built-in md5 and sha-1 implementation for
                        simplicity.  However the libcrypto ones
                        may be faster, and in a distro context it
                        may be highly desirable to use a common
                        library implementation for ease of security
                        upgrades.  Give this configure option
                        to disable the built-in ones and force use
                        of the libcrypto (part of openssl) ones.

--with-client-cert-dir=dir   tells the client ssl support where to
                             look for trust certificates to validate
                             the remote certificate against.

--enable-noping         Don't try to build the ping test app
                        It needs some unixy environment that
                        may choke in other build contexts, this
                        lets you cleanly stop it being built
                        
--enable-x-google-mux   Enable experimental x-google-mux support
                        in the build (see notes later in document)

--enable-builtin-getifaddrs  if your libc lacks getifaddrs, you can build an
                        implementation into the library.  By default your libc
                        one is used.

--without-testapps      Just build the library not the test apps


Externally configurable important constants
-------------------------------------------

You can control these from configure by just setting them as commandline
args throgh CFLAGS, eg

./configure CFLAGS="-DLWS_MAX_ZLIB_CONN_BUFFER=8192"

They all have defaults so you only need to take care about them if you want
to tune them to the amount of memory available.


 - FD_HASHTABLE_MODULUS default 32: size of the file descriptor hash map,
affects server performance with large numbers of connections, at the cost of
increased memory consumption

 - MAX_CLIENTS default 100: total number of simultaneous connections
allowed... reserves some memory even when not in use, so reduce for embedded
applications that only expect one or two connections

 - LWS_MAX_HEADER_NAME_LENGTH default 64: max characters in an HTTP header
name that libwebsockets can cope with

 - LWS_MAX_HEADER_LEN default 4096: largest HTTP header value string length
libwebsockets can cope with

 - LWS_INITIAL_HDR_ALLOC default 256: amount of memory to allocate initially,
tradeoff between taking too much and needless realloc

 - LWS_ADDITIONAL_HDR_ALLOC default 64: how much to additionally realloc if
the header value string keeps coming

 - MAX_USER_RX_BUFFER default 4096: max amount of user rx data to buffer at a
time and pass to user callback LWS_CALLBACK_RECEIVE or
LWS_CALLBACK_CLIENT_RECEIVE.  Large frames are passed to the user callback
in chunks of this size.  Tradeoff between per-connection static memory
allocation and if you expect to deal with large frames, how much you can
see at once which can affect efficiency.

 - MAX_BROADCAST_PAYLOAD default 4096: largest amount of user tx data we can
broadcast at a time

 - LWS_MAX_PROTOCOLS default 10: largest amount of different protocols the
server can serve

 - LWS_MAX_EXTENSIONS_ACTIVE default 10: largest amount of extensions we can
choose to have active on one connection

 - SPEC_LATEST_SUPPORTED default 13: only change if you want to remove support
for later protocol versions... unlikely

 - AWAITING_TIMEOUT default 5: after this many seconds without a response, the
server will hang up on the client

 - CIPHERS_LIST_STRING default "DEFAULT": SSL Cipher selection.  It's advisable
to tweak the ciphers allowed to be negotiated on secure connections for
performance reasons, otherwise a slow algorithm may be selected by the two
endpoints and the server could expend most of its time just encrypting and
decrypting data, severely limiting the amount of messages it will be able to
handle per second.  For example::

    "RC4-MD5:RC4-SHA:AES128-SHA:AES256-SHA:HIGH:!DSS:!aNULL"

 - SYSTEM_RANDOM_FILEPATH default "/dev/urandom": if your random device differs
you can set it here

 - LWS_MAX_ZLIB_CONN_BUFFER  maximum size a compression buffer is allowed to
grow to before closing the connection.  Default is 64KBytes.

 - LWS_SOMAXCONN  maximum number of pending connect requests the listening
socket can cope with.  Default is SOMAXCONN.  If you need to use synthetic
tests that just spam hundreds of connect requests at once without dropping
any, you can try setting this to MAX_CLIENTS and mess with your box's tcp
config like this (courtesy Edwin van der Oetelaar)

echo "2048 64512" > /proc/sys/net/ipv4/ip_local_port_range
echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle
echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse
echo "10" > /proc/sys/net/ipv4/tcp_fin_timeout
echo "65536" > /proc/sys/net/core/somaxconn
echo "65536" > /proc/sys/net/ipv4/tcp_max_syn_backlog
echo "262144" > /proc/sys/net/netfilter/nf_conntrack_max


Testing server with a browser
-----------------------------

If you point your browser (eg, Chrome) to

  http://127.0.0.1:7681

It will fetch a script in the form of test.html, and then run the
script in there on the browser to open a websocket connection.
Incrementing numbers should appear in the browser display.

Using SSL on the server side
----------------------------

To test it using SSL/WSS, just run the test server with

$ libwebsockets-test-server --ssl

and use the URL

  https://127.0.0.1:7681

The connection will be entirely encrypted using some generated
certificates that your browser will not accept, since they are
not signed by any real Certificate Authority.  Just accept the
certificates in the browser and the connection will proceed
in first https and then websocket wss, acting exactly the
same.

test-server.c is all that is needed to use libwebsockets for
serving both the script html over http and websockets.


Forkless operation
------------------

If your target device does not offer fork(), you can use
libwebsockets from your own main loop instead.  Use the
configure option --nofork and simply call libwebsocket_service()
from your own main loop as shown in the test app sources.


Fragmented messages
-------------------

To support fragmented messages you need to check for the final
frame of a message with libwebsocket_is_final_fragment. This
check can be combined with libwebsockets_remaining_packet_payload
to gather the whole contents of a message, eg:

    case LWS_CALLBACK_RECEIVE:
    {
        Client * const client = (Client *)user;
        const size_t remaining = libwebsockets_remaining_packet_payload(wsi);

        if (!remaining && libwebsocket_is_final_fragment(wsi)) {
            if (client->HasFragments()) {
                client->AppendMessageFragment(in, len, 0);
                in = (void *)client->GetMessage();
                len = client->GetMessageLength();
            }

            client->ProcessMessage((char *)in, len, wsi);
            client->ResetMessage();
        } else
            client->AppendMessageFragment(in, len, remaining);
    }
    break;

The test app llibwebsockets-test-fraggle sources also show how to
deal with fragmented messages.


Testing websocket client support
--------------------------------

If you run the test server as described above, you can also
connect to it using the test client as well as a browser.

$ libwebsockets-test-client localhost

will by default connect to the test server on localhost:7681
and print the dumb increment number from the server at the
same time as drawing random circles in the mirror protocol;
if you connect to the test server using a browser at the
same time you will be able to see the circles being drawn.


Testing SSL on the client side
------------------------------

To test SSL/WSS client action, just run the client test with

$ libwebsockets-test-client localhost --ssl

By default the client test applet is set to accept selfsigned
certificates used by the test server, this is indicated by the
use_ssl var being set to 2.  Set it to 1 to reject any server
certificate that it doesn't have a trusted CA cert for.


Using the websocket ping utility
--------------------------------

libwebsockets-test-ping connects as a client to a remote
websocket server using 04 protocol and pings it like the
normal unix ping utility.

$ libwebsockets-test-ping localhost                
handshake OK for protocol lws-mirror-protocol
Websocket PING localhost.localdomain (127.0.0.1) 64 bytes of data.
64 bytes from localhost: req=1 time=0.1ms
64 bytes from localhost: req=2 time=0.1ms
64 bytes from localhost: req=3 time=0.1ms
64 bytes from localhost: req=4 time=0.2ms
64 bytes from localhost: req=5 time=0.1ms
64 bytes from localhost: req=6 time=0.2ms
64 bytes from localhost: req=7 time=0.2ms
64 bytes from localhost: req=8 time=0.1ms
^C
--- localhost.localdomain websocket ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 7458ms
rtt min/avg/max = 0.110/0.185/0.218 ms
$

By default it sends 64 byte payload packets using the 04
PING packet opcode type.  You can change the payload size
using the -s= flag, up to a maximum of 125 mandated by the
04 standard.

Using the lws-mirror protocol that is provided by the test
server, libwebsockets-test-ping can also use larger payload
sizes up to 4096 is BINARY packets; lws-mirror will copy
them back to the client and they appear as a PONG.  Use the
-m flag to select this operation.

The default interval between pings is 1s, you can use the -i=
flag to set this, including fractions like -i=0.01 for 10ms
interval.

Before you can even use the PING opcode that is part of the
standard, you must complete a handshake with a specified
protocol.  By default lws-mirror-protocol is used which is
supported by the test server.  But if you are using it on
another server, you can specify the protcol to handshake with
by --protocol=protocolname


Fraggle test app
----------------

By default it runs in server mode

$ libwebsockets-test-fraggle
libwebsockets test fraggle
(C) Copyright 2010-2011 Andy Green <andy@warmcat.com> licensed under LGPL2.1
 Compiled with SSL support, not using it
 Listening on port 7681
server sees client connect
accepted v06 connection
Spamming 360 random fragments
Spamming session over, len = 371913. sum = 0x2D3C0AE
Spamming 895 random fragments
Spamming session over, len = 875970. sum = 0x6A74DA1
...

You need to run a second session in client mode, you have to
give the -c switch and the server address at least:

$ libwebsockets-test-fraggle -c localhost 
libwebsockets test fraggle
(C) Copyright 2010-2011 Andy Green <andy@warmcat.com> licensed under LGPL2.1
 Client mode
Connecting to localhost:7681
denied deflate-stream extension
handshake OK for protocol fraggle-protocol
client connects to server
EOM received 371913 correctly from 360 fragments
EOM received 875970 correctly from 895 fragments
EOM received 247140 correctly from 258 fragments
EOM received 695451 correctly from 692 fragments
...

The fraggle test sends a random number up to 1024 fragmented websocket frames
each of a random size between 1 and 2001 bytes in a single message, then sends
a checksum and starts sending a new randomly sized and fragmented message.

The fraggle test client receives the same message fragments and computes the
same checksum using websocket framing to see when the message has ended.  It
then accepts the server checksum message and compares that to its checksum.


proxy support
-------------

The http_proxy environment variable is respected by the client
connection code for both ws:// and wss://.  It doesn't support
authentication yet.

You use it like this

export http_proxy=myproxy.com:3128
libwebsockets-test-client someserver.com


debug logging
-------------

By default logging of severity "warn" or "err" is enabled to stderr.

Again by default other logging is comiled in but disabled from printing.

If you want to eliminate the debug logging below warn in severity, use the
--disable-debug configure option to have it removed from the code by the
preprocesser.

If you want to see more detailed debug logs, you can control a bitfield to
select which logs types may print using the lws_set_log_level() api, in the
test apps you can use -d <number> to control this.  The types of logging
available are (OR together the numbers to select multiple)

 1   ERR
 2   WARN
 4   INFO
 8   DEBUG
 16  PARSER
 32  HEADER
 64  EXTENSION
 128 CLIENT

Also using lws_set_log_level api you may provide a custom callback to actually
emit the log string.  By default, this points to an internal emit function
that sends to stderr.  Setting it to NULL leaves it as it is instead.


Websocket version supported
---------------------------

The final IETF standard is supported along with various older ones that will
be removed at some point, -76, -04 and -05.


External Polling Loop support
-----------------------------

libwebsockets maintains an internal poll() array for all of its
sockets, but you can instead integrate the sockets into an
external polling array.  That's needed if libwebsockets will
cooperate with an existing poll array maintained by another
server.

Four callbacks LWS_CALLBACK_ADD_POLL_FD, LWS_CALLBACK_DEL_POLL_FD,
LWS_CALLBACK_SET_MODE_POLL_FD and LWS_CALLBACK_CLEAR_MODE_POLL_FD
appear in the callback for protocol 0 and allow interface code to
manage socket descriptors in other poll loops.


x-google-mux support
--------------------

Experimental and super-preliminary x-google-mux support is available if
enabled in ./configure with --enable-x-google-mux.  Note that when changing
configurations, you will need to do a make distclean before, then the new
configure and then make ; make install.  Don't forget the necessary other
flags for your platform as described at the top of the readme.

It has the following notes:

    1) To enable it, reconfigure with --enable-x-google-mux
    
    2) It deviates from the google standard by sending full
       headers in the addchannel subcommand rather than just
       changed ones from original connect
    
    3) Quota is not implemented yet
    
However despite those caveats, in fact it can run the
test client reliably over one socket (both dumb-increment
and lws-mirror-protocol), you can open a browser on the
same test server too and see the circles, etc.

It also works compatibly with deflate-stream automatically.

2012-04-12  Andy Green <andy@warmcat.com>