docs/reference/client-howto.xml

   1 <?xml version="1.0"?>
   2 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
   3                "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">
   4 <refentry id="libsoup-client-howto">
   5 <refmeta>
   6 <refentrytitle>Soup Client Basics</refentrytitle>
   7 <manvolnum>3</manvolnum>
   8 <refmiscinfo>LIBSOUP Library</refmiscinfo>
   9 </refmeta>
  10
  11 <refnamediv>
  12 <refname>Soup Client Basics</refname><refpurpose>Client-side tutorial</refpurpose>
  13 </refnamediv>
  14
  15 <refsect2>
  16 <title>Creating a <type>SoupSession</type></title>
  17
  18 <para>
  19 The first step in using the client API is to create a <link
  20 linkend="SoupSession"><type>SoupSession</type></link>. The session object
  21 encapsulates all of the state that <application>libsoup</application>
  22 is keeping on behalf of your program; cached HTTP connections,
  23 authentication information, etc.
  24 </para>
  25
  26 <para>
  27 There are two subclasses of <link
  28 linkend="SoupSession"><type>SoupSession</type></link> that you can use, with
  29 slightly different behavior:
  30 </para>
  31
  32 <itemizedlist>
  33     <listitem><para>
  34         <link linkend="SoupSessionAsync"><type>SoupSessionAsync</type></link>,
  35         which uses callbacks and the glib main loop to provide
  36         asynchronous I/O.
  37     </para></listitem>
  38
  39     <listitem><para>
  40         <link linkend="SoupSessionSync"><type>SoupSessionSync</type></link>,
  41         which uses blocking I/O rather than callbacks, making it more
  42         suitable for threaded applications.
  43     </para></listitem>
  44 </itemizedlist>
  45
  46 <para>
  47 If you want to do a mix of mainloop-based and blocking I/O, you will
  48 need to create two different session objects.
  49 </para>
  50
  51 <para>
  52 When you create the session (with <link
  53 linkend="soup-session-async-new-with-options"><function>soup_session_async_new_with_options</function></link>
  54 or <link
  55 linkend="soup-session-sync-new-with-options"><function>soup_session_sync_new_with_options</function></link>),
  56 you can specify various additional options:
  57 </para>
  58
  59 <variablelist>
  60     <varlistentry>
  61         <term><link linkend="SOUP-SESSION-MAX-CONNS--CAPS"><literal>SOUP_SESSION_MAX_CONNS</literal></link></term>
  62         <listitem><para>
  63             Allows you to set the maximum total number of connections
  64             the session will have open at one time. (Once it reaches
  65             this limit, it will either close idle connections, or
  66             wait for existing connections to free up before starting
  67             new requests.)
  68         </para></listitem>
  69     </varlistentry>
  70     <varlistentry>
  71         <term><link linkend="SOUP-SESSION-MAX-CONNS-PER-HOST--CAPS"><literal>SOUP_SESSION_MAX_CONNS_PER_HOST</literal></link></term>
  72         <listitem><para>
  73             Allows you to set the maximum total number of connections
  74             the session will have open <emphasis>to a single
  75             host</emphasis> at one time.
  76         </para></listitem>
  77     </varlistentry>
  78     <varlistentry>
  79         <term><link linkend="SOUP-SESSION-USE-NTLM--CAPS"><literal>SOUP_SESSION_USE_NTLM</literal></link></term>
  80         <listitem><para>
  81             If <literal>TRUE</literal>, then Microsoft NTLM
  82             authentication will be used if available (and will be
  83             preferred to HTTP Basic or Digest authentication).
  84             If <literal>FALSE</literal>, NTLM authentication won't be
  85             used, even if it's the only authentication type available.
  86             (NTLM works differently from the standard HTTP
  87             authentication types, so it needs to be handled
  88             specially.)
  89         </para></listitem>
  90     </varlistentry>
  91     <varlistentry>
  92         <term><link linkend="SOUP-SESSION-SSL-CA-FILE--CAPS"><literal>SOUP_SESSION_SSL_CA_FILE</literal></link></term>
  93         <listitem><para>
  94             Points to a file containing certificates for recognized
  95             SSL Certificate Authorities. If this is set, then HTTPS
  96             connections will be checked against these authorities, and
  97             rejected if they can't be verified. (Otherwise all SSL
  98             certificates will be accepted automatically.)
  99         </para></listitem>
 100     </varlistentry>
 101     <varlistentry>
 102         <term><link linkend="SOUP-SESSION-ASYNC-CONTEXT--CAPS"><literal>SOUP_SESSION_ASYNC_CONTEXT</literal></link></term>
 103         <listitem><para>
 104             A <link
 105             linkend="GMainContext"><type>GMainContext</type></link>
 106             which the session will use for asynchronous operations.
 107             This can be set if you want to use a
 108             <type>SoupSessionAsync</type> in a thread other than the
 109             main thread.
 110         </para></listitem>
 111     </varlistentry>
 112     <varlistentry>
 113         <term><link linkend="SOUP-SESSION-ADD-FEATURE--CAPS"><literal>SOUP_SESSION_ADD_FEATURE</literal></link> and <link linkend="SOUP-SESSION-ADD-FEATURE-BY-TYPE--CAPS"><literal>SOUP_SESSION_ADD_FEATURE_BY_TYPE</literal></link></term>
 114         <listitem><para>
 115             These allow you to specify <link
 116             linkend="SoupSessionFeature"><type>SoupSessionFeature</type></link>s
 117             (discussed <link linkend="session-features">below</link>)
 118             to add at construct-time.
 119         </para></listitem>
 120     </varlistentry>
 121 </variablelist>
 122
 123 <para>
 124 If you don't need to specify any options, you can just use <link
 125 linkend="soup-session-async-new"><function>soup_session_async_new</function></link> or
 126 <link linkend="soup-session-sync-new"><function>soup_session_sync_new</function></link>,
 127 which take no arguments.
 128 </para>
 129
 130 </refsect2>
 131
 132 <refsect2 id="session-features">
 133 <title>Session features</title>
 134
 135 <para>
 136 Additional session functionality is provided as <link
 137 linkend="SoupSessionFeature"><type>SoupSessionFeature</type></link>s,
 138 which can be added to a session, via the <link
 139 linkend="SOUP-SESSION-ADD-FEATURE--CAPS"><literal>SOUP_SESSION_ADD_FEATURE</literal></link>
 140 and <link
 141 linkend="SOUP-SESSION-ADD-FEATURE-BY-TYPE--CAPS"><literal>SOUP_SESSION_ADD_FEATURE_BY_TYPE</literal></link>
 142 options at session-construction-time, or afterward via the <link
 143 linkend="soup-session-add-feature"><function>soup_session_add_feature</function></link>
 144 and <link
 145 linkend="soup-session-add-feature-by-type"><function>soup_session_add_feature_by_type</function></link>
 146 functions. Some of the features available in
 147 <application>libsoup</application> are:
 148 </para>
 149
 150 <variablelist>
 151     <varlistentry>
 152         <term><link linkend="SoupLogger"><type>SoupLogger</type></link></term>
 153         <listitem><para>
 154             A debugging aid, which logs all of libsoup's HTTP traffic
 155             to <literal>stdout</literal> (or another place you specify).
 156         </para></listitem>
 157     </varlistentry>
 158     <varlistentry>
 159         <term><link linkend="SoupCookieJar"><type>SoupCookieJar</type></link> and <link linkend="SoupCookieJarText"><type>SoupCookieJarText</type></link></term>
 160         <listitem><para>
 161             Support for HTTP cookies. <type>SoupCookieJar</type>
 162             provides non-persistent cookie storage, while
 163             <type>SoupCookieJarText</type> uses a text file to keep
 164             track of cookies between sessions.
 165         </para></listitem>
 166     </varlistentry>
 167 </variablelist>
 168
 169 <para>
 170 And in <application>libsoup-gnome</application>:
 171 </para>
 172
 173 <variablelist>
 174     <varlistentry>
 175         <term><link linkend="SOUP-TYPE-PROXY-RESOLVER-GNOME--CAPS"><type>SoupProxyResolverGNOME</type></link></term>
 176         <listitem><para>
 177             A feature that automatically determines the correct HTTP
 178             proxy to use for requests.
 179         </para></listitem>
 180     </varlistentry>
 181     <varlistentry>
 182         <term><link linkend="SoupCookieJarSqlite"><type>SoupCookieJarSqlite</type></link></term>
 183         <listitem><para>
 184             Support for HTTP cookies stored in an
 185             <application>SQLite</application> database.
 186         </para></listitem>
 187     </varlistentry>
 188 </variablelist>
 189
 190 <para>
 191 Use the "add_feature_by_type" property/function to add features that
 192 don't require any configuration (such as <link
 193 linkend="SOUP-TYPE-PROXY-RESOLVER-GNOME--CAPS"><type>SoupProxyResolverGNOME</type></link>),
 194 and the "add_feature" property/function to add features that must be
 195 constructed first (such as <link
 196 linkend="SoupLogger"><type>SoupLogger</type></link>). For example, an
 197 application might do something like the following:
 198 </para>
 199
 200 <informalexample><programlisting>
 201         session = soup_session_async_new_with_options (
 202 #ifdef HAVE_LIBSOUP_GNOME
 203                 SOUP_SESSION_ADD_FEATURE_BY_TYPE, SOUP_TYPE_PROXY_RESOLVER_GNOME,
 204 #endif
 205                 NULL);
 206         if (debug_level) {
 207                 SoupLogger *logger;
 208
 209                 logger = soup_logger_new (debug_level, -1);
 210                 soup_session_add_feature (session, SOUP_SESSION_FEATURE (logger));
 211                 g_object_unref (logger);
 212         }
 213 </programlisting></informalexample>
 214
 215 </refsect2>
 216
 217 <refsect2>
 218 <title>Creating and Sending SoupMessages</title>
 219
 220 <para>
 221 Once you have a session, you do HTTP traffic using <link
 222 linkend="SoupMessage"><type>SoupMessage</type></link>. In the simplest
 223 case, you only need to create the message and it's ready to send:
 224 </para>
 225
 226 <informalexample><programlisting>
 227         SoupMessage *msg;
 228
 229         msg = soup_message_new ("GET", "http://example.com/");
 230 </programlisting></informalexample>
 231
 232 <para>
 233 In more complicated cases, you can use various <link
 234 linkend="SoupMessage">SoupMessage</link>, <link
 235 linkend="SoupMessageHeaders">SoupMessageHeaders</link>, and <link
 236 linkend="SoupMessageBody">SoupMessageBody</link> methods to set the
 237 request headers and body of the message:
 238 </para>
 239
 240 <informalexample><programlisting>
 241         SoupMessage *msg;
 242
 243         msg = soup_message_new ("POST", "http://example.com/form.cgi");
 244         soup_message_set_request (msg, "application/x-www-form-urlencoded",
 245                                   SOUP_MEMORY_COPY, formdata, strlen (formdata));
 246         soup_message_headers_append (msg->request_headers, "Referer", referring_url);
 247 </programlisting></informalexample>
 248
 249 <para>
 250 (Although this is a bad example, because
 251 <application>libsoup</application> actually has convenience methods
 252 for dealing with <link linkend="libsoup-24-HTML-Form-Support">HTML
 253 forms</link>, as well as <link
 254 linkend="libsoup-24-XMLRPC-Support">XML-RPC</link>.)
 255 </para>
 256
 257 <para>
 258 You can also use <link
 259 linkend="soup-message-set-flags"><function>soup_message_set_flags</function></link>
 260 to change some default behaviors. For example, by default,
 261 <type>SoupSession</type> automatically handles responses from the
 262 server that redirect to another URL. If you would like to handle these
 263 yourself, you can set the <link linkend="SOUP-MESSAGE-NO-REDIRECT--CAPS"><literal>SOUP_MESSAGE_NO_REDIRECT</literal></link>
 264 flag.
 265 </para>
 266
 267 <refsect3>
 268 <title>Sending a Message Synchronously</title>
 269
 270 <para>
 271 To send a message and wait for the response, use <link
 272 linkend="soup-session-send-message"><function>soup_session_send_message</function></link>:
 273 </para>
 274
 275 <informalexample><programlisting>
 276         guint status;
 277
 278         status = soup_session_send_message (session, msg);
 279 </programlisting></informalexample>
 280
 281 <para>
 282 (If you use <function>soup_session_send_message</function> with a
 283 <link linkend="SoupSessionAsync"><type>SoupSessionAsync</type></link>,
 284 it will run the main loop itself until the message is complete.)
 285 </para>
 286
 287 <para>
 288 The return value from <function>soup_session_send_message</function>
 289 is a <link linkend="soup-status">libsoup status code</link>,
 290 indicating either a transport error that prevented the message from
 291 being sent, or the HTTP status that was returned by the server in
 292 response to the message. (The status is also available as
 293 <literal>msg->status_code</literal>.)
 294 </para>
 295
 296 </refsect3>
 297
 298 <refsect3>
 299 <title>Sending a Message Asynchronously</title>
 300
 301 <para>
 302 To send a message asynchronously, use <link
 303 linkend="soup-session-queue-message"><function>soup_session_queue_message</function></link>:
 304 </para>
 305
 306 <informalexample><programlisting>
 307         ...
 308         soup_session_queue_message (session, msg, my_callback, my_callback_data);
 309         ...
 310 }
 311
 312 static void
 313 my_callback (SoupSession *session, SoupMessage *msg, gpointer user_data)
 314 {
 315         /* Handle the response here */
 316 }
 317 </programlisting></informalexample>
 318
 319 <para>
 320 The message will be added to the session's queue, and eventually (when
 321 control is returned back to the main loop), it will be sent and the
 322 response be will be read. When the message is complete,
 323 <literal>callback</literal> will be invoked, along with the data you
 324 passed to <function>soup_session_queue_message</function>.
 325 </para>
 326
 327 <para>
 328 <link
 329 linkend="soup-session-queue-message"><function>soup_session_queue_message</function></link>
 330 steals a reference to the message object, and unrefs it after the last
 331 callback is invoked on it. So in the usual case, messages sent
 332 asynchronously will be automatically freed for you without you needing
 333 to do anything. (Of course, this wouldn't work when using the synchronous
 334 API, since you will usually need continue working with the message
 335 after calling <link
 336 linkend="soup-session-send-message"><function>soup_session_send_message</function></link>,
 337 so in that case, you must unref it explicitly when you are done with
 338 it.)
 339 </para>
 340
 341 <para>
 342 (If you use <link
 343 linkend="soup-session-queue-message"><function>soup_session_queue_message</function></link>
 344 with a <link
 345 linkend="SoupSessionSync"><type>SoupSessionSync</type></link>, the
 346 message will be sent in another thread, with the callback eventually
 347 being invoked in the session's <link linkend="SOUP-SESSION-ASYNC-CONTEXT--CAPS"><literal>SOUP_SESSION_ASYNC_CONTEXT</literal></link>.)
 348 </para>
 349
 350 </refsect3>
 351
 352 </refsect2>
 353
 354 <refsect2>
 355 <title>Processing the Response</title>
 356
 357 <para>
 358 Once you have received the response from the server, synchronously or
 359 asynchronously, you can look at the response fields in the
 360 <literal>SoupMessage</literal> to decide what to do next. The
 361 <structfield>status_code</structfield> and
 362 <structfield>reason_phrase</structfield> fields contain the numeric
 363 status and textual status response from the server.
 364 <structfield>response_headers</structfield> contains the response
 365 headers, which you can investigate using <link
 366 linkend="soup-message-headers-get"><function>soup_message_headers_get</function></link> and
 367 <link
 368     linkend="soup-message-headers-foreach"><function>soup_message_headers_foreach</function></link>.
 369 The response body (if any) is in the
 370 <structfield>response_body</structfield> field.
 371 </para>
 372
 373 <para>
 374 <link
 375 linkend="SoupMessageHeaders"><type>SoupMessageHeaders</type></link>
 376 automatically parses several important headers in
 377 <structfield>response_headers</structfield> for you and provides
 378 specialized accessors for them. Eg, <link
 379 linkend="soup-message-headers-get-content-type"><function>soup_message_headers_get_content_type</function></link>.
 380 There are several generic methods such as <link
 381 linkend="soup-header-parse-param-list"><function>soup_header_parse_param_list</function></link>
 382 (for parsing an attribute-list-type header) and <link
 383 linkend="soup-header-contains"><function>soup_header_contains</function></link>
 384 (for quickly testing if a list-type header contains a particular
 385 token). These handle the various syntactical oddities of parsing HTTP
 386 headers much better than functions like
 387 <function>g_strsplit</function> or <function>strstr</function>.
 388 </para>
 389
 390 </refsect2>
 391
 392 <refsect2>
 393 <title>Intermediate/Automatic Processing</title>
 394
 395 <para>
 396 You can also connect to various <type>SoupMessage</type> signals to do
 397 processing at intermediate stages of HTTP I/O. Eg, the <link
 398 linkend="SoupMessage-got-chunk"><literal>got-chunk</literal></link>
 399 signal is emitted as each piece of the response body is read (allowing
 400 you to provide progress information when receiving a large response,
 401 for example). <type>SoupMessage</type> also provides two convenience
 402 methods, <link
 403 linkend="soup-message-add-header-handler"><function>soup_message_add_header_handler</function></link>,
 404 and <link
 405 linkend="soup-message-add-status-code-handler"><function>soup_message_add_status_code_handler</function></link>,
 406 which allow you to set up a signal handler that will only be invoked
 407 for messages with certain response headers or status codes.
 408 <type>SoupSession</type> uses this internally to handle authentication
 409 and redirection.
 410 </para>
 411
 412 <para>
 413 When using the synchronous API, the callbacks and signal handlers will
 414 be invoked during the call to <link
 415 linkend="soup-session-send-message"><function>soup_session_send_message</function></link>.
 416 </para>
 417
 418 <para>
 419 To automatically set up handlers on all messages sent via a session,
 420 you can connect to the session's <link
 421 linkend="SoupSession-request-started"><literal>request_started</literal></link>
 422 signal, and add handlers to each message from there.
 423 </para>
 424
 425 </refsect2>
 426
 427 <refsect2>
 428 <title>Handling Authentication</title>
 429
 430 <para>
 431 <type>SoupSession</type> handles most of the details of HTTP
 432 authentication for you. If it receives a 401 ("Unauthorized") or 407
 433 ("Proxy Authentication Required") response, the session will emit the
 434 <link linkend="SoupSession-authenticate">authenticate</link> signal,
 435 providing you with a <link
 436 linkend="SoupAuth"><type>SoupAuth</type></link> object indicating the
 437 authentication type ("Basic", "Digest", or "NTLM") and the realm name
 438 provided by the server. If you have a username and password available
 439 (or can generate one), call <link
 440 linkend="soup-auth-authenticate"><function>soup_auth_authenticate</function></link>
 441 to give the information to libsoup. The session will automatically
 442 requeue the message and try it again with that authentication
 443 information. (If you don't call
 444 <function>soup_auth_authenticate</function>, the session will just
 445 return the message to the application with its 401 or 407 status.)
 446 </para>
 447
 448 <para>
 449 If the server doesn't accept the username and password provided, the
 450 session will emit <link
 451 linkend="SoupSession-authenticate">authenticate</link> again, with the
 452 <literal>retrying</literal> parameter set to <literal>TRUE</literal>. This lets the
 453 application know that the information it provided earlier was
 454 incorrect, and gives it a chance to try again. If this
 455 username/password pair also doesn't work, the session will contine to
 456 emit <literal>authenticate</literal> again and again until the
 457 provided username/password successfully authenticates, or until the
 458 signal handler fails to call <link
 459 linkend="soup-auth-authenticate"><function>soup_auth_authenticate</function></link>,
 460 at which point <application>libsoup</application> will allow the
 461 message to fail (with status 401 or 407).
 462 </para>
 463
 464 <para>
 465 If you need to handle authentication asynchronously (eg, to pop up a
 466 password dialog without recursively entering the main loop), you can
 467 do that as well. Just call <link
 468 linkend="soup-session-pause-message"><function>soup_session_pause_message</function></link>
 469 on the message before returning from the signal handler, and
 470 <function>g_object_ref</function> the <type>SoupAuth</type>. Then,
 471 later on, after calling <function>soup_auth_authenticate</function>
 472 (or deciding not to), call <link
 473 linkend="soup-session-unpause-message"><function>soup_session_unpause_message</function></link>
 474 to resume the paused message.
 475 </para>
 476
 477 </refsect2>
 478
 479 <refsect2>
 480 <title>Multi-threaded usage</title>
 481
 482 <para>
 483 The only explicitly thread-safe operations in
 484 <application>libsoup</application> are <link
 485 linkend="SoupSessionSync"><type>SoupSessionSync</type></link>'s
 486 implementations of the <link
 487 linkend="SoupSession"><type>SoupSession</type></link> methods. So
 488 after creating a <type>SoupSessionSync</type>, you can call <link
 489 linkend="soup-session-send-message"><function>soup_session_send_message</function></link>
 490 and <link
 491 linkend="soup-session-cancel-message"><function>soup_session_cancel_message</function></link>
 492 on it from any thread. But, eg, while the session is processing a
 493 message, you should not call any <link
 494 linkend="SoupMessage"><type>SoupMessage</type></link> methods on it
 495 from any thread other than the one in which it is being sent. (That
 496 is, you should not call any <type>SoupMessage</type> methods on it
 497 except from a message or session callback or signal handler.)
 498 </para>
 499
 500 <para>
 501 All other objects (including <link
 502 linkend="SoupSessionAsync"><type>SoupSessionAsync</type></link>)
 503 should only be used from a single thread, with objects that are also
 504 only be used from that thread. (And in particular, if you set a
 505 non-default <link
 506 linkend="GMainContext"><type>GMainContext</type></link> on a session,
 507 socket, etc, then you can only use that object from the thread in
 508 which that <type>GMainContext</type> is running.)
 509 </para>
 510
 511 </refsect2>
 512
 513 <refsect2>
 514 <title>Sample Programs</title>
 515
 516 <para>
 517 A few sample programs are available in the
 518 <application>libsoup</application> sources:
 519 </para>
 520
 521 <itemizedlist>
 522     <listitem><para>
 523         <emphasis role="bold"><literal>get</literal></emphasis> is a simple command-line
 524         HTTP GET utility using the asynchronous API.
 525     </para></listitem>
 526
 527     <listitem><para>
 528         <emphasis role="bold"><literal>getbug</literal></emphasis> is a trivial
 529         demonstration of the <link
 530         linkend="libsoup-XMLRPC-Support">XMLRPC</link> interface.
 531         (<emphasis
 532         role="bold"><literal>xmlrpc-test</literal></emphasis> provides
 533         a slightly more complicated example.)
 534     </para></listitem>
 535
 536     <listitem><para>
 537         <emphasis role="bold"><literal>auth-test</literal></emphasis> shows how to use
 538         authentication handlers and status-code handlers, although in
 539         a fairly unusual way.
 540     </para></listitem>
 541
 542     <listitem><para>
 543         <emphasis role="bold"><literal>simple-proxy</literal></emphasis> uses both the
 544         client and server APIs to create a simple (and not very
 545         RFC-compliant) proxy server. It shows how to use the <link
 546         linkend="SoupMessageFlags"><literal>SOUP_MESSAGE_OVERWRITE_CHUNKS</literal></link>
 547         flag when reading a message to save memory by processing each
 548         chunk of the message as it is read, rather than accumulating
 549         them all into a single buffer to process all at the end.
 550     </para></listitem>
 551 </itemizedlist>
 552
 553 <para>
 554 More complicated examples are available in GNOME CVS. The <ulink
 555 url="http://live.gnome.org/LibSoup"><application>libsoup</application>
 556 pages</ulink> on the GNOME wiki include a <ulink
 557 url="http://live.gnome.org/LibSoup/Users">list of applications using
 558 <application>libsoup</application></ulink>.
 559 </para>
 560
 561 </refsect2>
 562
 563 </refentry>