Thursday, December 10, 2015

Accessing a webservice using libsoup and liboauth

OAuth 1.0 (RFC 5849) is a common method used by web services for authorization. If you are working on a GLib based project there are two libraries that can help with this - libosoup and liboauth. The bad news for liboauth is it's not being developed anymore. The good news is OAuth 2.0 (RFC 6749) is replacing OAuth 1.0 so I guess that wont matter into the future.

There are a number of ways OAuth can be used, in this example I'll show how to obtain Ubuntu One credentials and use these to post a review for an Ubuntu application. This example will use JSON so refer to my previous tutorial on how to do this with JSON-GLib.

The first step is to get the credentials from Ubuntu One. After reading the API documentation this is fairly easy to acomplish.  For my application, I need to create a named token (I'll use "test-oauth"). To do this, I send my email address and password in JSON form to the server:

// Create JSON request
JsonBuilder *builder = json_builder_new ();

json_builder_begin_object (builder);
json_builder_set_member_name (builder, "email"); 
json_builder_add_string_value (builder, "test@example.com");
json_builder_set_member_name (builder, "password");
json_builder_add_string_value (builder, "secret");
json_builder_set_member_name (builder, "token_name");
json_builder_add_string_value (builder, "test-oauth"); 
json_builder_end_object (builder);

// Convert request into a string
JsonGenerator *generator = json_generator_new ();
json_generator_set_root (generator, json_builder_get_root (builder));
gsize length;
gchar *data = json_generator_to_data (generator, &length);

// Send request to the server
SoupSession *session = soup_session_new_with_options (SOUP_SESSION_USER_AGENT, "test-oauth", NULL);
SoupMessage *message = soup_message_new (SOUP_METHOD_POST, "https://login.ubuntu.com/api/v2/tokens/oauth");
soup_message_headers_append (message->request_headers, "Accept", "application/json");
soup_message_set_request (message, "application/json", SOUP_MEMORY_TAKE, data, length);
guint status_code = soup_session_send_message (session, message);
g_assert (status_code == SOUP_STATUS_CREATED);

If everything is correct, then the server will send back OAuth information for this token like this:

{

  "token_key": "vt81TzXi2DdNkS9WRa27FZOt4SMhrn4XlPI4YgyplS6TbxoCqnFZtVAWEDfn",
  "token_secret": "r232LlyogYQIN72AinyjGg32yKcEDP0w73Vwau0EpOzjC3XhQD9ID1X99OtaZcZk2yNUrG",
  "consumer_key": "K6Jl5et",
  "consumer_secret": "U23tFbCv9HCZJvvVZiBDBxZ2qcidOx"
}

The consumer_key is basically your user ID (named weirdly for historical reasons). token_key is what you use to mark requests with the "test-oauth" token. consumer_secret and token_secret are encryption keys you use to prove that consumer_key and token_key are yours. These must be kept secret (the hint is in the name)!

Now I have my OAuth token I can use it to review an Ubuntu package. I should also save this information so I can re-use it without returning to login.ubuntu.com.

I'll make a message to send to the reviews server:

// Create JSON request
JsonBuilder *builder = json_builder_new (); 
json_builder_begin_object (builder);
json_builder_set_member_name (builder, "package_name");
json_builder_add_string_value (builder, "no-such-package");
json_builder_set_member_name (builder, "rating");
json_builder_add_int_value (builder, 5);
// Add other required fields...
json_builder_end_object (builder);

// Convert request into a string
JsonGenerator *generator = json_generator_new ();
json_generator_set_root (generator, json_builder_get_root (builder));
gsize length;
gchar *data = json_generator_to_data (generator, &length);

// Prepare request for the server
SoupSession *session = soup_session_new_with_options (SOUP_SESSION_USER_AGENT, "test-oauth", NULL);
SoupMessage *message = soup_message_new (SOUP_METHOD_POST, "https://reviews.ubuntu.com/reviews/api/1.0/reviews/");
soup_message_headers_append (message->request_headers, "Accept", "application/json");
soup_message_set_request (message, "application/json", SOUP_MEMORY_TAKE, data, length);

If I send the message like this, the server will reject it because there's no valid HTTP Authorization header. To populate this I'll use liboauth.

The liboauth API is a bit confusing, so I'll run through the functions. Firstly the URL needs splitting into the parts that are needed for signing:

char **url_parameters = NULL;
int url_parameters_length = oauth_split_url_parameters ("https://reviews.ubuntu.com/reviews/api/1.0/reviews/", &url_parameters);

Next step is to generate the Oauth fields and the signature. These new fields are written back into url_parameters. Note the NULL argument - this is required if the HTTP request has content in the form application/x-www-form-urlencoded. In my case it's application/json so I don't have to bother.

oauth_sign_array2_process (&url_parameters_length, &url_parameters,
                           NULL, // No postargs
                           O_HMAC,
                           "POST",
                           oauth_consumer_key, oauth_consumer_secret,
                           oauth_token, oauth_token_secret);


Finally all the OAuth fields in url_parameters need to be combined into a valid OAuth field. The 6 is a bitfield (no defines in the liboauth API)  that specifies how to correctly choose and format the required fields.

char *p = oauth_serialize_url_sep (url_parameters_length, 1, url_parameters, ", ", 6);
gchar *authorization_text = g_strdup_printf ("OAuth realm=\"Ratings and Reviews\", %s", p);

And now authorization_text contains a valid Authorization HTTP header! I can now add this to the SoupMessage and send it to the server:

soup_message_headers_append (message->request_headers, "Authorization", authorization_text);
guint status_code = soup_session_send_message (session, message);
g_assert (status_code == SOUP_STATUS_OK);

And that's all it takes to send an OAuth authenticated request.

The full program:

// gcc -g -Wall example-oauth.c -o example-oauth `pkg-config --cflags --libs libsoup-2.4 oauth json-glib-1.0`

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#include <libsoup/soup.h>
#include <oauth.h>
#include <json-glib/json-glib.h>

static void
add_string_member (JsonBuilder *builder, const gchar *name, const gchar *value)
{
    json_builder_set_member_name (builder, name);
    json_builder_add_string_value (builder, value);
}

static void
add_int_member (JsonBuilder *builder, const gchar *name, gint64 value)
{
    json_builder_set_member_name (builder, name);
    json_builder_add_int_value (builder, value);
}

static void
set_request (SoupMessage *message, JsonBuilder *builder)
{
    JsonGenerator *generator = json_generator_new ();
    json_generator_set_root (generator, json_builder_get_root (builder));
    gsize length;
    gchar *data = json_generator_to_data (generator, &length);
    soup_message_set_request (message, "application/json", SOUP_MEMORY_TAKE, data, length);

    g_object_unref (generator);
}

static const char *
getfield (const char *prompt)
{
    static char value[1024];
    printf ("%s", prompt);
    fgets (value, 1024, stdin);
    g_strchomp (value);
    return value;
}

static void
sign_message (SoupMessage *message, const gchar *realm, OAuthMethod method,
              const gchar *oauth_consumer_key, const gchar *oauth_consumer_secret,
              const gchar *oauth_token, const gchar *oauth_token_secret)
{
    gchar *url, *oauth_authorization_parameters;
    gchar **url_parameters = NULL;
    int url_parameters_length;
    gchar *authorization_text;

    url = soup_uri_to_string (soup_message_get_uri (message), FALSE);

    url_parameters_length = oauth_split_url_parameters (url, &url_parameters);
    oauth_sign_array2_process (&url_parameters_length, &url_parameters,
                               NULL, // No postargs
                               method,
                               message->method,
                               oauth_consumer_key, oauth_consumer_secret,
                               oauth_token, oauth_token_secret);
    oauth_authorization_parameters = oauth_serialize_url_sep (url_parameters_length, 1, url_parameters, ", ", 6);
    authorization_text = g_strdup_printf ("OAuth realm=\"%s\", %s", realm, oauth_authorization_parameters);
    soup_message_headers_append (message->request_headers, "Authorization", authorization_text);

    g_free (url);
    oauth_free_array (&url_parameters_length, &url_parameters);
    free (oauth_authorization_parameters);
}

int main (int argc, char **argv)
{
    const gchar *token_name = "test-oauth";

    SoupSession *session = soup_session_new_with_options (SOUP_SESSION_USER_AGENT, "test-oauth", NULL);

    /* Load the OAuth key, token and secrets */
    GKeyFile *config = g_key_file_new ();
    gchar *path = g_build_filename (g_get_user_config_dir (), "oauth-test.conf", NULL);
    g_key_file_load_from_file (config, path, G_KEY_FILE_NONE, NULL);
    gchar *oauth_consumer_key = g_key_file_get_string (config, token_name, "consumer-key", NULL);
    gchar *oauth_consumer_secret = g_key_file_get_string (config, token_name, "consumer-secret", NULL);
    gchar *oauth_token = g_key_file_get_string (config, token_name, "token", NULL);
    gchar *oauth_token_secret = g_key_file_get_string (config, token_name, "token-secret", NULL);
    if (!oauth_consumer_key || !oauth_consumer_secret || !oauth_token || !oauth_token_secret)
    {
        /* Request a token from the Canonical Identity Provider */
        SoupMessage *message = soup_message_new (SOUP_METHOD_POST, "https://login.ubuntu.com/api/v2/tokens/oauth");
        soup_message_headers_append (message->request_headers, "Accept", "application/json");
        JsonBuilder *builder = json_builder_new ();
        json_builder_begin_object (builder);
        add_string_member (builder, "email", getfield ("Email address: "));
        add_string_member (builder, "password", getpass ("Password: "));
        add_string_member (builder, "otp", getfield ("Verification code: "));
        add_string_member (builder, "token_name", token_name);
        json_builder_end_object (builder);
        set_request (message, builder);
        g_object_unref (builder);
        guint status_code = soup_session_send_message (session, message);
        g_assert (status_code == SOUP_STATUS_CREATED);

        /* Parse response */
        JsonParser *parser = json_parser_new ();
        gboolean result = json_parser_load_from_data (parser, message->response_body->data, -1, NULL);
        g_assert (result);
        JsonNode *root = json_parser_get_root (parser);
        g_assert (JSON_NODE_HOLDS_OBJECT (root));
        JsonObject *object = json_node_get_object (root);
        oauth_consumer_key = g_strdup (json_object_get_string_member (object, "consumer_key"));
        oauth_consumer_secret = g_strdup (json_object_get_string_member (object, "consumer_secret"));     
        oauth_token = g_strdup (json_object_get_string_member (object, "token_key"));
        oauth_token_secret = g_strdup (json_object_get_string_member (object, "token_secret"));

        /* Write config so we don't do this the second time */
        g_key_file_set_string (config, token_name, "consumer-key", oauth_consumer_key);
        g_key_file_set_string (config, token_name, "consumer-secret", oauth_consumer_secret);
        g_key_file_set_string (config, token_name, "token", oauth_token);
        g_key_file_set_string (config, token_name, "token-secret", oauth_token_secret);
        g_key_file_save_to_file (config, path, NULL);

        g_object_unref (message);
        g_object_unref (parser);
    }
    g_free (path);

    /* Make the request using a HTTP POST signed with OAuth */
    SoupMessage *message = soup_message_new (SOUP_METHOD_POST, "https://reviews.ubuntu.com/reviews/api/1.0/reviews/");
    g_assert (message != NULL);
    soup_message_headers_append (message->request_headers, "Accept", "application/json");
    JsonBuilder *builder = json_builder_new ();
    json_builder_begin_object (builder);
    add_string_member (builder, "package_name", "no-such-package");
    add_string_member (builder, "summary", "★★★★★");
    add_string_member (builder, "review_text", "This is the best non existant application ever!");
    add_string_member (builder, "language", "en");
    add_string_member (builder, "origin", "ubuntu");
    add_string_member (builder, "distroseries", "xenial");
    add_string_member (builder, "version", "42");
    add_int_member (builder, "rating", 5);
    add_string_member (builder, "arch_tag", "amd64");
    json_builder_end_object (builder);
    set_request (message, builder);
    g_object_unref (builder);
    sign_message (message, "Ratings and Reviews", OA_HMAC, oauth_consumer_key, oauth_consumer_secret, oauth_token, oauth_token_secret);
    guint status_code = soup_session_send_message (session, message);
    g_printerr ("%d '%s'\n", status_code, message->response_body->data);
    g_assert (status_code == SOUP_STATUS_OK);

    /* Parse the data in JSON format */
    JsonParser *parser = json_parser_new ();
    gboolean result = json_parser_load_from_data (parser, message->response_body->data, -1, NULL);
    g_assert (result);
    JsonNode *root = json_parser_get_root (parser);
    g_assert (JSON_NODE_HOLDS_OBJECT (root));
    JsonObject *object = json_node_get_object (root);
    gint64 review_id = json_object_get_int_member (object, "id");
    g_print ("Review created with ID %" G_GUINT64_FORMAT "\n", review_id);

    /* Clean up */
    g_object_unref (session);
    g_object_unref (message);
    g_object_unref (parser);
    g_free (oauth_consumer_key);
    g_free (oauth_consumer_secret);
    g_free (oauth_token);
    g_free (oauth_token_secret);
 
    return 0;
}