Using async/await to simplify promises

Published 2017-11-03

In a series of prior posts, I had introduced the promise abstraction along with the promise Tcl package and how it can greatly simplify certain forms of asynchronous computation. As mentioned there, the Tcl package is roughly based on the promise framework implemented in ES6 (ECMAScript, aka Javascript). ES7 introduced the async and await functions which further simplify asynchronous programming with promises in certain scenarios. Accordingly, the Tcl promise package has been updated to include the equivalent commands. This post introduces their use.

Motivation

We will start a slightly modified form of an example from an earlier post to provide the motivation for the async/await commands. What we want to be able to do is to download a set of URL's into a directory, place them into a .zip archive and then email them. The catch is that the whole process of downloading, zipping and emailing must not block our process while it is ongoing. It must happen in the background while allowing our UI updates, user interaction etc. to function normally.

Our promise-based solution uses the download procedure defined below.

NOTE: All code samples assume the promise namespace is on the namespace path so that the commands can be used without qualification. Moreover, remember that promises require the event loop to be running.

proc download {dir urls} {
    return [all [lmap url $urls {
        [pgeturl $url] then [lambda {dir url http_state} {
            save_file $dir $url [dict get $http_state body]
        } $dir $url]
    }]]
}

This procedure will initiate download of the specified URL's in parallel and in the background without blocking. It returns a promise that is fulfilled when the downloads complete. If you do not follow the above, you will need to read the previous posts on promises.

Given the above procedure, we can write the sequence of background operations as the zipnmail procedure below.

proc zipnmail {dir urls} {
    set downloads [download $dir $urls]
    set zip [$downloads then [lambda {dir dontcare} {
        then_chain [pexec zip -r pages.zip $dir]
    } $dir]]
    set email [$zip then [lambda dontcare {
        then_chain [pexec blat pages.zip -to [email protected]]
    }]]
    $email done [lambda {dontcare} {
        tk_messageBox -message "Zipped and sent!"
    }]
}

This procedure "chains" together a sequence of asynchronous steps — the download, zip and email — abstracted as promises and then displays a message box when it's all done. Again, refer to previous posts if you are lost.

If you do not appreciate the value of the promise abstractions, try to implement the same functionality (remember it has to be asynchronous) using plain old Tcl as suggested in previous posts.

However, we can do better now that we have async and await. Here is the new and improved version:

async zipnmail {dir urls} {
    await [download $dir $urls]
    await [pexec zip -r pages.zip $dir]
    await [pexec blat pages.zip -to [email protected]]
    tk_messageBox -message "Zipped and sent!"
}

As you can see, this new procedure, defined using the async command and not proc, is significantly simpler and more readable than the previous version. The boilerplate of sequencing asynchronous operations is encapsulated by the async / await commands and the intent is clear. And although the flow looks sequential, the execution is asynchronous and does not block the application while the operations are being executed.

We can call it as follows:

set prom [zipnmail c:/temp/downloads {http://www.example.com http://www.magicsplat.com}]
$prom done

With that motivating example behind us, let us look at the async and await commands in more detail.

The `async` command

The async command is identical in form to Tcl's proc command, taking a name as its first argument, followed by parameter definitions and then the procedure body. Just like proc, it results in a command being created of the name passed as the first argument in the appropriate namespace context as well as an anonymous procedure defined using the supplied parameter definitions and body.

When the created command is invoked, it instantiates a hidden coroutine context and executes the anonymous procedure within that context passing it the supplied arguments. The return value from the command is a promise. This promise will be fulfilled with the value returned from the procedure if it completes normally and rejected with the error information if the procedure raises an error.

Let us start with an example that has no asynchronous operations.

async demo {a b} {
    return [expr {$a+$b}]
}

The above defines a procedure demo. However, calling the procedure does not return the concatenation of the two arguments. Rather, it returns a promise that will be fulfilled when the supplied body (run within a coroutine context) completes. In this case, there is no asynchronicity involved so it will complete immediately. We can try it out thus in a command shell.

% set prom [demo 40 2]
::oo::Obj70
% $prom done puts
42

Similarly, in case the command failed, the promise would be rejected as discussed in a previous post. For example, if we tried to pass non-numeric operands, the promise is rejected:

% set prom [demo a b]
::oo::Obj71
% $prom done puts [lambda {msg error_dict} {puts $msg}]
can't use non-numeric string as operand of "+"

Of course there is really no reason to use async above because there are no blocking operations that we want to avoid. So we will look at another example where we add asynchronous tasks to avoid blocking the whole application.

The example below simulates a "complex" computation. The result of the first computation is used in the second so there is a dependency that implies sequential execution between the two. However, remember we do not want to block the application so these two sequential steps need to be run asynchronously. The asynchronous procedure would be defined as follows.

async demo {a b} {
    set p1 [ptask "expr $a * 10"]
    set p2 [$p1 then [lambda {b val} {then_chain [ptask "expr $b +  $val"]} $b]]
    async_chain $p2
}

To summarize the operation of the above code, a task is initiated to compute expr $a*10. The second piece of computation is chained to it so that once the first one finishes, the second one is initiated. Finally, the async_chain links the promise returned by a call to the demo command to the result (promise) of the second computation. Usage would be as follows:

% set p [demo 4 2]
::oo::Obj73
% $p done puts
42

(A reminder that the done method is not synchronous either; it queues the command, puts in this case, to be invoked when the promise is fulfilled.)

Although considerably simpler than an equivalent version that did not use promises, our asynchronous demo above still needs some effort to wrap one's head around it. The flow of control via then and then_chain is non-obvious unless you are well-versed with promises. It would be nice to simplify all that boilerplate. This is where the await command comes in.

The `await` command

The await command allows a command created with async to suspend (without blocking the application as a whole) until a specified promise is settled. If the promise is fulfilled, the await command returns the value with which the promise was fulfilled. If the promise is rejected, the await command raises an error accordingly. Our last example can then be written as follows:

async demo {a b} {
    set 10x [await [ptask "expr $a * 10"]]
    set result [await [ptask "expr $10x + $b"]]
    return $result
}

Since a command defined via async always returns a promise even when the result is explicitly returned as above, usage is same as before.

% set p [demo 4 2]
::oo::Obj73
% $p done puts
42

Notice how much clearer the flow is for this sequence of asynchronous steps. Moreover, the result is available directly as the return value from await making it easier to use as opposed to the use of then, done and friends as is required with direct use of promises.

NOTE: Keep in mind that the await command can only be used within the context of an async command (including any procedures called within that context).

Error handling

A command created via async always returns a promise and thus all error handling is done in the same manner as has been described before using the reject handlers registered via then, catch etc. on the promise returned by the command. Any errors within the body of the command are automatically translated to the corresponding promise being rejected.

The await command, used within a async procedure body, raises exceptions on error like any other Tcl command and can be handled using catch, try etc. If unhandled, it percolates up and causes the containing async procedure promise to be rejected.

Motivation

The async command

The await command

Error handling

Further reading

The `async` command

The `await` command