Working with asynchronous code can be challenging. This cookbook provides concrete examples of many different kinds of asynchronous calling mechanisms you may encounter. The cookbook assumes you have previously looked at the Async Reading which provides a higher-level view of async development. All of the code in this document can be found in the Async Cookbook Repo if you are a current CS student at UBC.
Async programming can be deceptively tricky. There are four main traps you should be aware of:
-
Async code often doesn't behave as you might expect when you run it. The easiest way to get a handle on how your code is executing is either with the debugger or with logging statements (e.g., before, during, and after calls).
-
Errors can be harder to detect. Be sure to always check your async code in failing conditions (e.g., by intentionally inducing failure) to ensure errors are handled appropriately.
-
With callbacks and other anonymous functions, the
thisvariable often does not point to the object you think it does. Using arrow notation (e.g.,() => { .. }) is the easiest way to deal with this, although there are many other approaches. -
Working with many async tasks at once (e.g.,
Promise.allorfor await) can sometimes obscure whether your code is waiting for one async task to complete before staring the next. This can unexpectedly degrade program performance.
These examples below all rely on the MathJS REST library. We have developed two asynchronous methods for calling the MathJS endpoint:
public async calcAsync(expression: string): Promise<string>public calcCB(expression: string,
callback: (err: string, result: string) => void): voidFor all examples below, you can assume you have access to a field called private math: MathJS that provides both methods declared above. Additionally, a string expression (e.g., 300+10) is provided in the exp variable.
Calling asynchronous methods requires more care than synchronous methods because the method does not return as you might normally expect.
For instance, the simplest example of an async method is setTimeout:
console.log(' before');
setTimeout( () => {
console.log(' during');
}, 0);
console.log(' after');While reading this code from top to bottom, you might expect it to print before during after, at runtime it actually prints before after during. This is because the setTimeout method is asynchronous and its anonymous callback is not executed in the callback queue (sometimes called the task queue) until after console.log(' after'); has executed, or in other words, until the call stack is empty. To learn more about the callback queue and call stacks, you can refer to this in-depth video by Philip Roberts.
Three predominant mechanisms exist for executing asynchronous work in Typescript: callbacks, promises, and async/await. Each have their benefits and drawbacks.
All three mechanisms work well for making a single async call, although callbacks are the most verbose, and async/await are the most visually similar to synchronous code.
Callbacks were the original mechanism in Typescript for completing asynchronous work.
this.math.calcCB(exp, (err: string, result: string) => {
if (err) {
console.log("singleCallback - ERROR; err: " + err);
} else {
console.log("singleCallback - done; result: " + result);
}
});The second parameter to calcCB is a callback: this is a method that is called by the math calcCB when it is done asynchronously computing the calculation. The callback takes two parameters: err and result. By convention, callback-based code returns an error parameter first that can be checked for errors, and returned results as subsequent parameters.
If a Promise settles successfully, it calls its then function; if an error is encountered, the promise settles by calling its catch function.
this.math.calcAsync(exp).then( (result) => {
console.log("singlePromise - done; result: " + result);
}).catch( (err) => {
console.log("singlePromise - ERROR; err: " + err);
});While both the .then and .catch function are optional, the result of the async call cannot be retrieved without including .then, and difficult-to-diagnose errors can arise if .catch is omitted as the promise will silently fail.
The async/await version of the code almost looks like synchronous code. Only two differences exist: first, the await keyword specifies that the code following the call depends on the result and computation should be yielded until the async method has completed. Second, while not shown below, methods using the await keyword must be declared async in their method declaration.
try {
const result = await this.math.calcAsync(exp);
console.log("singleAsync - done; result: " + result);
} catch (err) {
console.log("singleAsync - ERROR; err: " + err);
}It often arises that several async calls need to be used to complete a single task. While the calls themselves are async, organizing them into a synchronous sequence so the output of an earlier call can be used in a subsequent call is often needed.
As the number of asynchronous callbacks increase, the difficulty in dealing with them grows. The code below provides a glimpse of 'the pyramid of doom' where nested callbacks create deep indentation structures that are error-prone and hard to debug.
this.math.calcCB(exp, (err: string, result: string) => {
if (err) {
console.log("sequentialCallback - ERROR; err: " + err);
} else {
this.math.calcCB(result + "*10", (err: string, result: string) => {
if (err) {
console.log("sequentialCallback inner - ERROR; err: " + err);
} else {
console.log("sequentialCallback inner - done; result: " + result);
}
});
console.log("sequentialCallback - done; result: " + result);
}
});One of the greatest strengths of promises is to enable async calls to be marshalled into easier-to-understand sequences of actions. While the code below shows only two nested calls to calcAsync, these could be interspersed with any number of other async calls as well. Errors are handled with catch clauses, and these clauses can occur at any point in the sequence; errors will always be handled by their 'next' .catch clause.
Note in this example, the return is crucial for the second calcAsync call for the promise chain to maintain its flat structure.
this.math.calcAsync(exp).then((result) => {
console.log("sequentialPromise - done; result: " + result);
return this.math.calcAsync(exp + "*10");
}).then((result) => {
console.log("sequentialPromise inner - done; result: " + result);
}).catch((err) => {
console.log("sequentialPromise - ERROR; err: " + err);
});As with the single async call example, async/await provide the cleanest mechanism for writing async calls.
try {
let result = await this.math.calcAsync(exp);
console.log("sequentialAsync - done; result: " + result);
result = await this.math.calcAsync(exp + "*10");
console.log("sequentialAsync inner - done; result: " + result);
} catch (err) {
console.log("sequentialAsync - ERROR; err: " + err);
}Another common use case is when many independent concurrent jobs need to be executed. This may arise when reading or writing many different file-based or network-based resources.
Concurrent callbacks are extremely hard to manage, especially when there are an unknown number of them at runtime (the examples below for promises and async/await are handling five jobs). If you can avoid it, don't write them with traditional callbacks: you'll live a happier and more fruitful life.
Promise.all is used for exactly this case: this Promise takes a list of async tasks, waits for them to all complete (or fail), and then settles itself.
This is different to prior async calls. Note that jobs.push is pushing the unsettled Promise objects returned by calcAsync onto a standard list.
The .then clause of the Promise.all call is triggered when the jobs have completed (the .catch clause is triggered on error in any of the jobs).
let jobs = [];
let total = 1;
const opts = [11, 20, 30, 40, 51];
// create a list of asynchronous work
for (const o of opts) {
jobs.push(this.math.calcAsync(o + "+" + (o + 1)));
}
// wait for all asynchronous work to finish
Promise.all(jobs).then( (jobResults) => {
for (const result of jobResults) {
// each result is the settled value from the Promise
total += Number(result);
}
console.log("concurrentPromises done; total: " + total);
}).catch( (err) => {
// handle error
console.log("concurrentPromises - ERROR: " + err);
});While using the for await construct feels appealing as it helps the code look as close to the synchronous version as possible, this mechanism is prone to two shortcomings. First, it is extremely common to make mistakes that can result in the code running sequentially instead of in concurrent. Second, the try..catch does not reliably catch errors that may arise if any of the async jobs fail (this is not true for normal await calls, only for for await). Fortunately, a hybrid of these two approaches can address both of these shortcomings.
let jobs = [];
let total = 1;
const opts = [11, 20, 30, 40, 51];
// create a list of asynchronous work
for (const o of opts) {
jobs.push(this.math.calcAsync(o + "+" + (o + 1)));
}
try {
// wait for all asynchronous work to finish with await
const jobResults = await Promise.all(jobs);
for (const result of jobResults) {
// each result is the settled value from the Promise
total += Number(result);
}
console.log("concurrentHybridPromises done; total: " + total);
} catch (err) {
// handle error
console.log("concurrentHybridPromises - ERROR: " + err);
}Fortunately, both promises and async/await can be combined to strike a nice balance of understandability and reliability. Here Promise.all is used to wait for all work to be done, but rather than handling .all being done using a .then or .catch, we are using await which also enables us to use the standard exception handling mechanisms for errors.
try {
let jobs = [];
const opts = [11, 20, 30, 40, 51];
let total = 1;
// create a list of asynchronous work
for (const o of opts) {
// jobs.push(this.math.doStringyMath(o + "+" + (o + 1)));
jobs.push(this.math.calcAsync(o + "+XXX" + (o + 1)));
}
// wait for all asynchronous work to finish
const jobResults = await Promise.all(jobs);
for (const result of jobResults) {
// each result is the settled value from the Promise
total += Number(result);
}
console.log("concurrentAwait done; total: " + total);
} catch (err) {
// handle error
console.log("concurrentAwait - ERROR: " + err);
}THIS IS IMPORTANT
Testing is crucial to ensuring your asynchronous code is behaving as you might expect. Given how tricky these functions can be, it is important to check both the successful and failing conditions.
Testing can easily give a false sense of security though. While writing tests on asynchronous code, always make sure that succeeding tests can actually fail. Also ensure that tests of error conditions also still fail when the code unexpectedly succeeds.
For example, after writing passing successful test case, change the expected value and ensure that the test case actually fails. Similarly, for a passing failure test case, change the expected value and ensure that test case fails if the code does not fail as expected.
The test framework needs a way to tell when the callback has been completed. In Mocha, this is accomplished with the done callback. When the done function parameter is included, the test will not be considered complete until the done() method has been called or the test times out.
The test below ensures that the err callback parameter is not set for a successful test, an that the expected value is also correct.
it("Test single callback success", function (done) {
const expression = "155*2";
const expected = "310";
math.calcCB(expression, (err, result) => {
console.log("err: " + err + "; result: " + result);
expect(err).to.be.null;
expect(result).to.equal(expected);
done(); // signal that async function is complete
});
});The test below ensures that the err callback parameter is set for a failing test, and that it has the right value. It also checks that result is not set.
it("Test single callback failure", function (done) {
const expression = "INVALID";
const expected = "StatusCodeError: 400 - Error: Undefined symbol INVALID";
math.calcCB(expression, (err, result) => {
console.log("err: " + err + "; result: " + result);
expect(err).to.not.be.null;
expect(err).to.equal(expected);
expect(result).to.be.null;
done(); // signal that async function is complete
});
});When testing code with promises or async/await, we do not use the done callback, but instead let Mocha know we are testing async code by declaring the test function async in the test case declaration, and by returning a promise from within the test case.
In the test below, note that the promise is returned (e.g., return math.calcAsync). This form of test is somewhat verbose, but enables multiple assertions to be places on .then and .catch clauses to ease diagnosing failures. For a more succinct format, see the async test below.
it("Test single promise success", async function () {
const expression = "155*2";
const expected = "310";
// NOTE: this form of checking allows for multiple assertions
// on the success of the function. for a simpler form,
// see 'single async success' below
return math.calcAsync(expression).then((result) => {
console.log("result: " + result);
expect(result).to.not.be.null;
expect(result).to.equal(expected);
}).catch((err) => {
// catch is optional as the unhandled error will fail the test case
// but having it explicitly here makes it easier to understand the failure
console.log("err: " + err);
expect(err).to.be.null;
});
});Although the code below is expected to fail, it is important to also check that it did not succeed (e.g., expect(result).to.be.null). Additionally, if this line did fail, it would itself raise an exception that would be handled by the .catch, so this exception must not only check that the exception happened, but also its value was correct (to distinguish the expected failure from a failure caused by the success assertion failing above).
it("Test single promise failure", async function () {
const expression = "INVALID";
const expected = "StatusCodeError: 400 - \"Error: Undefined symbol INVALID\"";
// NOTE: this form of checking allows for multiple assertions
// on the failure of the function. for a simpler form,
// see 'single async success' below
return math.calcAsync(expression).then((result) => {
console.log("result: " + result);
expect(result).to.be.null;
}).catch((err) => {
// catch is optional as the unhandled error will fail the test case
// but having it explicitly here makes it easier to understand the failure
console.log("err: " + err);
expect(err.toString()).equal(expected); // check value so prior assertion failure won't slip through
});
});The below examples are best-practice approaches for testing your async code. These are preferred because you can set breakpoints at any point in the test: when the CUT is being invoked, after, in the catch, or when the assertions are being checked. These tests can be written more succinctly using the chai-as-promised package, but we strongly recommend not doing this as these tests often do not behave as you think they do.
it("Test single async success", async function () {
const expression = "155*2";
const expected = "310";
let result;
try {
result = await math.calcAsync(expression);
} catch (err) {
result = err;
} finally {
expect(result).to.equal(expected);
}
});While the failing path could be tested with try...catch, it is also possible to specify directly that the code under test is expected to throw, and to ensure that the error message provided is correct.
it("Test single async failure", async function () {
const expression = "INVALID";
const expected = "400 - \"Error: Undefined symbol INVALID\"";
let result;
try {
result = await math.calcAsync(expression);
} catch (err) {
result = err;
} finally {
expect(result).to.be.instanceOf(StatusCodeError);
expect(result.message).to.equal(expected);
}
});Testing more complex async code (e.g., sequential calls or calls that use Promise.all) often ends up being verbose. In this case instead of telling Mocha to wait until the promise settles, the test case instead awaits, just as any other async function might. This allows the test code to look similar to how this code probably executes within the program.
it("Test concurrent hybrid success", async function () {
const expression = "12+18";
const expected = 310;
let jobs = [];
let total = 1;
const opts = [13, 3, 2, 5, 54];
// create a list of asynchronous work
for (const o of opts) {
jobs.push(math.calcAsync(o + "+" + expression + "+" + (o + 1)));
}
// wait for all asynchronous work to finish
const jobResults = await Promise.all(jobs);
for (const result of jobResults) {
// each result is the settled value from the Promise
total += Number(result);
}
expect(total).to.equal(expected);
});While in the above example an error would fail the one assertion, sometimes it is desireable to validate that your error cases are also failing in the expected way, as can be seen in the following example:
it("Test concurrent hybrid failure", async function () {
const expression = "INVALID";
const expected = "400 - \"Error: Undefined symbol INVALID\"";
let jobResults;
try {
let jobs = [];
const opts = [13, 3, 2, 5, 54];
// create a list of asynchronous work
for (const o of opts) {
jobs.push(math.calcAsync(o + "+" + expression + "+" + (o + 1)));
}
// wait for all asynchronous work to finish
jobResults = await Promise.all(jobs);
} catch (err) {
jobResults = err;
} finally {
expect(jobResults).to.be.instanceOf(StatusCodeError);
expect(jobResults.message).to.equal(expected);
}
});Methods that return promises can be used using either .then and .catch or async/await. But sometimes you will have to work with methods that only use callbacks, but you still want to use the more advanced async mechanisms to handle them. To do this, you will have to wrap the callback-based implementation in a promise.
For example, setTimeout only offers a callback-based implementation. If we wanted to use setTimeout to add a delay to our code, we would have to restructure our code as follows:
setTimeout( () => {
// code to run after the delay here
}, 250);But that's not especially aesthetic. Instead, one might wish to create a promise-based wrapper on setTimeout so you could instead say:
await delay(250);
// code to be run after the delay hereTo do this, you could wrap setTimeout in a Promise. For example, the code below resolves the promise after ms has elapsed:
private delay(ms: number): Promise<void> {
return new Promise<void>( (resolve) => setTimeout(resolve, ms));
}A more comprehensive version is shown below; here we wrap the calcCB which we have been calling and testing with callbacks in this document in a promise, so it can be used as calcAsync has been.
public calcCBWrapped(expression: string): Promise<string> {
return new Promise<string>( (resolve, reject) => {
this.calcCB(expression, (err, result) => {
if (err !== null) {
reject(err);
} else {
resolve(result);
}
});
});
}This allows us to call this previously callback-based function as we might prefer:
const result = await calcCBWrapped("155*2");- Avoiding Async Hell
- Parallel vs Concurrent Execution in Node
- Nolan Lawson has some great examples about async handling in PouchDB:
