expLog

The Async Keyword

This note is part of a series on asyncio.

The very first change to start using asyncio is to insert the async keyword right in front of the function: but what does that do?

To keep things extremely focused, I'm only going to compare stripped versions of the hello world program:

async def hello_world():
    print("Hello, world")

and

def sync_hello_world():
    print("Hello, world")

Return values

The more obvious result is that calling the sync function immediately triggers a print, and returns nothing;

print(f"{sync_hello_world()=}")

calling the async function returns a coroutine object, doesn't print any output (or run the underlying code), and also prints a warning about a coroutine that was never called:

print(f"{hello_world()=}")

Compilation

So what makes Python behave so completely differently in this case?

Compiling

The Python lexer has to deal with a new keyword to parse this variant of hello_world: we can look at it using the ast module.

import ast
import pprint
pprint.pp(ast.dump(ast.parse("""
async def hello_world():
    print("Hello, world!")
""")))
("Module(body=[AsyncFunctionDef(name='hello_world', "
 'args=arguments(posonlyargs=[], args=[], kwonlyargs=[], kw_defaults=[], '
 "defaults=[]), body=[Expr(value=Call(func=Name(id='print', ctx=Load()), "
 "args=[Constant(value='Hello, world!')], keywords=[]))], decorator_list=[])], "
 'type_ignores=[])')

The function is explicitly detected as an AsyncFunctionDef to mark the addition of the async tag. 1

We can also look at the disassembly of the function definition to see what happens after compiling it down to bytecode. I've also included a plain function for comparison.

import ast
import dis

code = dis.Bytecode(compile("""
async def async_hello_world():
    print("Async Hello, world!")

def hello_world():
    print("Hello, world!")
""", filename='<string>', mode='exec'))

print(code.dis())
2           0 LOAD_CONST               0 (<code object async_hello_world at 0x7fe266b1c450, file "<string>", line 2>)
            2 LOAD_CONST               1 ('async_hello_world')
            4 MAKE_FUNCTION            0
            6 STORE_NAME               0 (async_hello_world)

5           8 LOAD_CONST               2 (<code object hello_world at 0x7fe266b1c5b0, file "<string>", line 5>)
           10 LOAD_CONST               3 ('hello_world')
           12 MAKE_FUNCTION            0
           14 STORE_NAME               1 (hello_world)
           16 LOAD_CONST               4 (None)
           18 RETURN_VALUE

Surprisingly enough, the generated opcodes for defining the functions are exactly the same. Looking inside the functions themselves also gives the same result.

import dis

async def async_hello_world():
    print("Hello, world!")

def hello_world():
    print("Hello, world!")

print("Async")
dis.dis(async_hello_world)

print("Function")
dis.dis(hello_world)
Async
  4           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 ('Hello, world!')
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE
Function
  7           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 ('Hello, world!')
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE

The only difference is a flag set on the defined code: CO_COROUTINE which makers a function as a coroutine. The actual compiler code is somewhere around here; but it's worth explicitly looking at the flags.

Once more into the breach:

import dis

async def async_hello_world():
    print("Hello, world!")

def hello_world():
    print("Hello, world!")

async_flags = async_hello_world.__code__.co_flags
standard_flags = hello_world.__code__.co_flags

only_async_flags = async_flags & (~standard_flags)
print(f"{only_async_flags=}, aka {dis.COMPILER_FLAG_NAMES[only_async_flags]}")

only_standard_flags = standard_flags & (~async_flags)
print(f"{only_standard_flags=}")
only_async_flags=128, aka COROUTINE
only_standard_flags=0

And that's the literally the magic bit that makes the difference in how coroutines are evaluated.

Execution

Executing an async function doesn't directly run the inner function anymore, and instead returns a a coroutine object that wraps the computation.

hello_world()
Hello, world!
1

Just for contrast:

import ast
import pprint

pprint.pprint(ast.dump(ast.parse("""
def hello_world():
    print("Hello, world!")
""")))
("Module(body=[FunctionDef(name='hello_world', args=arguments(posonlyargs=[], "
 'args=[], kwonlyargs=[], kw_defaults=[], defaults=[]), '
 "body=[Expr(value=Call(func=Name(id='print', ctx=Load()), "
 "args=[Constant(value='Hello, world!')], keywords=[]))], decorator_list=[])], "
 'type_ignores=[])')

Normal functions are parsed, unsurprisingly, as FunctionDef.

view source