The Async Keyword

The Python lexer has to deal with a new keyword to parse this variant of hello_world: we can look at it using the ast module.

import ast
import pprint
pprint.pp(ast.dump(ast.parse("""
async def hello_world():
    print("Hello, world!")
""")))

("Module(body=[AsyncFunctionDef(name='hello_world', "
 'args=arguments(posonlyargs=[], args=[], kwonlyargs=[], kw_defaults=[], '
 "defaults=[]), body=[Expr(value=Call(func=Name(id='print', ctx=Load()), "
 "args=[Constant(value='Hello, world!')], keywords=[]))], decorator_list=[], "
 'type_params=[])], type_ignores=[])')

The function is explicitly detected as an AsyncFunctionDef to mark the addition of the async tag. ¹

We can also look at the disassembly of the function definition to see what happens after compiling it down to bytecode. I've also included a plain function for comparison.

import ast
import dis

code = dis.Bytecode(compile("""
async def async_hello_world():
    print("Async Hello, world!")

def hello_world():
    print("Hello, world!")
""", filename='<string>', mode='exec'))

print(code.dis())

0           0 RESUME                   0

2           2 LOAD_CONST               0 (<code object async_hello_world at 0x7f3d1b7dd110, file "<string>", line 2>)
            4 MAKE_FUNCTION            0
            6 STORE_NAME               0 (async_hello_world)

5           8 LOAD_CONST               1 (<code object hello_world at 0x7f3d1b711b50, file "<string>", line 5>)
           10 MAKE_FUNCTION            0
           12 STORE_NAME               1 (hello_world)
           14 RETURN_CONST             2 (None)

Surprisingly enough, the generated opcodes for defining the functions are exactly the same. Looking inside the functions themselves also gives the same result.

import dis

async def async_hello_world():
    print("Hello, world!")

def hello_world():
    print("Hello, world!")

print("Async")
dis.dis(async_hello_world)

print("Function")
dis.dis(hello_world)

Async
  3           0 RETURN_GENERATOR
              2 POP_TOP
              4 RESUME                   0

  4           6 LOAD_GLOBAL              1 (NULL + print)
             16 LOAD_CONST               1 ('Hello, world!')
             18 CALL                     1
             26 POP_TOP
             28 RETURN_CONST             0 (None)
        >>   30 CALL_INTRINSIC_1         3 (INTRINSIC_STOPITERATION_ERROR)
             32 RERAISE                  1
ExceptionTable:
  4 to 28 -> 30 [0] lasti
Function
  6           0 RESUME                   0

  7           2 LOAD_GLOBAL              1 (NULL + print)
             12 LOAD_CONST               1 ('Hello, world!')
             14 CALL                     1
             22 POP_TOP
             24 RETURN_CONST             0 (None)

The only difference is a flag set on the defined code: CO_COROUTINE which makers a function as a coroutine. The actual compiler code is somewhere around here; but it's worth explicitly looking at the flags.

Once more into the breach:

import dis

async def async_hello_world():
    print("Hello, world!")

def hello_world():
    print("Hello, world!")

async_flags = async_hello_world.__code__.co_flags
standard_flags = hello_world.__code__.co_flags

only_async_flags = async_flags & (~standard_flags)
print(f"{only_async_flags=}, aka {dis.COMPILER_FLAG_NAMES[only_async_flags]}")

only_standard_flags = standard_flags & (~async_flags)
print(f"{only_standard_flags=}")

only_async_flags=128, aka COROUTINE
only_standard_flags=0

And that's the literally the magic bit that makes the difference in how coroutines are evaluated.

Executing an async function doesn't directly run the inner function anymore, and instead returns a a coroutine object that wraps the computation.

hello_world()

Hello, world!