#+TITLE: The Async Keyword

This note is part of a series on asyncio.

The very first change to start using asyncio is to insert the async
keyword right in front of the function: but what does that do?

To keep things extremely focused, I'm only going to compare stripped
versions of the hello world program:

#+CAPTION: Gratituously async
#+begin_src python :session hello
async def hello_world():
    print("Hello, world")
#+end_src

#+RESULTS:

and

#+CAPTION: Boringly sync
#+begin_src python :session hello
def sync_hello_world():
    print("Hello, world")
#+end_src

#+RESULTS:

** Return values
The more obvious result is that calling the sync function immediately
triggers a print, and returns nothing; 

#+begin_src python :session hello :results output
print(f"{sync_hello_world()=}")
#+end_src

#+RESULTS:
: Hello, world
: sync_hello_world()=None

calling the async function returns a coroutine object, doesn't print
any output (or run the underlying code), and also prints a warning
about a coroutine that was never called:
#+begin_src python :session hello :results output
print(f"{hello_world()=}")
#+end_src

#+RESULTS:
: /tmp/babel-UFSwjX/python-7u8A8Z:1: RuntimeWarning: coroutine 'hello_world' was never awaited
:   print(f"{hello_world()=}")
: RuntimeWarning: Enable tracemalloc to get the object allocation traceback
: hello_world()=<coroutine object hello_world at 0x7f3b6c847140>

** Compilation
So what makes Python behave so completely differently in this case?

# TODO Look at classical function dispatch to make sure I'm not
# missing anything major
# TODO Also look at the Python internals chapter on asyncio


*** Compiling
The Python lexer has to deal with a new keyword to parse this variant
of hello_world: we can look at it using the ast module.

#+begin_src python :results output :exports both 
import ast
import pprint
pprint.pp(ast.dump(ast.parse("""
async def hello_world():
    print("Hello, world!")
""")))
#+end_src

#+RESULTS:
: ("Module(body=[AsyncFunctionDef(name='hello_world', "
:  'args=arguments(posonlyargs=[], args=[], vararg=None, kwonlyargs=[], '
:  'kw_defaults=[], kwarg=None, defaults=[]), '
:  "body=[Expr(value=Call(func=Name(id='print', ctx=Load()), "
:  "args=[Constant(value='Hello, world!', kind=None)], keywords=[]))], "
:  'decorator_list=[], returns=None, type_comment=None)], type_ignores=[])')

The function is explicitly detected as an AsyncFunctionDef to mark the
addition of the async tag. [fn:functions]

We can also look at the disassembly of the function definition to see
what happens after compiling it down to bytecode. I've also included a
plain function for comparison.

#+begin_src python :results output :exports both 
import ast
import dis

code = dis.Bytecode(compile("""
async def async_hello_world():
    print("Async Hello, world!")

def hello_world():
    print("Hello, world!")
""", filename='<string>', mode='exec'))

print(code.dis())
#+end_src

#+RESULTS:
#+begin_example
  2           0 LOAD_CONST               0 (<code object async_hello_world at 0x7f07b95252f0, file "<string>", line 2>)
              2 LOAD_CONST               1 ('async_hello_world')
              4 MAKE_FUNCTION            0
              6 STORE_NAME               0 (async_hello_world)

  5           8 LOAD_CONST               2 (<code object hello_world at 0x7f07b9525450, file "<string>", line 5>)
             10 LOAD_CONST               3 ('hello_world')
             12 MAKE_FUNCTION            0
             14 STORE_NAME               1 (hello_world)

             16 LOAD_CONST               4 (None)
             18 RETURN_VALUE
#+end_example

Surprisingly enough, the generated opcodes for defining the functions
are exactly the same. Looking inside the functions themselves also
gives the same result.

#+begin_src python :results output :exports both :session hello
import dis

async def async_hello_world():
    print("Hello, world!")

def hello_world():
    print("Hello, world!")

print("Async")
dis.dis(async_hello_world)

print("Function")
dis.dis(hello_world)
#+end_src

#+RESULTS:
#+begin_example
Async
  4           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 ('Hello, world!')
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE
Function
  7           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 ('Hello, world!')
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE
#+end_example

The only difference is a flag set on the defined code: CO_COROUTINE
which makers a function as a coroutine. The actual compiler code is
somewhere around here; but it's worth explicitly looking at the flags.

Once more into the breach:
#+begin_src python :results output :exports both 
import dis

async def async_hello_world():
    print("Hello, world!")

def hello_world():
    print("Hello, world!")

async_flags = async_hello_world.__code__.co_flags
standard_flags = hello_world.__code__.co_flags

only_async_flags = async_flags & (~standard_flags)
print(f"{only_async_flags=}, aka {dis.COMPILER_FLAG_NAMES[only_async_flags]}")

only_standard_flags = standard_flags & (~async_flags)
print(f"{only_standard_flags=}")
#+end_src

#+RESULTS:
: only_async_flags=128, aka COROUTINE
: only_standard_flags=0

And that's the literally the magic bit that makes the difference in
how coroutines are evaluated.

[fn:functions] Just for contrast:
#+begin_src python :results output :exports both 
import ast
import pprint

pprint.pprint(ast.dump(ast.parse("""
def hello_world():
    print("Hello, world!")
""")))
#+end_src

#+RESULTS:
: ("Module(body=[FunctionDef(name='hello_world', args=arguments(posonlyargs=[], "
:  'args=[], vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None, '
:  "defaults=[]), body=[Expr(value=Call(func=Name(id='print', ctx=Load()), "
:  "args=[Constant(value='Hello, world!', kind=None)], keywords=[]))], "
:  'decorator_list=[], returns=None, type_comment=None)], type_ignores=[])')

Normal functions are parsed, unsurprisingly, as FunctionDef.

*** Execution
Executing an async function doesn't directly run the inner function anymore,
and instead returns a a coroutine object that wraps the computation.

#+begin_src python :results output :exports both :session hello
hello_world()
#+end_src

#+RESULTS:
: Hello, world!