Asyncio: Queue consumer gets out of while loop without break. Where exactly and how?

[saavedra29]

I have the following code:

import asyncio

PARALLEL_DOWNLOADS = 4
names = ['Aris', 'Kostas', 'Nikos', 'Makis', 'Christos', 'Andreas', 'Vasilis', 'Thanasis', 'Petros', 'Pavlos']

async def say_hello(q):
    while True:
        name = await q.get()
        print('Hello {}'.format(name))
        q.task_done()
        print('After task done')
        
async def main():
    q = asyncio.Queue()

    for name in names:
        q.put_nowait(name)

    workers = []
    for _ in range(PARALLEL_DOWNLOADS):
        worker = asyncio.create_task(say_hello(q))
        workers.append(worker)

    await q.join()
    for worker in workers:
        worker.cancel()
    await asyncio.gather(*workers, return_exceptions=True)
    
asyncio.run(main())

Here when all the queue data has been consumed the while loop in the say_hello function breaks at the end of the loop. And this without any break, continue or return statement! I would say "automagically".
OK, this is the wanted result for me but i can't understand how asyncio does this. At which exact point of the code the while loop "breaks", or yields or whatever?
Maybe at "name = await q.get()" it sees that there is nothing else in the queue and yields (or returns) from the say_hello coroutine function?

[deanhystad]

An exception is raised by "name = await q.get()". It is a very quiet exception, but it kicks you out of say_hello().

If I remove return_exceptions=True from this line the exception gets a lot less quiet. :

    await asyncio.gather(*workers)

Error:
Traceback (most recent call last):
  File "C:\Python\Python39\lib\asyncio\queues.py", line 166, in get
    await getter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "myprogram", line 23, in main
    await asyncio.gather(*workers)
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "myprogram", line 25, in <module>
    asyncio.run(main())
  File "C:\Python\Python39\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Python\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
asyncio.exceptions.CancelledError

exception asyncio.CancelledError
The operation has been cancelled.

This exception can be caught to perform custom operations when asyncio Tasks are cancelled. In almost all situations the exception must be re-raised.

Here I will catch the cancelled error so I can perform some important cleanup.

import asyncio
names = ['Aris', 'Kostas', 'Nikos', 'Makis', 'Christos', 'Andreas', 'Vasilis', 'Thanasis', 'Petros', 'Pavlos']

async def say_hello(id, q):
    while True:
        try:
            name = await q.get()
            print(f'{id} {name}')
            await asyncio.sleep(0.1)
            q.task_done()
        except asyncio.CancelledError as msg:
            print(msg)
            break
    print(f"{id} cancelled")

async def main():
    q = asyncio.Queue()

    for name in names:
        q.put_nowait(name)

    workers = [asyncio.create_task(say_hello(i, q)) for i in range(4)]

    await q.join()
    for worker in workers:
        worker.cancel()

    await asyncio.gather(*workers)

asyncio.run(main())

Output:
0 Aris
1 Kostas
2 Nikos 
3 Makis 
0 Christos
2 Andreas 
1 Vasilis 
3 Thanasis
0 Petros
1 Pavlos

0 cancelled

1 cancelled

2 cancelled

3 cancelled

Wow, cancelled without even leaving behind a note. That's cold.

[saavedra29]

Oh!!! Thank you very much! Very good explaination for such a complex module
So the eventloop breaks and this raises the CancelledError in the coroutine! Thak you!