How to reuse session and client in a more complex app ?

• Async AWS SDK for Python version: 9.0.0
• Python version: 3.7
• Operating System: Lambda (Linux)

Description

I would like to reuse session or client in multiple concurrent calls to S3 and SQS. This is in order to have faster execution of the code. If I build the session and client on each call then it takes too much time. I tried creating the client globally in the file where all aws functions are. I also tried creating it in the main file and passing it as a parameter to the function that would use it.

What I Did

Here is a minimalistic code of my app. The complete code will span in multiple files and functions. All aws functions are in separate file from the logic that creates the loops and lists of tasks needed to be run (so I cannot create all my logic under one async with aioboto3.Session().client('s3') as s3: .

import asyncio
from io import BytesIO
import aioboto3

s3_client = aioboto3.Session().client('s3')

def lambda_handler(a,b):
    asyncio.run(main())


async def main():
    filelike1 = BytesIO()
    filelike2 = BytesIO()
    await asyncio.wait([s3_get(filelike1),s3_get(filelike2)])
    # later or maybe in some other function
    await asyncio.wait([s3_get(filelike3),s3_get(filelike3)])


async def s3_get(filelike):
    async with s3_client as s3:  # to make it work here i create new session and client async with aioboto3.Session().client('s3') as s3:
        return await s3.download_fileobj('s3-files-003', '.versions.txt', filelike)

future: <Task finished coro=<s3_get() done, defined at /var/task/lambda_function.py:19> exception=RuntimeError('cannot reuse already awaited coroutine')>
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 20, in s3_get
    async with s3_client as s3:
  File "/opt/python/lib/python3.7/site-packages/aiobotocore/session.py", line 37, in __aenter__
    self._client = await self._coro
RuntimeError: cannot reuse already awaited coroutine