support moto client wrappers

This bug arises in pytest with moto 1.3.14 and althoughrequirements-dev.txt has a dev-version, that fix is for something else, i.e. this is irrelevant:

# We need: https://github.com/spulec/moto/pull/2436
moto==1.3.14.dev326

See also:

• https://github.com/spulec/moto/issues/2706
• https://github.com/boto/botocore/blob/develop/botocore/endpoint.py#L38
• https://github.com/boto/botocore/blob/develop/botocore/parsers.py#L257
  • it uses a lowercase_dict function already

Below is an exception detail, when testing the following pytest fixtures:

from moto import mock_config
from moto import mock_batch

@pytest.fixture(scope="module")
def aws_region():
    return "us-west-2"

@pytest.fixture
@pytest.mark.moto
def aio_aws_session(event_loop):
    with mock_config():
        aws_session = aiobotocore.get_session(loop=event_loop)
        yield aws_session

@pytest.fixture
@pytest.mark.moto
async def aio_aws_batch_client(aio_aws_session, aws_region):
    with mock_config():
        with mock_batch():
            async with aio_aws_session.create_client("batch", region_name=aws_region) as client:
                yield client

This raises a simple exception when trying to parse a moto response (below) and the source code for botocore seems to match (there is no AWSResponse.raw_headers attr). Maybe there are API version differences between aiobotocore, botocore and moto (at the time of posting this issue). In the project, the requirements pull in the aiobotocore deps for boto3/botocore and moto is the latest release:

aiobotocore==0.11.1
boto==2.49.0
boto3==1.10.14
botocore==1.13.14
moto==1.3.14

$ python --version
Python 3.6.7
$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.4 LTS"

The simple test function is:

@pytest.mark.asyncio
async def test_async_aws_batch_client(aio_aws_batch_client):
    assert isinstance(aio_aws_batch_client, BaseClient)
    job_queues = await aio_aws_batch_client.describe_job_queues()
    # AttributeError: 'AWSResponse' object has no attribute 'raw_headers'

The moto job-queues should be an empty list (and it is, see pdb details below).

>       job_queues = await aio_aws_batch_client.describe_job_queues()

tests/aws/test_async_aws_batch.py:56: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/opt/conda/envs/python-notes/lib/python3.6/site-packages/aiobotocore/client.py:89: in _make_api_call
    operation_model, request_dict, request_context)
/opt/conda/envs/python-notes/lib/python3.6/site-packages/aiobotocore/client.py:110: in _make_request
    request_dict)
/opt/conda/envs/python-notes/lib/python3.6/site-packages/aiobotocore/endpoint.py:73: in _send_request
    request, operation_model, context)
/opt/conda/envs/python-notes/lib/python3.6/site-packages/aiobotocore/endpoint.py:106: in _get_response
    request, operation_model)
/opt/conda/envs/python-notes/lib/python3.6/site-packages/aiobotocore/endpoint.py:154: in _do_get_response
    operation_model)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

http_response = <botocore.awsrequest.AWSResponse object at 0x7eff6ebdc6d8>, operation_model = OperationModel(name=DescribeJobQueues)

    async def convert_to_response_dict(http_response, operation_model):
        """Convert an HTTP response object to a request dict.
    
        This converts the requests library's HTTP response object to
        a dictionary.
    
        :type http_response: botocore.vendored.requests.model.Response
        :param http_response: The HTTP response from an AWS service request.
    
        :rtype: dict
        :return: A response dictionary which will contain the following keys:
            * headers (dict)
            * status_code (int)
            * body (string or file-like object)
    
        """
        response_dict = {
            # botocore converts keys to str, so make sure that they are in
            # the expected case. See detailed discussion here:
            # https://github.com/aio-libs/aiobotocore/pull/116
            # aiohttp's CIMultiDict camel cases the headers :(
            'headers': HTTPHeaderDict(
                {k.decode('utf-8').lower(): v.decode('utf-8')
>                for k, v in http_response.raw_headers}),
            'status_code': http_response.status_code,
            'context': {
                'operation_name': operation_model.name,
            }
        }
E       AttributeError: 'AWSResponse' object has no attribute 'raw_headers'

/opt/conda/envs/python-notes/lib/python3.6/site-packages/aiobotocore/endpoint.py:43: AttributeError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /opt/conda/envs/python-notes/lib/python3.6/site-packages/aiobotocore/endpoint.py(43)convert_to_response_dict()
-> for k, v in http_response.raw_headers}),

(Pdb) http_response
<botocore.awsrequest.AWSResponse object at 0x7fed5d7c62b0>
(Pdb) dir(http_response)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_content', 'content', 'headers', 'raw', 'status_code', 'text', 'url']

(Pdb) http_response.headers
{'server': 'amazon.com'}
(Pdb) http_response.content
b'{"jobQueues": []}'
(Pdb) http_response.status_code
200
(Pdb) http_response.text
'{"jobQueues": []}'
(Pdb) http_response.url
'https://batch.us-west-2.amazonaws.com/v1/describejobqueues'

(Pdb) http_response.raw
<moto.core.models.MockRawResponse object at 0x7eff6ed909e8>
(Pdb) dir(http_response.raw)
['__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '_checkClosed', '_checkReadable', '_checkSeekable', '_checkWritable', 'close', 'closed', 'detach', 'fileno', 'flush', 'getbuffer', 'getvalue', 'isatty', 'read', 'read1', 'readable', 'readinto', 'readinto1', 'readline', 'readlines', 'seek', 'seekable', 'stream', 'tell', 'truncate', 'writable', 'write', 'writelines']
(Pdb) http_response.raw.readlines()
[]

Note that the moto response is an botocore.awsrequest.AWSResponse and not a

• :type http_response: botocore.vendored.requests.model.Response