Bug report

🐛

What’s wrong

Trying to use django-stubs together with django-configurations is a big pain since the latter interferes and controls how django settings are loaded. By default, the following exception gets thrown when trying to run mypy:

(mypackage-vfeg9Sk1-py3.6) ➜  mypackage IN-3486 ✗ python -m mypy -p src.mypackage                                                       
Error constructing plugin instance of NewSemanalDjangoPlugin

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/mypy/__main__.py", line 12, in <module>
    main(None, sys.stdout, sys.stderr)
  File "mypy/main.py", line 89, in main
  File "mypy/build.py", line 180, in build
  File "mypy/build.py", line 227, in _build
  File "mypy/build.py", line 465, in load_plugins
  File "mypy/build.py", line 443, in load_plugins_from_config
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/mypy_django_plugin/main.py", line 80, in __init__
    self.django_context = DjangoContext(django_settings_module)
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/mypy_django_plugin/django/context.py", line 88, in __init__
    apps, settings = initialize_django(self.django_settings_module)
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/mypy_django_plugin/django/context.py", line 70, in initialize_django
    settings._setup()
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/django/conf/__init__.py", line 66, in _setup
    self._wrapped = Settings(settings_module)
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/django/conf/__init__.py", line 157, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/sarunas/Documents/projects/mypackage/mypackage/project/settings.py", line 6, in <module>
    from mypackage.settings import Base, IntegratedTests
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/mypackage/settings.py", line 26, in <module>
    class Base(Configuration):  # pylint: disable=no-init
  File "/home/sarunas/.cache/pypoetry/virtualenvs/mypackage-vfeg9Sk1-py3.6/lib/python3.6/site-packages/configurations/base.py", line 28, in __new__
    raise ImproperlyConfigured(install_failure)
django.core.exceptions.ImproperlyConfigured: django-configurations settings importer wasn't correctly installed. Please use one of the starter functions to install it as mentioned in the docs: https://django-configurations.readthedocs.io/

That makes sense, since configurations disallows loading the settings manually and requires to run these two calls to configure django, both of them are for example found in manage.py and wsgi.py:

        import configurations
        configurations.setup()

mypy configuration in setup.cfg (it follows [metadata] and [options] package metadata definitions)

[mypy.plugins.django-stubs]
django_settings_module = project.settings

[mypy]
plugins =
    mypy_django_plugin.main

ignore_errors =            false
warn_no_return =           true
strict_equality =          true
strict_optional =          true
warn_return_any =          true
warn_unreachable =         true
implicit_reexport =        false
check_untyped_defs =       true
local_partial_types =      true
warn_unused_ignores =      true
warn_unused_configs =      true
warn_redundant_casts =     true
no_implicit_optional =     true

namespace_packages =       true
fine_grained_incremental = true
show_error_codes =         true

strict =                   true
allow_any_generics =       false
allow_subclassing_any =    true
ignore_missing_imports =   false

[mypy-mypackage.migrations.*]
ignore_errors = true

I tried looking for a workaround which didn’t involve changing neither mypy nor django-stubs, nor django-configurations source files and I’m afraid I have failed. Unfortunately this also blocks DRF stubs from being useful.

How is that should be

django-configurations should be supported and allowed to load django settings if the user uses it (unless it is somehow incompatible with the way the stubs are generated?).

System information

• OS: “Ubuntu 18.04.4 LTS”
• python version: 3.6.9
• django version: 2.2.14
• mypy version: 0.770
• django-stubs version: 1.5.0
• django-configurations version: 2.2

Possible solution

One of possible options would be to include those two calls in mypy_django_plugin.django.context initialize_django call, just before the models get imported:

def initialize_django(settings_module: str) -> Tuple['Apps', 'LazySettings']:
    with temp_environ():
        os.environ['DJANGO_SETTINGS_MODULE'] = settings_module

        # add current directory to sys.path
        sys.path.append(os.getcwd())

        def noop_class_getitem(cls, key):
            return cls
        
        try:
             # or use importlib.import_module, or check whether django uses it
            import configurations
            configurations.setup()
        except ModuleNotFoundError:
            pass

        from django.db import models

        models.QuerySet.__class_getitem__ = classmethod(noop_class_getitem)  # type: ignore
        models.Manager.__class_getitem__ = classmethod(noop_class_getitem)  # type: ignore
        ...

I found that this works for me - though I haven’t tested it extensively yet. I’d be happy to add a PR for this upon your confirmation.