RuntimeError: cannot cache function 'square': no locator available for file

When trying to import a numba-cached function from a python egg package, I get the following error message about no cache file locator being found:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/home/username/.local/lib/python3.6/site-packages/mytestpackage-1.2.3-py3.6.egg/mytestpackage/mymodule.py", line 3, in <module>
  File "/home/username/.local/lib/python3.6/site-packages/numba/decorators.py", line 194, in wrapper
    disp.enable_caching()
  File "/home/username/.local/lib/python3.6/site-packages/numba/dispatcher.py", line 679, in enable_caching
    self._cache = FunctionCache(self.py_func)
  File "/home/username/.local/lib/python3.6/site-packages/numba/caching.py", line 614, in __init__
    self._impl = self._impl_class(py_func)
  File "/home/username/.local/lib/python3.6/site-packages/numba/caching.py", line 349, in __init__
    "for file %r" % (qualname, source_path))
RuntimeError: cannot cache function 'square': no locator available for file '/home/username/.local/lib/python3.6/site-packages/mytestpackage-1.2.3-py3.6.egg/mytestpackage/mymodule.py'

The bug occurs at least with numba versions 0.45.1 and 0.46.1. The bug appears to be operating system-agnostic. Exporting a user cache directory does not help: export NUMBA_CACHE_DIR=/tmp. I believe that the various caching file locators fail because they assume that the python files are located in plain directories, when they can in fact be in an egg, for example here:

def from_function(cls, py_func, py_file):
    if not (os.path.exists(py_file) or getattr(sys, 'frozen', False)):
        # Perhaps a placeholder (e.g. "<ipython-XXX>")
        # stop function exit if frozen, since it uses a temp placeholder
        return
    self = cls(py_func, py_file)
    try:
        self.ensure_cache_path()
    except OSError:
        # Cannot ensure the cache directory exists or is writable
        return
    return self

To produce a minimal package and reproduce this bug, run this code in a shell:

echo "UEsDBBQDAAAAADd9fU8AAAAAAAAAAAAAAAAOAAAAbXl0ZXN0cGFja2FnZS9QSwMEFAMAAAAAvXx9TwAAAAAAAAAAAAAAABwAAABteXRlc3RwYWNrYWdlL215dGVzdHBhY2thZ2UvUEsDBAoDAAAAAMVqfU9D34g0AgAAAAIAAAAnAAAAbXl0ZXN0cGFja2FnZS9teXRlc3RwYWNrYWdlL19faW5pdF9fLnB5IApQSwMEFAMAAAgAuXx9TxKRbY9SAAAAUwAAACcAAABteXRlc3RwYWNrYWdlL215dGVzdHBhY2thZ2UvbXltb2R1bGUucHkdxbEKgCAQAND9vuJo0ohmEYI+oi0azE4y0OrywM8vesuL6Tq5YJa0OoDxv89HLKoJRgUz20U3HXrndxomFtKwUcDnFsekqraAH6YinLFiixVeUEsDBBQDAAAIALB8fU9A2V6/YwAAAI0AAAAWAAAAbXl0ZXN0cGFja2FnZS9zZXR1cC5weVWMSwqFMAxF51lF6EihCO857lqkaJSibUoTBXev+Bl4h+dcToiZiyILjIUjCumalXkRvM1NLI4hDV32/ewnEoCLVoDnko/kTNyVRJ+DsZfZqEjg5Myv+TftA9+G+xSr2kINB1BLAQI/AxQDAAAAADd9fU8AAAAAAAAAAAAAAAAOACQAAAAAAAAAEIDtQQAAAABteXRlc3RwYWNrYWdlLwoAIAAAAAAAAQAYAICaOR/DptUBgPSbIcOm1QGAmjkfw6bVAVBLAQI/AxQDAAAAAL18fU8AAAAAAAAAAAAAAAAcACQAAAAAAAAAEIDtQSwAAABteXRlc3RwYWNrYWdlL215dGVzdHBhY2thZ2UvCgAgAAAAAAABABgAgJBTl8Km1QEAizQiw6bVAYCQU5fCptUBUEsBAj8DCgMAAAAAxWp9T0PfiDQCAAAAAgAAACcAJAAAAAAAAAAggKSBZgAAAG15dGVzdHBhY2thZ2UvbXl0ZXN0cGFja2FnZS9fX2luaXRfXy5weQoAIAAAAAAAAQAYAIBuvZ6vptUBAIs0IsOm1QGAbr2er6bVAVBLAQI/AxQDAAAIALl8fU8SkW2PUgAAAFMAAAAnACQAAAAAAAAAIICkga0AAABteXRlc3RwYWNrYWdlL215dGVzdHBhY2thZ2UvbXltb2R1bGUucHkKACAAAAAAAAEAGACA3I6SwqbVAQCLNCLDptUBgNyOksKm1QFQSwECPwMUAwAACACwfH1PQNlev2MAAACNAAAAFgAkAAAAAAAAACCApIFEAQAAbXl0ZXN0cGFja2FnZS9zZXR1cC5weQoAIAAAAAAAAQAYAADebIjCptUBgJVG28Km1QEA3myIwqbVAVBLBQYAAAAABQAFACgCAADbAQAAAAA=" | base64 -d > mytestpackage.zip
unzip mytestpackage.zip
cd mytestpackage
python setup.py install --user
cd /tmp
python -c "import mytestpackage.mymodule"

Or alternatively, if you do not trust this base64-encoded zip-file, you can create a package manually. The directory structure is:

mytestpackage
├── mytestpackage
│   ├── __init__.py
│   └── mymodule.py
└── setup.py

The content of the file mymodule.py is:

import numba

@numba.njit("f8(f8)", cache=True)
def square(x):
    return x * x

The content of the file setup.py is:

from setuptools import setup, find_packages

setup(
    name="mytestpackage",
    version="1.2.3",
    packages=find_packages(),
)

The file __init__.py is empty.

Often it might be preferable to compile the functions ahead of time during setup with numba.pycc if the required features are available (e.g. no parallel=True (yet)).

Another workaround is to install the package with pip install ., which will not create an egg, instead of python setup.py install.