readthedocs build failure: cannot cache function: no locator available for file

See original GitHub issue

My project uses numba 0.43.1 (from conda-forge) and has functions decorated with @numba.jit('<signature>', nopython=True, nogil=True, cache=True). Everything works fine locally, on Travis, and AppVeyor. readthedocs fails as soon as it tries importing the modules, with the following log:

WARNING: autodoc: failed to import module 'copula' from module 'pyscenarios'; the following exception was raised:
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 232, in import_module
    __import__(modname)
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
  File "/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/pyscenarios-0.2.0-py3.7.egg/pyscenarios/copula.py", line 12, in <module>
    from .sobol import sobol
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 668, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 638, in _load_backward_compatible
  File "/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/pyscenarios-0.2.0-py3.7.egg/pyscenarios/sobol.py", line 67, in <module>
    def _calc_v_kernel(directions: np.ndarray) -> np.ndarray:
  File "/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/numba/decorators.py", line 191, in wrapper
    disp.enable_caching()
  File "/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/numba/dispatcher.py", line 566, in enable_caching
    self._cache = FunctionCache(self.py_func)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/numba/caching.py", line 614, in __init__
    self._impl = self._impl_class(py_func)
  File "/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/numba/caching.py", line 349, in __init__
    "for file %r" % (qualname, source_path))
RuntimeError: cannot cache function '_calc_v_kernel': no locator available for file '/home/docs/checkouts/readthedocs.org/user_builds/pyscenarios/conda/v0.2.0/lib/python3.7/site-packages/pyscenarios-0.2.0-py3.7.egg/pyscenarios/sobol.py'

I’ve worked around the problem with the following hack:

def jit(sig: str):
    def decorator(func):
        try:
            return numba.jit(sig, nopython=True, nogil=True, cache=True)(func)
        except RuntimeError:
            # Work around issues with permissions on readthedocs
            return numba.jit(sig, nopython=True, nogil=True, cache=False)(func)
    return decorator

Issue Analytics

State:
Created 4 years ago
Comments:20 (8 by maintainers)

Top GitHub Comments

9reactions

bjarthurcommented, Nov 18, 2020

@SumNeuron thanks for the workaround. i found though that NUMBA_CACHE_DIR was not persisted to the container because it was in a RUN command. so i put it in an ENV and it worked beautifully. also found that i did not have to actualy create the numba_cache directory either, so i deleted the rest of the RUN command. so i ended up simply with:

ENV NUMBA_CACHE_DIR=/tmp/numba_cache

5reactions

SumNeuroncommented, Oct 28, 2019

@esc that may be… but I have other .env vars in that file that do get evaluated. I can double check this tomorrow.

I can add a layer to the Dockerfile:

RUN mkdir /tmp/numba_cache & chmod 777 /tmp/numba_cache & NUMBA_CACHE_DIR=/tmp/numba_cache

I am still of the opinion that in the next version, this is added as fallback by numba - with a warning readout. This will prevent individuals from having to rebuild dockerfiles just to make sure a cache dir is set up. Yes it is a quality of life feature and not strictly required, but I would sure appreciate it ^•^

Top Results From Across the Web

1 Answer - 1 - Stack Overflow

But getting the following error. RuntimeError at / cannot cache function '__shear_dense': no locator available for file ...

Notes on Caching — Numba 0.50.1 documentation

The cache is invalidated when the corresponding source file is modified. However, it is necessary sometimes to clear the cache directory manually. For...

Read the Docs Documentation

Read the Docs simplifies software documentation by building, versioning, and hosting of your docs, automatically.

fsspec Documentation - Read the Docs

You can use fsspec's file objects with any python function that accepts file ... We have been involved in building a number of...

PyInstaller Documentation - Read the Docs

Some Python scripts import modules in ways that PyInstaller cannot detect: for ... Clean PyInstaller cache and remove temporary files before building.