BUG: pip install scipy from git fails with `"openblas" not found`

See original GitHub issue

Describe your issue.

This comes from SymPy CI: https://github.com/sympy/sympy/pull/23513

A few days ago it successfully built and tested NumPy and SciPy on Python 3.11b1 from git using pip but now the same setup fails to build SciPy with

      ../../scipy/meson.build:120:0: ERROR: Dependency "openblas" not found, tried pkgconfig and cmake

Maybe this would work with apt install libopenblas-dev liblapack-dev but it was working before so should it be expected to work still?

This was the last successful run (the build was successful although a test later failed): https://github.com/sympy/sympy/runs/6598088659?check_suite_focus=true

Reproducing Code Example

# Ubuntu 20.04.4 with Python 3.11b1 running in GA
sudo apt install antlr4 libgfortran5 gfortran libmpfr-dev libmpc-dev
pip install --upgrade pip wheel
pip install git+https://github.com/cython/cython@master
pip install git+https://github.com/numpy/numpy@main
pip install git+https://github.com/scipy/scipy@main

Error message

Run pip install git+https://github.com/numpy/numpy@main
  pip install git+https://github.com/numpy/numpy@main
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.11.0-beta.1/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.0-beta.1/x64/lib
Collecting git+https://github.com/numpy/numpy@main
  Cloning https://github.com/numpy/numpy (to revision main) to /tmp/pip-req-build-a992dsfx
  Running command git clone --filter=blob:none --quiet https://github.com/numpy/numpy /tmp/pip-req-build-a992dsfx
  Resolved https://github.com/numpy/numpy to commit 68f3a12f4cc9310a7d60f9878af72cf98e7233f3
  Running command git submodule update --init --recursive -q
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: numpy
  Building wheel for numpy (pyproject.toml): started
  Building wheel for numpy (pyproject.toml): still running...
  Building wheel for numpy (pyproject.toml): still running...
  Building wheel for numpy (pyproject.toml): still running...
  Building wheel for numpy (pyproject.toml): still running...
  Building wheel for numpy (pyproject.toml): finished with status 'done'
  Created wheel for numpy: filename=numpy-1.24.0.dev0+75.g68f3a12f4-cp311-cp311-linux_x86_64.whl size=22029574 sha256=ed58cdbe5b547632f5a45e8ab48e0c200b8f247a91e8e945c457a7899185724a
  Stored in directory: /tmp/pip-ephem-wheel-cache-jo9owv4z/wheels/ad/e8/55/3b4372ddf4700df1fb35dc513aacd83756396e870c5a6f8ece
Successfully built numpy
Installing collected packages: numpy
Successfully installed numpy-1.24.0.dev0+75.g68f3a12f4

Run pip install git+https://github.com/scipy/scipy@main
Collecting git+https://github.com/scipy/scipy@main
  Cloning https://github.com/scipy/scipy (to revision main) to /tmp/pip-req-build-b76e9_br
  Running command git clone --filter=blob:none --quiet https://github.com/scipy/scipy /tmp/pip-req-build-b76e9_br
  Resolved https://github.com/scipy/scipy to commit 6c51ed7aa4230d9fc08d099773b66f61e9828e97
  Running command git submodule update --init --recursive -q
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [72 lines of output]
      The Meson build system
      Version: 0.62.1
      Source dir: /tmp/pip-req-build-b76e9_br
      Build dir: /tmp/pip-req-build-b76e9_br/.mesonpy-_wsc8qd3/build
      Build type: native build
      Project name: SciPy
      Project version: 1.9.0.dev0
      C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
      C linker for the host machine: cc ld.bfd 2.34
      C++ compiler for the host machine: c++ (gcc 9.4.0 "c++ (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
      C++ linker for the host machine: c++ ld.bfd 2.34
      Host machine cpu family: x86_64
      Host machine cpu: x86_64
      Compiler for C supports arguments -Wno-unused-but-set-variable: YES
      Library m found: YES
      Fortran compiler for the host machine: gfortran (gcc 9.4.0 "GNU Fortran (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
      Fortran linker for the host machine: gfortran ld.bfd 2.34
      Program cython found: YES (/tmp/pip-build-env-4bh3e24n/overlay/bin/cython)
      Program pythran found: YES (/tmp/pip-build-env-4bh3e24n/overlay/bin/pythran)
      Program cp found: YES (/usr/bin/cp)
      Program python3 found: YES (/opt/hostedtoolcache/Python/3.11.0-beta.1/x64/bin/python)
      Found pkg-config: /usr/bin/pkg-config (0.29.1)
      Library npymath found: YES
      Library npyrandom found: YES
      Found CMake: /usr/local/bin/cmake (3.23.1)
      Run-time dependency openblas found: NO (tried pkgconfig and cmake)
      
      ../../scipy/meson.build:120:0: ERROR: Dependency "openblas" not found, tried pkgconfig and cmake
      
      A full log can be found at /tmp/pip-req-build-b76e9_br/.mesonpy-_wsc8qd3/build/meson-logs/meson-log.txt
      + meson setup --native-file=/tmp/pip-req-build-b76e9_br/.mesonpy-native-file.ini -Ddebug=false -Dstrip=true -Doptimization=2 --prefix=/opt/hostedtoolcache/Python/3.11.0-beta.1/x64 /tmp/pip-req-build-b76e9_br /tmp/pip-req-build-b76e9_br/.mesonpy-_wsc8qd3/build
      Traceback (most recent call last):
        File "/opt/hostedtoolcache/Python/3.11.0-beta.1/x64/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
          ^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.0-beta.1/x64/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.0-beta.1/x64/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4bh3e24n/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 805, in get_requires_for_build_wheel
          with _project(config_settings) as project:
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.0-beta.1/x64/lib/python3.11/contextlib.py", line 137, in __enter__
          return next(self.gen)
                 ^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4bh3e24n/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 784, in _project
          with Project.with_temp_working_dir(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.0-beta.1/x64/lib/python3.11/contextlib.py", line 137, in __enter__
          return next(self.gen)
                 ^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4bh3e24n/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 479, in with_temp_working_dir
          yield cls(source_dir, tmpdir, build_dir)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4bh3e24n/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 423, in __init__
          self._configure(reconfigure=bool(build_dir) and not native_file_mismatch)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4bh3e24n/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 450, in _configure
          self._meson(
          ^^^^^^^^^^^^
        File "/tmp/pip-build-env-4bh3e24n/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 438, in _meson
          return self._proc('meson', *args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-4bh3e24n/overlay/lib/python3.11/site-packages/mesonpy/__init__.py", line 434, in _proc
          subprocess.check_call(list(args))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/opt/hostedtoolcache/Python/3.11.0-beta.1/x64/lib/python3.11/subprocess.py", line 402, in check_call
          raise CalledProcessError(retcode, cmd)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      subprocess.CalledProcessError: Command '['meson', 'setup', '--native-file=/tmp/pip-req-build-b76e9_br/.mesonpy-native-file.ini', '-Ddebug=false', '-Dstrip=true', '-Doptimization=2', '--prefix=/opt/hostedtoolcache/Python/3.11.0-beta.1/x64', '/tmp/pip-req-build-b76e9_br', '/tmp/pip-req-build-b76e9_br/.mesonpy-_wsc8qd3/build']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Error: Process completed with exit code 1.

SciPy/NumPy/Python version information

scipy/numpy from git and python 3.11b1

Issue Analytics

  • State:open
  • Created a year ago
  • Reactions:2
  • Comments:53 (25 by maintainers)

github_iconTop GitHub Comments

3reactions
eli-schwartzcommented, May 29, 2022

But also this is one of the items at #16293 and the goal is to teach Meson some of these fallback detection methods. 😃

1reaction
wleperacommented, Dec 6, 2022

@rgommers, I added my own openblas.pc based on your template sample, set the PKG_CONFIG_PATH, and was able to build scipy successfully - and got my TVM build and install complete as well. Thank you for the assist!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Impossible to install scipy (libraries openblas, lapack not found)
I've read that installing from a wheel could be the way, but no wheel of scipy for python 3.10 (found there) is supported...
Read more >
Unable to install Scipy with MKL using Meson - Stack Overflow
I am trying to run the installation on Cygwin and it seems the issue is that openblas-devel just doesn't exist. – Severus Snape....
Read more >
Building from source on Windows — SciPy v1.7.0 Manual
First, we need to install the software required to build OpenBLAS, which is the BLAS ... In turn, if an error about the...
Read more >
scipy does not install - Google Groups
Running setup.py install for scipy: finished with status 'error'. Complete output from command C:\Users\Namai\.thonny\BundledPython36\Scripts\python.exe -u ...
Read more >
How to build SciPy with Meson
Building SciPy on Windows has always been a little tricky - it does work provided that all build dependencies are installed (most importantly,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found