[Help] Failed to resolve requirements from PEX environment
See original GitHub issueI am trying to build a .pex file and then have it run on an EMR cluster. I am getting the following error: Failed to resolve requirements from PEX environment
I am creating a .pex file using the following commands;
export virtual_environment="my_application_env"
python3.7 -m venv $virtual_environment --copies
source $virtual_environment/bin/activate
pip install -U pip
pip download -r requirements.txt \
--dest ./build/wheelhouse
pip install pex
pex . -r requirements.txt \
-o ./$virtual_environment.pex \
--no-index \
-f ./build/wheelhouse
rm -rf ./build/wheelhouse
deactivate
Full stack trace:
PEXWarning: PEX_ROOT is configured as /home/.pex but that path is un-writeable, falling back to a temporary P
EX_ROOT of /tmp/tmp_vltgti0 which will hurt performance.
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /mnt/yarn/usercache/hdfs/filecache/1/24059176/connect_cl
oud_base_dq_env.pex.
Needed manylinux_2_26_aarch64-cp-37-cp37m compatible dependencies for:
1: MarkupSafe>=0.23
Required by:
Jinja2 2.11.3
But this pex had no 'MarkupSafe' distributions.
2: pyrsistent>=0.14.0
Required by:
jsonschema 3.2.0
But this pex had no 'pyrsistent' distributions.
3: numpy
Required by:
altair 4.1.0
But this pex had no 'numpy' distributions.
4: pandas>=0.18
Required by:
altair 4.1.0
But this pex had no 'pandas' distributions.
5: pyzmq>=13
Required by:
jupyter-client 6.2.0
But this pex had no 'pyzmq' distributions.
6: tornado>=4.1
Issue Analytics
- State:
- Created 2 years ago
- Comments:10 (7 by maintainers)
Top Results From Across the Web
PEX_PATH transitivity · Issue #1423 · pantsbuild/pex - GitHub
ResolveError: Failed to resolve requirements from PEX environment ... The change in Pex to support creating a venv by using symlinks solves any...
Read more >pyyaml dependency error while executing with python script in ...
I resolved it by changing env to 3.9 pex requests 'XlsxWriter==1.3.7' 'urllib3==1.26.2' 'PyYAML==5.3.1' ./app -o app.pex --python-shebang '# ...
Read more >Building .pex files — pex 2.1.119 documentation
This command creates a pex file containing pex and requests, using the console script named “pex”, saving it in ~/bin/pex. At this point,...
Read more >pex Changelog - pyup.io
This release fixes resolving requirements from a lock using arbitrary equality (``===``). In addition, you can now "inject" runtime environment variables ...
Read more >Third-party dependencies - Pants build system
dependencies field, Pants will error with a helpful message when building your code with goals like ... Failed to resolve requirements from PEX...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Oh, I see you’ve already found the correct target host platform string. Yeah, so you just need to build the PEX file with:
Aha - yes - exactly. Pex has no magic up its sleeves. To build for arm on an intel host you’ll need two things:
For 1, you can find out the correct platform string by running
PEX_TOOLS=1 ./my_application_env.pex interpreter -von the target host (you’ll need to have built the PEX file with--include-tools. You’ll get something like:You want to use the “platform” value as the argument to
--platformwhen building your PEX.For 2, you can use switches similar to those you already have, like --no-index and --find-links pointing to the pre-built wheelhouse.