Feast init: ImportError: cannot import name '_message' from 'google.protobuf.pyext'

After running the command feast init feature-repo-name, I received an error

ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/opt/homebrew/Caskroom/miniforge/base/envs/intel_py39/lib/python3.9/site-packages/google/protobuf/pyext/__init__.py)

Here is what I have tried but none was working as I received the exact same error

Reinstall protobuf to the latest version using

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
pip3 install --upgrade --force-reinstall protobuf

Switch to a Mac with an Intel chip and run everything all over again

Specifications

Version: feast 0.14.1
Platform:
- macOS Big Sur (version 11.6) (Apple M1 chip)
- python 3.9.7
Dependencies:
- protobuf 3.19.1
- google-api-core 2.2.1
- googleapis-common-protos 1.52.0
- google-auth 2.3.2

Issue Analytics

State:
Created 2 years ago
Comments:7

Top GitHub Comments

4reactions

TeoZosacommented, Nov 10, 2021

I also ran into the same issue but was able to get it working. My solution (originally posted in https://github.com/protocolbuffers/protobuf/issues/8820#issuecomment-961552604):

I had the same problem and discovered it was because required shared libraries are missing from the -py2.py3-none-any.whl pure python wheel that gets pulled in automatically.

I was able to solve it by building the protobuf wheel from source:
brew install protobuf
export CFLAGS="-I$(brew --prefix protobuf)/include"
export LDFLAGS="-L$(brew --prefix protobuf)/lib"
# In my virtual environment: 
pip install protobuf=="$(brew list --version protobuf | awk '{print $2}')" --force-reinstall --no-deps --install-option="--cpp_implementation"
Hope this helps!

Edit (2021/11/09):

Note that this installs the version of the protobuf Python library that matches the version of the pre-compiled protoc binary that was installed via brew (brew install protobuf).
If you need a newer version than what brew provides, you’re going to have to build protoc from source. This would look something like:

# Download and unpack protobuf compiler source code
PROTOBUF_RELEASES_URL="https://github.com/protocolbuffers/protobuf/releases/download"
TARGET_PROTOBUF_VERSION=$(pip show protobuf | grep Version | sed -r 's/Version: (.*)/\1/') # get the version of the (broken) protobuf wheel currently installed
PROTOC_SRC_ARCHIVE="protobuf-cpp-${TARGET_PROTOBUF_VERSION}.tar.gz"
curl -sSL "${PROTOBUF_RELEASES_URL}/v${TARGET_PROTOBUF_VERSION}/${PROTOC_SRC_ARCHIVE}" | tar -C /tmp -xzf -

# Build protobuf compiler from source (this will take a while)
#   see: https://github.com/protocolbuffers/protobuf/blob/master/src/README.md
PROTOC_SRC_PATH="/tmp/protobuf-${TARGET_PROTOBUF_VERSION}" && \
pushd "${PROTOC_SRC_PATH}" && \
./configure && \
make -j8 && \
make check && \
sudo make install && \
popd && \
rm -rf "${PROTOC_SRC_PATH}" # Clean src/build files

# In your virtual environment:
INSTALL_PREFIX_PATH="/usr/local" && \
export CFLAGS="-I${INSTALL_PREFIX_PATH}/include" && \
export LDFLAGS="-L${INSTALL_PREFIX_PATH}/lib" && \
pip install protobuf=="${TARGET_PROTOBUF_VERSION}" --force-reinstall --no-deps --install-option="--cpp_implementation"

1reaction

duy-effectivcommented, Nov 5, 2021

I also ran into the same issue but was able to get it working. My solution (originally posted in protocolbuffers/protobuf#8820 (comment)):
I had the same problem and discovered it was because required shared libraries are missing from the -py2.py3-none-any.whl pure python wheel that gets pulled in automatically. I was able to solve it by building the protobuf wheel from source:
brew install protobuf
export CFLAGS="-I$(brew --prefix protobuf)/include"
export LDFLAGS="-L$(brew --prefix protobuf)/lib"
# In my virtual environment: 
pip install protobuf=="$(brew list --version protobuf | awk '{print $2}')" --force-reinstall --no-deps --install-option="--cpp_implementation"
Hope this helps!

Thank you so much @TeoZosa . It works for me now. Finally!!