pip install AttributeError: _DistInfoDistribution__dep_map

  • A+
Category:Languages

I'm trying to spin up an nvidia-docker (2.0) container in Ubuntu 16.04 running Conda with a few python libraries (GPU-enabled tensorflow, opencv, and gdal) and their various dependencies.

General explanation of the problem

I have a few libraries that I need to install using pip within that environment (e.g. tensorflow-gpu 1.10.0 and a couple of other custom libraries), but anytime I try to install a package using pip either in my Dockerfile or after, I get the following error:

Exception: Traceback (most recent call last):   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in _dep_map     return self.__dep_map   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2663, in __getattr__     raise AttributeError(attr) AttributeError: _DistInfoDistribution__dep_map  During handling of the above exception, another exception occurred:  Traceback (most recent call last):   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 93, in __init__     req = REQUIREMENT.parseString(requirement_string)   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1632, in parseString     raise exc   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1622, in parseString     loc, tokens = self._parse( instring, 0 )   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache     loc,tokens = self.parseImpl( instring, preloc, doActions )   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3395, in parseImpl     loc, exprtokens = e._parse( instring, loc, doActions )   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache     loc,tokens = self.parseImpl( instring, preloc, doActions )   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3183, in parseImpl     raise ParseException(instring, loc, self.errmsg, self) pip._vendor.pyparsing.ParseException: Expected stringEnd (at char 33), (line:1, col:34)  During handling of the above exception, another exception occurred:  Traceback (most recent call last):   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2949, in __init__     super(Requirement, self).__init__(requirement_string)   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__     requirement_string[e.loc:e.loc + 8])) pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'; extra '"  During handling of the above exception, another exception occurred:  Traceback (most recent call last):   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/basecommand.py", line 141, in main     status = self.run(options, args)   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 330, in run     self._warn_about_conflicts(to_install)   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 456, in _warn_about_conflicts     package_set, _dep_info = check_install_conflicts(to_install)   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts     package_set = create_package_set_from_installed()   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed     package_set[name] = PackageDetails(dist.version, dist.requires())   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2607, in requires     dm = self._dep_map   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2871, in _dep_map     self.__dep_map = self._compute_dependencies()   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2881, in _compute_dependencies     reqs.extend(parse_requirements(req))   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2942, in parse_requirements     yield Requirement(line)   File "/opt/conda/envs/tf_keras/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2951, in __init__     raise RequirementParseError(str(e)) pip._vendor.pkg_resources.RequirementParseError: Invalid requirement, parse error at "'; extra '" 

This same error occurs if I try to install other packages using pip instead of conda install (others I've tried are setuptools, h5py, and several more I can't remember)

Dockerfile

Here's the Dockerfile used to generate this, up to where the error is thrown (while installing tensorflow):

FROM nvidia/cuda:9.0-devel-ubuntu16.04 LABEL maintainer "[deleted]"  # get correct version of CUDNN for my system's CUDA ENV CUDNN_VERSION 7.3.0.29 LABEL com.nvidia.cudnn.version="${CUDNN_VERSION}"  RUN apt-get update && apt-get install -y --no-install-recommends /             libcudnn7=$CUDNN_VERSION-1+cuda9.0 /             libcudnn7-dev=$CUDNN_VERSION-1+cuda9.0 && /     apt-mark hold libcudnn7 && /     rm -rf /var/lib/apt/lists/*  # install underlying requirements RUN apt-get update /   && apt-get install -y --no-install-recommends /     bc /     bzip2 /     ca-certificates /     curl /     git /     libgl1 /     jq /     nfs-common /     parallel /     python-pip /     python-wheel /     python-setuptools /     unzip /     wget /     build-essential /   && apt-get clean /   && rm -rf /var/lib/apt/lists/*  # install anaconda RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh -O ~/miniconda.sh && /     /bin/bash ~/miniconda.sh -b -p /opt/conda && /     rm ~/miniconda.sh && /     /opt/conda/bin/conda clean -tipsy && /     ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && /     echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && /     echo "conda activate base" >> ~/.bashrc  ENV TINI_VERSION v0.16.1 ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /usr/bin/tini RUN chmod +x /usr/bin/tini  # add conda to $PATH and create a conda environment ENV PATH /opt/conda/bin:$PATH RUN conda update conda && /     conda config --remove channels defaults && /     conda config --add channels conda-forge && /     conda create -n tf_keras python=3.6 /                     && echo "source activate tf_keras" > ~/.bashrc ENV PATH /opt/conda/envs/tf_keras/bin:$PATH  SHELL ["/bin/bash", "-c"]  # install required libraries (and some dependencies) RUN conda install -n tf_keras /               osmnx=0.7.3 /               affine /               pyproj /               pyhamcrest=1.9.0 /               cython /               fiona /               h5py /               ncurses /               jupyter /               jupyterlab /               ipykernel /               libgdal /               matplotlib /               numpy /               opencv /               pandas /               pillow /               pip /               scipy /               scikit-image /               scikit-learn /               shapely /               gdal /               rtree /               tqdm /               pandas /               geopandas /               rasterio  # get tensorflow ARG TENSORFLOW_VERSION=1.10.0 ARG TENSORFLOW_DEVICE=gpu ARG TENSORFLOW_APPEND=_gpu RUN source activate tf_keras && /     pip --no-cache-dir install https://storage.googleapis.com/tensorflow/linux/${TENSORFLOW_DEVICE}/tensorflow${TENSORFLOW_APPEND}-${TENSORFLOW_VERSION}-cp36-cp36m-linux_x86_64.whl 

The last RUN command throws the error.

Things I've tried:

  • Different versions of pip (10.0.1 and 18.0)
  • pip upgrade pip or conda upgrade pip (trying to update pip this way throws the same error)
  • updating setuptools (throws the same error if I try to do it with pip)
  • I've checked to make sure that the correct pip is being used by prepending echo $(which pip) && before my pip install commands - it returns the virtual environment's pip.

I would just install everything using conda and ignore it, but there are a couple of libraries internal to my group which aren't available for conda.

 


You should downgrade your version of testpath:

conda install 'testpath<0.4' 

See issue here: https://github.com/conda-forge/testpath-feedstock/issues/7

Comment

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: