Failed to build jupyter-scipy-notebook-ee image.

[jmasselink]

While attempting to build jupyter-scipy-notebook-ee image, I encountered the following error which prevented me from building the image properly.

I am using Windows 10 Home and Docker Toolbox, with Virtualization enabled.

failed
ERROR conda.core.link:_execute(502): An error occurred while installing package 'conda-forge::ipywebrtc-0.4.0-py36_1'.
LinkError: post-link script failed for package conda-forge::ipywebrtc-0.4.0-py36_1
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.ipywebrtc-post-link.sh
==> script messages <==
<None>

Attempting to roll back.

Rolling back transaction: ...working... done

LinkError: post-link script failed for package conda-forge::ipywebrtc-0.4.0-py36_1
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.ipywebrtc-post-link.sh
==> script messages <==
<None>

[xgirouxb]

I have an identical problem, though the issue seems related to the bqplot package. Failed to build on a Linux Mint 19.2 machine, with Docker version 18.09.7

ERROR conda.core.link:_execute(502): An error occurred while installing package 'conda-forge::bqplot-0.11.2-py36_1000'.
LinkError: post-link script failed for package conda-forge::bqplot-0.11.2-py36_1000
running your command again with `-v` will provide additional information
location of failed script: /opt/conda/bin/.bqplot-post-link.sh
==> script messages <==
Traceback (most recent call last):
  File "/opt/conda/bin/jupyter-nbextension", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.6/site-packages/jupyter_core/application.py", line 266, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 988, in start
    super(NBExtensionApp, self).start()
  File "/opt/conda/lib/python3.6/site-packages/jupyter_core/application.py", line 255, in start
    self.subapp.start()
  File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 896, in start
    self.toggle_nbextension_python(self.extra_args[0])
  File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 872, in toggle_nbextension_python
    logger=self.log)
  File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 483, in enable_nbextension_python
    logger=logger)
  File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 380, in _set_nbextension_state_python
    m, nbexts = _get_nbextension_metadata(module)
  File "/opt/conda/lib/python3.6/site-packages/notebook/nbextensions.py", line 1122, in _get_nbextension_metadata
    m = import_item(module)
  File "/opt/conda/lib/python3.6/site-packages/traitlets/utils/importstring.py", line 42, in import_item
    return __import__(parts[0])
  File "/opt/conda/lib/python3.6/site-packages/bqplot/__init__.py", line 64, in <module>
    from .figure import *
  File "/opt/conda/lib/python3.6/site-packages/bqplot/figure.py", line 36, in <module>
    from .scales import Scale, LinearScale
  File "/opt/conda/lib/python3.6/site-packages/bqplot/scales.py", line 45, in <module>
    from .traits import Date
  File "/opt/conda/lib/python3.6/site-packages/bqplot/traits.py", line 33, in <module>
    import pandas as pd
  File "/opt/conda/lib/python3.6/site-packages/pandas/__init__.py", line 23, in <module>
    from pandas.compat.numpy import *
  File "/opt/conda/lib/python3.6/site-packages/pandas/compat/numpy/__init__.py", line 10, in <module>
    _np_version = np.__version__
AttributeError: module 'numpy' has no attribute '__version__'

Attempting to roll back.

[xgirouxb]

For those of you who come across this post, I've have found a workaround by updating the jupyter scipy notebook docker that serves as the base to build the jupyter-scipy-notebook-ee, as well as updating the package versions for the different packages that are subsequently installed. Here is the Dockerfile I have modified to achieve a succesful build, but keep in mind I got here by trial and error, not a deep understanding of Docker builds:

FROM jupyter/scipy-notebook:2ce7c06a61a1
LABEL base-image="jupyter/scipy-notebook:2ce7c06a61a1 10/08/2019"
LABEL maintainer="Tyler Erickson <[email protected]>"

# Altair - Declarative visualization in Python
#     docs: https://altair-viz.github.io
#   source: https://github.com/altair-viz/altair
#     pkgs: conda search -c conda-forge altair
ARG ALTAIR_VERSION=3.2.0

# bqplot - Grammar of Graphics-based interactive plotting framework for the Jupyter notebook
#     docs: https://bqplot.readthedocs.io
#   source: https://github.com/bloomberg/bqplot
#     pkgs: conda search -c conda-forge bqplot
#           npm search bqplot
ARG BQPLOT_VERSION=0.11.4
ARG BQPLOT_NPM_VERSION=0.4.7

# Earth Engine Python API 
#     home: https://earthengine.google.com
#     docs: https://developers.google.com/earth-engine
#   source: https://github.com/google/earthengine-api
#     pkgs: conda search -c conda-forge earthengine-api
ARG EARTHENGINE_API_VERSION=0.1.192

# Google API Python Client 
#     home: http://google.github.io/google-api-python-client/
#     docs: https://developers.google.com/api-client-library/python/apis/
#   source: https://github.com/google/google-api-python-client/
#     pkgs: conda search -c conda-forge google-api-python-client
ARG GOOGLE_API_PYTHON_VERSION=1.7.11

# ipyleaflet - Leaflet-based interactiving mapping widget
#     docs: https://ipyleaflet.readthedocs.io
#   source: https://github.com/jupyter-widgets/ipyleaflet
#     pkgs: conda search -c conda-forge ipyleaflet
ARG IPYLEAFLET_VERSION=0.11.1

# ipyvolume - 3d plotting in the Jupyter notebook
#     docs: https://ipyvolume.readthedocs.io/
#   source: https://github.com/maartenbreddels/ipyvolume
#     pkgs: conda search -c conda-forge ipyvolume
#           npm search jupyter-threejs
ARG IPYVOLUME_VERSION=0.5.2
ARG THREEJS_VERSION=2.1.1

# jupyterlab-manager
#   source: https://github.com/jupyter-widgets/ipywidgets/tree/master/packages/jupyterlab-manager
#     pkgs: npm search @jupyter-widgets/jupyterlab-manager
ARG WIDGETS_JUPYTERLAB_MANAGER_VERSION=1.0.2

# nbdime - Notebook Diff & Merge tool
#     docs: https://nbdime.readthedocs.io
#   source: https://github.com/jupyter/nbdime
#     pkgs: conda search -c conda-forge nbdime
ARG NBDIME_VERSION=1.1.0

# Palettable - color maps
#     docs: https://jiffyclub.github.io/palettable
#   source: https://github.com/jiffyclub/palettable
#     pkgs: conda search -c conda-forge palettable
ARG PALETTABLE_VERSION=3.2.0

# PyDrive - Simplifies many Google Drive API tasks
#     home: https://pypi.org/project/PyDrive
#     docs: https://pythonhosted.org/PyDrive
#   source: https://github.com/gsuitedevs/PyDrive
#     pkgs: conda search -c conda-forge pydrive
ARG PYDRIVE_VERSION=1.3.1

# TensorFlow - Computation using data flow graphs for scalable machine learning
#     home: https://www.tensorflow.org
#     docs: https://www.tensorflow.org/api_docs
#   source: https://github.com/tensorflow/tensorflow
#     pkgs: conda search -c conda-forge tensorflow
ARG TENSORFLOW_VERSION=1.13.1

# Vega Datasets - Collection of datasets used in Vega and Vega-Lite examples
#     docs: http://vega.github.io/vega-datasets
#   source: https://github.com/vega/vega-datasets
#     pkgs: conda search -c conda-forge vega_datasets
ARG VEGA_DATASETS_VERSION=0.7.0

#### Other packages
#
# jupyterlab/toc - Table of Contents extension for JupyterLab
#   source: https://github.com/ian-r-rose/jupyterlab-toc
#
# nbgitpuller - Notebook Git Puller
#   source: https://github.com/data-8/nbgitpuller
#
# Sidecar - A sidecar output widget for JupyterLab
#   source: https://github.com/jupyter-widgets/jupyterlab-sidecar
#
# ipythonblocks - colored grids
#   home: http://www.ipythonblocks.org/

RUN conda update -n base conda &&\ 
    conda install --quiet --yes \
        altair=${ALTAIR_VERSION} \
        bqplot=${BQPLOT_VERSION} \
        earthengine-api=${EARTHENGINE_API_VERSION} \
        google-api-python-client=${GOOGLE_API_PYTHON_VERSION} \
        ipyleaflet=${IPYLEAFLET_VERSION} \
        ipyvolume=${IPYVOLUME_VERSION} \
        nbdime=${NBDIME_VERSION} \
        palettable=${PALETTABLE_VERSION} \
        pydrive=${PYDRIVE_VERSION} \
        tensorflow=${TENSORFLOW_VERSION} \
        vega_datasets=${VEGA_DATASETS_VERSION} \
      && \
    conda upgrade --all && \
    conda build purge-all && \
    jupyter nbextension enable --py widgetsnbextension --sys-prefix && \
    # JupyterLab Extensions
    pip install sidecar && \
    jupyter labextension install --no-build jupyter-leaflet@${IPYLEAFLET_VERSION} && \
    jupyter labextension install --no-build ipyvolume@${IPYVOLUME_VERSION} && \
    jupyter labextension install --no-build jupyter-threejs@${THREEJS_VERSION} && \
    jupyter labextension install --no-build bqplot@${BQPLOT_NPM_VERSION} && \
    jupyter labextension install --no-build @jupyterlab/toc && \
    pip install git+https://github.com/data-8/nbgitpuller \
      && jupyter serverextension enable --py nbgitpuller --sys-prefix && \
    pip install ipythonblocks && \
    export NODE_OPTIONS=--max-old-space-size=16000 && \
    jupyter lab build && \
    # Clean up
    npm cache clean --force && \
    rm -rf $CONDA_DIR/share/jupyter/lab/staging && \
    rm -rf /home/$NB_USER/.cache/yarn && \
    rm -rf /home/$NB_USER/.node-gyp && \
    fix-permissions $CONDA_DIR && \
    fix-permissions /home/$NB_USER

[tylere]

Thanks for working on updating the Dockerfile, which as you discovered is using quite a few old versions. Feel free to submit a Pull Request for review with the changes you found to work.

[xgirouxb]

Done! I must say I am by no means a Docker expert. However I am a little puzzled, as my limited understanding of Docker is that it provides stable and consistent builds of things that just work, and therefore don't really have to be updated. Is the issue then that some of the packages being installed were pointing to broken versions?

[tylere]

That understanding of Docker is correct. However, in this case Docker is being used to build an environment of "front end applications" including the Earth Engine Python API, which needs to be kept in sync with the version of the Earth Engine backend servers (i.e. the cloud service). And we push updates to Earth Engine backend servers about once a week.