[Release] Candidate Kepler-Jupyter v0.3.4

[lixun910]

Please help to test the release candidate: kepler-jupyter v0.3.4 alpha3. Thanks!

[netlify]

✅ Deploy Preview for keplergl2 ready!

Name	Link
🔨 Latest commit	55648b3
🔍 Latest deploy log	https://app.netlify.com/sites/keplergl2/deploys/676e19053331be0008661f5e
😎 Deploy Preview	https://deploy-preview-2588--keplergl2.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[lixun910]

Thanks, Ib!

@ilyabo Hi Ilya, can you help to test kepler-jupyter 0.3.4a3? I've noticed there are some conflicts of dependencies which can be ignored on my side (Mac and Ubuntu in dockers). However, I want to make sure it is working on others as well :) Thank you in advance!

[kopp]

Hi @lixun910 , I wanted to take a peek into this pre-release and did on Ubuntu 20.04 with python3.10

uv pip install keplergl==v0.3.4a3

but after trying to display KeplerGl in jupyterlab I only ended up with

Error displaying widget: model not found

i.e. #2687 .

Can you share the packages you have installed so that it is working?

[lixun910]

@kopp Hi, I was using python 3.10 on my Macbook. I remember two things might cause the issues:

first is the jupyter and jupyter kernel: if you have many different versions of jupyter, the which jupyter will always point to the same jupyter even you are in different virtual environment. So I need to make sure I am running the jupyter with right kernel. You can use jupyter kernelspec list to check your kernel, and python -m ipykernel install --user --name=myenv to add your virtual env to jupyter kernel.

second is the notebook version, we set the notebook==6.0.1 for kepler.gl-jupyter. However, if you have a newer notebook version installed somewhere, the notebook==6.0.1 will not be installed even in a virtual environment. So I remember I have to downgrade it pip install --upgrade notebook==6.0.1.

I also tested on Google colabs with the default python setup, and it works as shown in the screenshot.

Let me know if this helps. Thanks!

(.venv) xun@xuns-mbp kepler.gl-test % jupyter --version
Selected Jupyter core packages...
IPython : 8.23.0
ipykernel : 6.29.4
ipywidgets : 7.8.1
jupyter_client : 8.6.1
jupyter_core : 5.7.2
jupyter_server : 2.14.0
jupyterlab : 4.1.6
nbclient : 0.10.0
nbconvert : 7.16.3
nbformat : 5.10.4
notebook : 6.0.1
qtconsole : 5.5.1
traitlets : 5.14.3

[zxweed]

Hi @lixun910, I see (by pipdeptree) that stable keplergl package requires the >=4.2.0 version of jupyterlab. I understood correctly that you forcibly installed jupyterlab version 4.1.6 and no "model not found" error occurs with it?

[lixun910]

@zxweed Thanks for pointing this out! I didn't notice the version of jupyterlab is 4.1.6. I just did a quick test upgrading jupyterlab to 4.2.0, and it works as well. So, I think it is just the version of notebook requires to be 6.0.1

I will try to work on it to support notebook > 6.0.1 in another pr, which I think will be a better solution...

 % jupyter --version
Selected Jupyter core packages...
IPython          : 8.23.0
ipykernel        : 6.29.4
ipywidgets       : 7.8.1
jupyter_client   : 8.6.1
jupyter_core     : 5.7.2
jupyter_server   : 2.14.0
jupyterlab       : 4.2.0
nbclient         : 0.10.0
nbconvert        : 7.16.3
nbformat         : 5.10.4
notebook         : 6.0.1
qtconsole        : 5.5.1
traitlets        : 5.14.3

[lixun910]

ref #1993 (comment)
ref #2545

[lixun910]

kepler-jupyter v0.3.4 alpha4 (will be published soon)

• Support ipywidgets > 8 and notebook > 6
• Add Dockerfile under bindings/kepler.gl-jupyter to have a clean test environment

/kepler.gl# .venv/bin/jupyter --version
Selected Jupyter core packages...
IPython          : 8.29.0
ipykernel        : 6.29.5
ipywidgets       : 8.1.5
jupyter_client   : 8.6.3
jupyter_core     : 5.7.2
jupyter_server   : 2.14.2
jupyterlab       : 4.2.5
nbclient         : 0.10.0
nbconvert        : 7.16.4
nbformat         : 5.10.4
notebook         : 7.2.2
qtconsole        : not installed
traitlets        : 5.14.3

[rbavery]

@zxweed Thanks for pointing this out! I didn't notice the version of jupyterlab is 4.1.6. I just did a quick test upgrading jupyterlab to 4.2.0, and it works as well. So, I think it is just the version of notebook requires to be 6.0.1

I will try to work on it to support notebook > 6.0.1 in another pr, which I think will be a better solution...

 % jupyter --version
Selected Jupyter core packages...
IPython          : 8.23.0
ipykernel        : 6.29.4
ipywidgets       : 7.8.1
jupyter_client   : 8.6.1
jupyter_core     : 5.7.2
jupyter_server   : 2.14.0
jupyterlab       : 4.2.0
nbclient         : 0.10.0
nbconvert        : 7.16.3
nbformat         : 5.10.4
notebook         : 6.0.1
qtconsole        : 5.5.1
traitlets        : 5.14.3

I tried these versions with the alpha3 release but still run into

Error displaying widget: model not found

in the console:

528.46ebd3d2bc93085c64d3.js?v=46ebd3d2bc93085c64d3:1 Error: Module keplergl-jupyter, semver range ^0.3.4-a3 is not registered as a widget module
    at x.loadClass (528.46ebd3d2bc93085c64d3.js?v=46ebd3d2bc93085c64d3:1:5893)
    at x._make_model (316.59c86d96339382334dbb.js?v=59c86d96339382334dbb:1:6681)
    at x.new_model (316.59c86d96339382334dbb.js?v=59c86d96339382334dbb:1:6572)
    at x.handle_comm_open (316.59c86d96339382334dbb.js?v=59c86d96339382334dbb:1:5424)
    at _handleCommOpen (528.46ebd3d2bc93085c64d3.js?v=46ebd3d2bc93085c64d3:1:2813)
    at C._handleCommOpen (default.js:1118:19)
    at async C._handleMessage (default.js:1285:43)

here are the versions I have from pip list in my environment, which is also where jupyter is installed. there are no other environments or kernels.

IPython==8.23.0
ipykernel==6.29.4
ipywidgets==7.8.1
jupyter_client==8.6.1
jupyter_core==5.7.2
jupyter_server==2.14.0
jupyterlab==4.1.6
nbclient==0.10.0
nbconvert==7.16.3
nbformat==5.10.4
notebook==6.0.1
qtconsole==5.5.1
traitlets==5.14.3

This is run in an Ubuntu 22.04 image. I'm down to try the alpha4 release once it is up

[lixun910]

This should be fixed in v0.3.4 alpha4, however we are waiting for the kepler.gl admin to publish the v0.3.4 alpha4 cc @heshan0131 .
You can also create and install package locally using this PR and try to verify it. I did try it using latest Ubuntu: https://github.com/keplergl/kepler.gl/pull/2588/files#diff-77018316c61afb9e5aef5167e2c24cc2309f880731459276e8ae0b9bfade162f

[kopp]

Hi @lixun910 , thanks a lot for trying to make this work. If I get it right, the last successful push to NPM was https://github.com/keplergl/kepler.gl/releases/tag/v3.1.0-alpha.1 which used the workflow

name: Node.js Package

on:
  release:
    types: [created]

jobs:
  publish-npm:
    runs-on: ubuntu-latest

    env:
      NPM_AUTH_TOKEN: ${{secrets.npm_token}}

    steps:
      - uses: actions/checkout@v4

      # use Volta to manage yarn/node versions
      - uses: volta-cli/action@v4

      - run: yarn install
      - run: yarn bootstrap
      - run: npm i -g [email protected]

      - name: Login to NPM
        run: npm config set "//registry.npmjs.org/:_authToken=${NPM_AUTH_TOKEN}"

      - name: Publish
        run: npm publish --workspaces --access public

The notable difference I see to what failed here is that the auth token is set to a string that references an environment variable, while in the failing code the value is directly set. Depending on what characters are included in the token and how the value is parsed, this can make a difference (e.g. if it contains a ; and this ; can end to parse the string).

To my understanding (and the documentation of https://github.com/actions/setup-node/tree/v4) it should be sufficient to set the registry in setup-node and store/provide the token in environment variable NODE_AUTH_TOKEN, then 'it should just work' ;)

So either the token is not correctly provided to npm or the token does not have the correct permissions.

[lixun910]

Thanks, @kopp! Yes, I think you are right, and it’s the token issue: the current token has been updated recently and is for publishing kepler.gl npm package, and can not be used to publish the kepler-Jupyter npm package. Once the token been updated, we will be able to publish this candidate release.

[kopp]

@lixun910 , for the time being I wanted to use the Dockerfile that you provided to build the plugin locally, but I failed. I assume that it is due to different setups (me running on an Arch Linux system). I wanted to provide the possibility to build everything (including yarn) in docker so that this becomes portable, hence I opened #2841 . Unfortunately there are still errors which I do not understand. Can you please have a look at that PR? Maybe you can find how the setup in the docker differs from your local (working) one.

[kopp]

BTW: Did you consider to provide a Dev Container for the keplergl project? Then all builds could happen in this dev container and the environment would be super portable.

[lixun910]

https://pypi.org/project/keplergl/
https://www.npmjs.com/package/keplergl-jupyter

For a clean test:

1. Build a docker use the following dockerfile:
   docker build -t kepler-jupyter .

FROM ubuntu:latest

# Install python3
RUN apt-get update && apt-get install -y python3 python3-pip python3-venv

# Install gdal
RUN apt-get install -y gdal-bin

# Fix: No such file or directory: 'gdal-config'
RUN apt-get install -y libgdal-dev

# Create a virtual environment .venv in a specific location
RUN python3 -m venv /opt/.venv

# Install keplergl-jupyter in the virtual environment from the current directory
RUN /opt/.venv/bin/pip install geopandas jupyter jupyterlab ipywidgets notebook jupyter_packaging pyarrow setuptools wheel
RUN /opt/.venv/bin/pip install keplergl==0.3.5

# Add a working directory
WORKDIR /notebooks

# Run jupyter notebook with token exposed in logs
CMD ["bash", "-c", "cd /notebooks && /opt/.venv/bin/jupyter notebook --ip=0.0.0.0 --port=8888 --no-browser --allow-root --NotebookApp.log_level='INFO'"]

EXPOSE 8888

2. Run the docker under your notebook directory

docker run -v $(pwd):/notebooks --rm -p 8888:8888/tcp kepler-jupyter:latest

3. Run your notebooks in browser

e.g. http://127.0.0.1:8888/tree?token=f5be411ce5567b49baaaa80977f100aa4819d61fd66d3559

[miguelbalmeida]

Hi everyone, I just parachute-landed into this thread. I want to share a different use-case with you all and possibly volunteer to test the new RC, if you feel like you want to support that use-case.

I am a data scientist in the field of smart cities. I am currently trying to visualize the before/after situation of street interventions in a city (e.g. a street is closed off, another changes from being one-way to being two-way, etc). Data analysis can help, but for now I'm just interested in providing a good visualization of this. I've tried Streamlit, but support for maps is not that good. Colleagues of mine have tried PowerBI and Grafana, but both are less powerful than what we want for the specific use-case of maps and mobility.

What I am currently doing is a simple Docker application which serves an HTML file. That HTML file is the result of Kepler's save_to_html() function. Ideally, I would create a clone of Kepler's excellent demo app, but I saw in another issue that the maintainers have no intention of releasing it as a Docker app (which I respect), and I am not skilled in React/JS, so this is the next best option within my skillset.

I will also have separate components which will make sure there is updated data on the machine serving this Docker app, and will also periodically refresh the data within the Docker app. The Python code within the Docker app will be something like

import time
from keplergl import KeplerGl

kmap = KeplerGl()

def load_latest_data():
    # something that loads the latest data from a mounted Docker volume
    # a separate component, outside Docker, will make sure there is updated data there

while True:
    latest_data = load_latest_data()
    time.sleep(300)  # sleep for 5 minutes
    # add the latest data to the Kepler map
    kmap.save_to_html(...)

I bring you this use-case because I am not interested in running Kepler from a Jupyter notebook. If you'd like, I can try to test this release candidate and see how it works for this use-case, since I am very interested in the new release supporting my use-case. I guess I'd basically be testing the save_to_html() function, but with a pretty significant amount of data and layers in that HTML file.

If you want to support this, kindly let me know how I should test the release candidate and I'll get on it.