Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Make Docker build script to accept custom base image #373

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[WIP] Make Docker build script to accept custom base image #373

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5fa1d63
Merge pull request #24 from NVIDIA-AI-IOT/master
tokk-nv Jan 15, 2021
35fcf29
Accept optional BASE_IMAGE
tokk-nv Jan 20, 2021
9d6207e
Limit to r32.4.4
tokk-nv Jan 20, 2021
149b8fa
Remove JETBOT_DOCKER_RETMOTE env variable
tokk-nv Jan 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions docker/base/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
cp /etc/apt/trusted.gpg.d/jetson-ota-public.asc ../.. # copy to jetbot root

if [[ "$L4T_VERSION" == "32.4.3" ]]
then
BASE_IMAGE=nvcr.io/nvidia/l4t-pytorch:r32.4.3-pth1.6-py3
elif [[ "$L4T_VERSION" == "32.4.4" ]]
then
BASE_IMAGE=nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3
fi
BASE_IMAGE=$1

echo "sudo docker build \
--build-arg BASE_IMAGE=$BASE_IMAGE \
-t jetbot/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-f Dockerfile \
../.." # jetbot repo root as context

sudo docker build \
--build-arg BASE_IMAGE=$BASE_IMAGE \
-t $JETBOT_DOCKER_REMOTE/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t jetbot/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason for removing JETBOT_DOCKER_REMOTE?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not find this is actually being used, so this is just for simplifying things.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was used previously for hosting/sharing test containers with different remotes (ie: private ones). I'm not sure if we'll need to do this anymore, but may be easiest / safest to leave it as is until we're sure we don't want to revert back.

-f Dockerfile \
../.. # jetbot repo root as context

20 changes: 19 additions & 1 deletion docker/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,22 @@
cd base && ./build.sh && cd ..
BASE_IMAGE=$1

# find L4T_VERSION
source ../scripts/l4t-version.sh

if [ -z $BASE_IMAGE ]; then
if [ $L4T_VERSION = "32.4.4" ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to drop 32.4.3 which was previously handled in docker/base/build.sh?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, can we move this logic to configure.sh as an environment variable? This is currently where all other docker build configuration is done.

BASE_IMAGE="nvcr.io/nvidia/l4t-pytorch:r32.4.4-pth1.6-py3"
else
echo "cannot build jetbot docker containers for L4T R$L4T_VERSION"
echo "please upgrade to the latest JetPack, or build jetson-inference natively"
exit 1
fi
fi

echo "cd base && ./build.sh $BASE_IMAGE && cd .."
cd base && ./build.sh $BASE_IMAGE && cd ..

echo "cd models && ./build.sh && cd .."
cd models && ./build.sh && cd ..
cd display && ./build.sh && cd ..
cd jupyter && ./build.sh && cd ..
Expand Down
4 changes: 2 additions & 2 deletions docker/camera/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
sudo docker build \
--build-arg BASE_IMAGE=$JETBOT_DOCKER_REMOTE/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t $JETBOT_DOCKER_REMOTE/jetbot:camera-$JETBOT_VERSION-$L4T_VERSION \
--build-arg BASE_IMAGE=jetbot/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t jetbot/jetbot:camera-$JETBOT_VERSION-$L4T_VERSION \
-f Dockerfile .
2 changes: 1 addition & 1 deletion docker/camera/enable.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ sudo docker run -it -d \
--volume /tmp/argus_socket:/tmp/argus_socket \
--privileged \
--name=jetbot_camera \
$JETBOT_DOCKER_REMOTE/jetbot:camera-$JETBOT_VERSION-$L4T_VERSION
jetbot/jetbot:camera-$JETBOT_VERSION-$L4T_VERSION
13 changes: 4 additions & 9 deletions docker/configure.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,14 @@
#!/bin/bash

export JETBOT_VERSION=0.4.2
echo "export JETBOT_VERSION=$JETBOT_VERSION"

L4T_VERSION_STRING=$(head -n 1 /etc/nv_tegra_release)
L4T_RELEASE=$(echo $L4T_VERSION_STRING | cut -f 2 -d ' ' | grep -Po '(?<=R)[^;]+')
L4T_REVISION=$(echo $L4T_VERSION_STRING | cut -f 2 -d ',' | grep -Po '(?<=REVISION: )[^;]+')

export L4T_VERSION="$L4T_RELEASE.$L4T_REVISION"

if [[ "$L4T_VERSION" == "32.4.3" ]]
then
# docker hub
export JETBOT_DOCKER_REMOTE=jetbot
elif [[ "$L4T_VERSION" == "32.4.4" ]]
then
export JETBOT_DOCKER_REMOTE=jetbot
fi
echo "export L4T_VERSION=$L4T_RELEASE.$L4T_REVISION"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of deleting JETBOT_DOCKER_REMOTE. I would recommend hard-setting setting a default value of

export JETBOT_DOCKER_REMOTE=jetbot

This is the smallest change to accomplish what I believe the intent was (to handle L4T versions which are not specified here).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok


./set_nvidia_runtime.sh
sudo systemctl enable docker
Expand All @@ -25,6 +18,8 @@ SYSTEM_RAM_KILOBYTES=$(awk '/^MemTotal:/{print $2}' /proc/meminfo)

if [ $SYSTEM_RAM_KILOBYTES -gt 3000000 ]
then
echo "export JETBOT_JUPYTER_MEMORY=500m"
echo "export JETBOT_JUPYTER_MEMORY_SWAP=3G"
export JETBOT_JUPYTER_MEMORY=500m
export JETBOT_JUPYTER_MEMORY_SWAP=3G
fi
Expand Down
4 changes: 2 additions & 2 deletions docker/display/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
sudo docker build \
--build-arg BASE_IMAGE=$JETBOT_DOCKER_REMOTE/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t $JETBOT_DOCKER_REMOTE/jetbot:display-$JETBOT_VERSION-$L4T_VERSION \
--build-arg BASE_IMAGE=jetbot/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t jetbot/jetbot:display-$JETBOT_VERSION-$L4T_VERSION \
-f Dockerfile .

2 changes: 1 addition & 1 deletion docker/display/enable.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ sudo docker run -it -d \
--network host \
--privileged \
--name=jetbot_display \
$JETBOT_DOCKER_REMOTE/jetbot:display-$JETBOT_VERSION-$L4T_VERSION
jetbot/jetbot:display-$JETBOT_VERSION-$L4T_VERSION
4 changes: 2 additions & 2 deletions docker/jupyter/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
sudo docker build \
--build-arg BASE_IMAGE=$JETBOT_DOCKER_REMOTE/jetbot:models-$JETBOT_VERSION-$L4T_VERSION \
-t $JETBOT_DOCKER_REMOTE/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION \
--build-arg BASE_IMAGE=jetbot/jetbot:models-$JETBOT_VERSION-$L4T_VERSION \
-t jetbot/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION \
-f Dockerfile .
4 changes: 2 additions & 2 deletions docker/jupyter/enable.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ then
--name=jetbot_jupyter \
--memory-swap=$JETBOT_JUPYTER_MEMORY_SWAP \
--env JETBOT_DEFAULT_CAMERA=$JETBOT_CAMERA \
$JETBOT_DOCKER_REMOTE/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION
jetbot/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION

else

Expand All @@ -43,6 +43,6 @@ else
--memory=$JETBOT_JUPYTER_MEMORY \
--memory-swap=$JETBOT_JUPYTER_MEMORY_SWAP \
--env JETBOT_DEFAULT_CAMERA=$JETBOT_CAMERA \
$JETBOT_DOCKER_REMOTE/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION
jetbot/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION

fi
9 changes: 7 additions & 2 deletions docker/models/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
echo "sudo docker build \
--build-arg BASE_IMAGE=jetbot/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t jetbot/jetbot:models-$JETBOT_VERSION-$L4T_VERSION \
-f Dockerfile ."

sudo docker build \
--build-arg BASE_IMAGE=$JETBOT_DOCKER_REMOTE/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t $JETBOT_DOCKER_REMOTE/jetbot:models-$JETBOT_VERSION-$L4T_VERSION \
--build-arg BASE_IMAGE=jetbot/jetbot:base-$JETBOT_VERSION-$L4T_VERSION \
-t jetbot/jetbot:models-$JETBOT_VERSION-$L4T_VERSION \
-f Dockerfile .
6 changes: 3 additions & 3 deletions docker/push.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash

sudo docker push $JETBOT_DOCKER_REMOTE/jetbot:camera-$JETBOT_VERSION-$L4T_VERSION
sudo docker push $JETBOT_DOCKER_REMOTE/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION
sudo docker push $JETBOT_DOCKER_REMOTE/jetbot:display-$JETBOT_VERSION-$L4T_VERSION
sudo docker push jetbot/jetbot:camera-$JETBOT_VERSION-$L4T_VERSION
sudo docker push jetbot/jetbot:jupyter-$JETBOT_VERSION-$L4T_VERSION
sudo docker push jetbot/jetbot:display-$JETBOT_VERSION-$L4T_VERSION
49 changes: 49 additions & 0 deletions scripts/l4t-version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/bin/bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this script used. Can we remove?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is tied to the change above to build.sh.
If we don't take that change above, we don't need this.

#
# Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
#

L4T_VERSION_STRING=$(head -n 1 /etc/nv_tegra_release)

if [ -z "$L4T_VERSION_STRING" ]; then
echo "reading L4T version from \"dpkg-query --show nvidia-l4t-core\""

L4T_VERSION_STRING=$(dpkg-query --showformat='${Version}' --show nvidia-l4t-core)
L4T_VERSION_ARRAY=(${L4T_VERSION_STRING//./ })

#echo ${L4T_VERSION_ARRAY[@]}
#echo ${#L4T_VERSION_ARRAY[@]}

L4T_RELEASE=${L4T_VERSION_ARRAY[0]}
L4T_REVISION=${L4T_VERSION_ARRAY[1]}
else
echo "reading L4T version from /etc/nv_tegra_release"

L4T_RELEASE=$(echo $L4T_VERSION_STRING | cut -f 2 -d ' ' | grep -Po '(?<=R)[^;]+')
L4T_REVISION=$(echo $L4T_VERSION_STRING | cut -f 2 -d ',' | grep -Po '(?<=REVISION: )[^;]+')
fi

L4T_REVISION_MAJOR=${L4T_REVISION:0:1}
L4T_REVISION_MINOR=${L4T_REVISION:2:1}

L4T_VERSION="$L4T_RELEASE.$L4T_REVISION"

echo "L4T BSP Version: L4T R$L4T_VERSION"