Skip to content

Commit

Permalink
chore: enable Jupyter Notebook gateway for the Dataproc cluster (#651)
Browse files Browse the repository at this point in the history
Sometimes it is very useful to be able to spin up a Jupyter notebook on a Dataproc cluster which has access to Gentropy and all its configured environment. Previously, I already added the “enable_component_gateway” option; however, it doesn't really take any effect unless you also specify the list of components to enable, which is what I'm doing in this PR.
  • Loading branch information
tskir authored Jun 24, 2024
1 parent 2fd6d1f commit a2a7a82
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions src/airflow/dags/common_airflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ def create_cluster(
num_workers=num_workers,
image_version=GCP_DATAPROC_IMAGE,
enable_component_gateway=True,
optional_components=["JUPYTER"],
init_actions_uris=INITIALISATION_EXECUTABLE_FILE,
metadata={
"CONFIGTAR": CONFIG_TAG,
Expand Down

0 comments on commit a2a7a82

Please sign in to comment.