Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecated Apache Beam Version Causing Error in weather-dl tool. #362

Open
dabhicusp opened this issue Jul 19, 2023 · 0 comments
Open

Deprecated Apache Beam Version Causing Error in weather-dl tool. #362

dabhicusp opened this issue Jul 19, 2023 · 0 comments

Comments

@dabhicusp
Copy link
Collaborator

dabhicusp commented Jul 19, 2023

I encountered an issue while using the following command to download data from cds using weather-dl tool. Since this is an urgent task, I would greatly appreciate any assistance in resolving this error.

command used : python "weather_dl/weather-dl" weather-tools/config_files/*.cfg --runner DataflowRunner --project $PROJECT --region $REGION --temp_location "gs://$BUCKET/tmp/" --disk_size_gb 260 --sdk_container_image "gcr.io/*****/miniconda3-beam:weather-tools-with-aria2" --experiment use_runner_v2

I performed the above command in a fresh environment of weather-tools, and within the config_files directory, there is only one file with the following content:

[parameters]
client=cds
dataset=reanalysis-era5-complete
target_path=gs://gcp-public-data-arco-era5/raw/ERA5GRIB/HRES/Daily/{date:%%Y/%%Y%%m%%d}_hres_dve.grb2
partition_keys=
    date

[parameters.sahil]
api_url=https://cds.climate.copernicus.eu/api/v2
api_key=****:****-****-****

[selection]
class=ea
stream=oper
expver=1
type=an
levtype=ml
levelist=1/to/137
date=2023-04-01/to/2023-04-30
time=00/to/23
param=138/155

while researching the above issue I discovered that the version of Apache Beam used in tool is deprecated.

The error I got is ::
e1
e2
e3
e4
e5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant