Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0.7.290 issue with apero_reject.py #789

Open
larnoldgithub opened this issue Oct 18, 2024 · 3 comments
Open

0.7.290 issue with apero_reject.py #789

larnoldgithub opened this issue Oct 18, 2024 · 3 comments

Comments

@larnoldgithub
Copy link
Collaborator

A test with apero_reject.py failed.

apero_reject.py --identifier=3036202o,3036203o,3036204o,3036235o --test=True --autofill="1,1,1,OBJECT_grade5_24BQ03-Aug16"

20:55:27.763- |REJECT| ***************************************************************************
20:55:27.774- |REJECT| *
20:55:27.774- |REJECT| * SPIROU @PID-00017291985269974710-MS7R (V0.7.290)
20:55:27.775- |REJECT| * py3.9.16 git:v0.7.290-stable-test
20:55:27.775- |REJECT| *
20:55:27.785- |REJECT| ***************************************************************************
20:55:27.794- |REJECT| █████╗ ██████╗ ███████╗██████╗ ██████╗
20:55:27.800- |REJECT| ██╔══██╗██╔══██╗██╔════╝██╔══██╗██╔═══██╗
20:55:27.806- |REJECT| ███████║██████╔╝█████╗ ██████╔╝██║ ██║
20:55:27.811- |REJECT| ██╔══██║██╔═══╝ ██╔══╝ ██╔══██╗██║ ██║
20:55:27.817- |REJECT| ██║ ██║██║ ███████╗██║ ██║╚██████╔╝
20:55:27.823- |REJECT| ╚═╝ ╚═╝╚═╝ ╚══════╝╚═╝ ╚═╝ ╚═════╝
20:55:27.828- |REJECT| ***************************************************************************
20:55:27.837-|REJECT| DRS Setup:
20:55:27.843-|REJECT|
20:55:27.844-|REJECT| DRS_ROOT: /apero/apero-drs-v0.7.290/apero
20:55:27.844-|REJECT| DRS_DATA_RAW: /data/spirou4/arlinks
20:55:27.844-|REJECT| DRS_DATA_REDUC: /data/spirou4/apero-data/offline290/red
20:55:27.844-|REJECT| DRS_DATA_WORKING: /data/spirou4/apero-data/offline290/tmp
20:55:27.844-|REJECT| DRS_CALIB_DB: /data/spirou4/apero-data/offline290/calib
20:55:27.845-|REJECT| DRS_TELLU_DB: /data/spirou4/apero-data/offline290/tellu
20:55:27.845-|REJECT| DRS_DATA_ASSETS: /data/spirou4/apero-data/offline290/asset
20:55:27.845-|REJECT| DRS_DATA_OUT: /data/spirou4/apero-data/offline290/out
20:55:27.845-|REJECT| DRS_DATA_MSG: /data/spirou4/apero-data/offline290/msg
20:55:27.845-|REJECT| DRS_DATA_RUN: /data/spirou4/apero-data/offline290/run
20:55:27.845-|REJECT| DRS_DATA_PLOT: /data/spirou4/apero-data/offline290/plot
20:55:27.846-|REJECT| DRS_DATA_OTHER: /data/spirou4/apero-data/offline290/other
20:55:27.846-|REJECT| LBL_PATH: /data/spirou4/apero-data/offline290/lbl
20:55:27.846-|REJECT| DRS_CONFIG: /apero/config/offline290/user_config.ini
20:55:27.846-|REJECT| DRS_CONFIG: /apero/config/offline290/user_constants.ini
20:55:27.846-|REJECT| DRS_CONFIG: apero.constants.default.default_config
20:55:27.846-|REJECT| DRS_CONFIG: apero.constants.default.default_keywords
20:55:27.847-|REJECT| DRS_CONFIG: config.instruments.spirou.default_config.py
20:55:27.847-|REJECT| DRS_CONFIG: config.instruments.spirou.default_keywords.py
20:55:27.847-|REJECT| DRS_CONFIG: core.instruments.default.default_constants.py
20:55:27.847-|REJECT| DRS_CONFIG: core.instruments.spirou.default_constants.py
20:55:27.847-|REJECT| DATABASE: MYSQL
20:55:27.847-|REJECT| DATABASE-calib: aperooffline290@localhost:calib_offline290_db
20:55:27.847-|REJECT| DATABASE-tellu: aperooffline290@localhost:tellu_offline290_db
20:55:27.848-|REJECT| DATABASE-findex: aperooffline290@localhost:findex_offline290_db
20:55:27.848-|REJECT| DATABASE-log: aperooffline290@localhost:log_offline290_db
20:55:27.848-|REJECT| DATABASE-astrom: aperooffline290@localhost:astrom_offline290_db
20:55:27.848-|REJECT| DATABASE-lang: aperooffline290@localhost:lang_offline290_db
20:55:27.848-|REJECT| DATABASE-reject: aperooffline290@localhost:reject_offline290_db
20:55:27.848-|REJECT| PRINT_LEVEL: all
20:55:27.849-|REJECT| LOG_LEVEL: all
20:55:27.849-|REJECT| DRS_PLOT: 0
20:55:27.854- |REJECT| ***************************************************************************
20:55:27.860-|REJECT| Arguments used:
20:55:27.866-|REJECT|
20:55:27.866-|REJECT| --IDENTIFIER: 3036202o,3036203o,3036204o,3036235o
20:55:27.866-|REJECT| --OBJNAME: None
20:55:27.866-|REJECT| --AUTOFILL: 1,1,1,OBJECT_grade5_24BQ03-Aug16
20:55:27.866-|REJECT| --TEST: True
20:55:27.872- |REJECT| ***************************************************************************
20:55:29.359-!!|REJECT| E[01-010-00001]: Unhandled error has occurred: Error <class 'AttributeError'>
20:55:29.360-!!|REJECT|
20:55:29.361-!!|REJECT| Traceback (most recent call last):
20:55:29.361-!!|REJECT| File "/apero/apero-drs-v0.7.290/apero/core/utils/drs_startup.py", line 433, in run
20:55:29.361-!!|REJECT| llmain = func(recipe, params)
20:55:29.361-!!|REJECT| File "/apero/apero-drs-v0.7.290/tools/bin/apero_reject.py", line 114, in main
20:55:29.361-!!|REJECT| drs_reject.add_file_reject(params, recipe, identifier)
20:55:29.362-!!|REJECT| File "/apero/apero-drs-v0.7.290/apero/tools/module/listing/drs_reject.py", line 79, in add_file_reject
20:55:29.362-!!|REJECT| google_sheet = gspd.spread.Spread(sheet_id)
20:55:29.362-!!|REJECT| File "/conda/miniconda3/envs/apero-env-v0.7.290/lib/python3.9/site-packages/gspread_pandas/spread.py", line 126, in init
20:55:29.362-!!|REJECT| self.client = Client(user, config, scope, creds)
20:55:29.363-!!|REJECT| File "/conda/miniconda3/envs/apero-env-v0.7.290/lib/python3.9/site-packages/gspread_pandas/client.py", line 96, in init
20:55:29.363-!!|REJECT| self._root = self._drive_request(file_id="root", params={"fields": "name,id"})
20:55:29.363-!!|REJECT| File "/conda/miniconda3/envs/apero-env-v0.7.290/lib/python3.9/site-packages/gspread_pandas/client.py", line 197, in _drive_request
20:55:29.363-!!|REJECT| res = self.request(method, url, params=params, json=data)
20:55:29.364-!!|REJECT| File "/conda/miniconda3/envs/apero-env-v0.7.290/lib/python3.9/site-packages/gspread_pandas/util.py", line 292, in request
20:55:29.364-!!|REJECT| return ClientV4.request(client, *args, kwargs)
20:55:29.364-!!|REJECT| AttributeError: type object 'Client' has no attribute 'request'
20:55:29.364-!!|REJECT|
20:55:29.474-|REJECT| ***************************************************************************
20:55:29.487-@!|REJECT| W[40-003-00005]: Recipe apero_reject has NOT been successfully completed
20:55:29.500-**|REJECT| ***************************************************************************
@njcuk9999
Copy link
Owner

This is the same error with the gspread function

20:55:29.364-!!|REJECT| return ClientV4.request(client, *args, kwargs)
20:55:29.364-!!|REJECT| AttributeError: type object 'Client' has no attribute 'request'

Can you check which conda environment you were in and your gspread-pandas and gspread functions?

@larnoldgithub
Copy link
Collaborator Author

larnoldgithub commented Oct 19, 2024 •
edited
Loading 

 offline290 Fri Oct 18 15:31:03 spdrs@apikipiha: ~
>>   pip show gspread
Name: gspread
Version: 6.1.3
Summary: Google Spreadsheets Python API
Home-page: 
Author: 
Author-email: Anton Burnashev <[email protected]>
License: 
Location: /conda/miniconda3/envs/apero-env-v0.7.290/lib/python3.9/site-packages
Requires: google-auth, google-auth-oauthlib
Required-by: gspread-pandas
 offline290 Fri Oct 18 15:31:47 spdrs@apikipiha: ~
>>   pip show gspread-pandas
Name: gspread-pandas
Version: 3.2.2
Summary: A package to easily open an instance of a Google spreadsheet and interact with worksheets through Pandas DataFrames.
Home-page: https://github.com/aiguofer/gspread-pandas
Author: Diego Fernandez
Author-email: [email protected]
License: BSD
Location: /conda/miniconda3/envs/apero-env-v0.7.290/lib/python3.9/site-packages
Requires: decorator, google-auth, google-auth-oauthlib, gspread, pandas
Required-by:

Another version 290 installed in a container and succefully used to add object to the obj_db

spdrs@32dcd4761891:/opt/apero-drs$ pip show gspread
Name: gspread
Version: 5.12.4
Summary: Google Spreadsheets Python API
Home-page: 
Author: 
Author-email: Anton Burnashev <[email protected]>
License: 
Location: /usr/local/lib/python3.9/dist-packages
Requires: google-auth, google-auth-oauthlib
Required-by: gspread-pandas
spdrs@32dcd4761891:/opt/apero-drs$ pip show gspread-pandas
Name: gspread-pandas
Version: 3.2.2
Summary: A package to easily open an instance of a Google spreadsheet and interact with worksheets through Pandas DataFrames.
Home-page: https://github.com/aiguofer/gspread-pandas
Author: Diego Fernandez
Author-email: [email protected]
License: BSD
Location: /usr/local/lib/python3.9/dist-packages
Requires: decorator, google-auth, google-auth-oauthlib, gspread, pandas
Required-by:

Ok so it seems the Offline 290 is using the 6.3.1. gspread and apero_reject.py fails. I used apero_reject.py with the version using gspread 5.1.4 and it worked, I could upload odometers to the db.

Is it possible to downgrade the gspread package from 6.3.1 to 5.1.4 in the offline environnement without corrupting my offline290 data and setup? @njcuk9999 @cusher I'm currently running the offline290, preprocessing of all SPI files that will be completed by tuesday or wednesday next week, so it's lilely better to wait before touching anything of the apero-env-v0.7.290 on apikipiha. :)

@njcuk9999
Copy link
Owner

What I would do personally is just delete the conda environment and re-create it (with the same name) using the requirements_developer.txt - you wont have to re-reduce anything or reinstall apero and it should all work the same (this assumes gspread is the only module that is wrong if others are there could be differences but I think you should be fine).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@njcuk9999 @larnoldgithub