P2G Stopped working - PaymentRequired error

[rearviewjohnny]

I have been using P2G to upload my workouts for months. Lately they workouts stopped appearing in my Garmin Connect. I ran PelotonToGarminConsole.exe manually, and got the following errors:

[14:58:36 ERR] Http Call Failed. PaymentRequired
[14:58:36 ERR] Failed to upload workouts. GUpload had exit code of -1
[14:58:36 ERR] GUpload returned an error code. Failed to upload workouts.

I have verified that my Peloton and Garmin passwords are correct and my Peloton membership is paid up.

[philosowaffle]

Looks like you're on the legacy way of uploading workouts to Garmin. Trying moving to the latest version of P2G and modifying your config file/settings to use upload strategy 2 (NativeImplV1).

[tg2012]

I haven't used it since Aug, had the same problem with v2.2.0, I downloaded v3.6.0 and get the same thing
I'm using the windows exe on Win11.

"[12:29:56 ERR] Http Call Failed. PaymentRequired
[12:29:56 ERR] Failed to upload workouts. GUpload had exit code of -1
[12:29:56 ERR] GUpload returned an error code. Failed to upload workouts.
Garmin.GarminUploadException: GUpload Exit Code: -1 - NativeImplV1 failed to upload workout ."

++++++++++++++++++++++++++++++++++++
I removed the email and pw:

"Garmin": {
"Email": "",
"Password": "",
"TwoStepVerificationEnabled": false,
"Upload": false,
"FormatToUpload": "fit",
"UploadStrategy": 2

[mgeis]

Helping a friend debug this one, we're seeing it as well. Verified we're using upload strategy 2 (in both the UI and in the settings db file). Tried it against multiple different docker tags/versions, same result across the board. The other verification that we're using the right upload strategy is that it's named in the error message in the stack trace, right before the error message from Garmin about payment required.

Failed to upload workouts to Garmin Connect. GUpload Exit Code: -1 - NativeImplV1 failed to upload workout /app/working/upload/72f3afd2bcfd4272a0a0ec770eeeb7c6_20_min_80s_Ride_with_Denis_Morton.fit, Call failed with status code 402 (Payment Required): POST

[philosowaffle]

Hmmm so that's atleast 4 people seeing this issue. I synced some workouts today and wasn't able to reproduce on my end.

The fact you're able to reproduce across multiple tags/versions indicates to me that Garmin or Cloudflare are doing something on their end to flag your requests. A very difficult thing to workaround.

There might be two more things to try:

1. If available, try hopping on a VPN before running P2G to see if changing the location of the network requests bypasses the problem
2. Can also try tweaking the UserAgent header that P2G will use when making requests to Garmin. You can do this by adding a new section to your config file that looks like the below. Just place it right before line 2. This can make your requests appear as if they are coming from a different device which occasionally makes a difference.

"Developer":
{
   "UserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/119.0"
},

[mgeis]

We moved the failing configuration to a different laptop, both on and off a VPN, and it worked for me. Then added the UserAgent entry to the config on the original laptop, and it failed again. Is there any persistent data (like a cookie, or something else that can be cleared out or removed) stored on the host machine? This is a docker gui installation.

[philosowaffle]

P2G caches some information in memory, however this is not persisted, meaning it is cleared every time P2G restarts. The only information P2G persists across restarts can be found in the mounted data directory.

Next two ideas I have are:

1. Change the log level from Information to Verbose. Then reproduce the issue. With this log level, P2G will log all of the HTTP request details to the log file (stripping out username/password first). There may be something interesting in this log, or atleast proof of which service (Cloudflare vs. Garmin) is kicking y'all out.
2. Start over with a clean install. Make a backup of the data directory and move it somewhere else. Then using docker, completely stop all running P2G containers and remove them + remove all images. Create a fresh instance of P2G with default config and with a new empty data directory. The downside is ya'll will need to reconfigure the settings again in the UI, but this would eliminate any chance of there being something cached or stored on the machine.

[q20]

Following your instructions, I have gone ahead and removed everything but my configuration file, changing the log level to verbose.
Running through a single (unsuccessful) sync attempt after recreating the container has generated a log file of almost 1MB in size. As I cannot be sure that no personally-identifiable information is stored in said file, I'm loath to upload it here. I am, of course, will to share it with @philosowaffle privately. Is there a preferred method for this?
Thanks!

[rearviewjohnny]

philosowaffle, I followed your original instructions to get the latest version of P2G ad update my config file, and it has been working fine. Thanks for your help.

[q20]

philosowaffle, I followed your original instructions to get the latest version of P2G ad update my config file, and it has been working fine. Thanks for your help.

I should note; adding the user agent did not help resolve my sync issues. Bummer! 😜

[tg2012]

I tried the user agent change and used a different ISP. then another laptop.
no change

[philosowaffle]

@q20 you can email me or add me to a file share of your choice with email: [email protected]

[q20]

@q20 you can email me or add me to a file share of your choice with email: [email protected]

Emailed. 👍

[q20]

Eesh... well, reply-to-all ballsed that up. Christ. Going to change passwords now. 😬

[philosowaffle]

@q20 Got your logs and read through them. One thing I noticed is you're still on 3.3.0-rc while the latest version of P2G is 3.6.1. I suspect you may still be on the older Docker tag that I accidentally changed a while back. The version of P2G you're on uses an older Upload url, there's a chance moving up to v3.6.1 may resolve your issue.

Another interesting thing I found while reading through your logs is that you did successfully authenticate with Garmin (my assumption so far was that folks were seeing this error during the authentication phase with Garmin). A few other requests to Garmin are also successful, so this error seems isolated to the Upload endpoint on Garmin.

I did some research and found some other occurrences of this error message on other projects. Documenting below for myself to reference.

• "Payment Required" to access garmin connect activities api? cyberjunky/python-garminconnect#44
  • Fix was to change the Upload Url, latest versions of P2G have the latest url
• Payment Required? pe-st/garmin-connect-export#54
• Download: 402 Client Error tcgoetz/GarminDB#109 (comment)
  • Fix was to add a special Header to the request, P2G already has this header
• 402 Client Error: Payment Required for url mkuthan/garmin-workouts#68
  • This issue implies the root cause is MFA, but I don't yet understand why they think its related
• Error message 402 (Payment Required) - any idea how to overcome this ? gsingers/garmin_golf#1
• Payment required kjkjava/garmin-connect-export#50
• 402 Error Downloading Files petergardfjall/garminexport#106
  • They are using a different upload url

2023-11-17 08:31:20.405 +00:00 [VRB] HTTP Request: POST - "https://connect.garmin.com/modern/proxy/upload-service/upload/.fit" 
- Cookie: GARMIN-SSO=1; GarminNoCache=true; GARMIN-SSO-GUID=<redacted>; GARMIN-SSO-CUST-GUID=<redacted>; __cflb=<redacted>; SESSIONID=<redacted>; ADRUM_BTa=<redacted>; SameSite=None; ADRUM_BT1=<redacted>
NK: NT
Origin: https://sso.garmin.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/50.0
 - Flurl.Http.Content.CapturedMultipartContent

2023-11-17 08:31:20.469 +00:00 [INF] HTTP Response: "PaymentRequired" - POST - "https://connect.garmin.com/modern/proxy/upload-service/upload/.fit" - Date: Fri, 17 Nov 2023 08:31:20 GMT
Connection: keep-alive
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=<redacted>"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}
Vary: Accept-Encoding
Server: cloudflare
CF-RAY: 82769de89959ca54-HAM
 - 

Note: latest version of P2G uses https://connectapi.garmin.com/upload-service/upload url for uploading activities.

[q20]

@q20 Got your logs and read through them. One thing I noticed is you're still on 3.3.0-rc while the latest version of P2G is 3.6.1. I suspect you may still be on the older Docker tag that I accidentally changed a while back. The version of P2G you're on uses an older Upload url, there's a chance moving up to v3.6.1 may resolve your issue.

Another interesting thing I found while reading through your logs is that you did successfully authenticate with Garmin (my assumption so far was that folks were seeing this error during the authentication phase with Garmin). A few other requests to Garmin are also successful, so this error seems isolated to the Upload endpoint on Garmin.

I did some research and found some other occurrences of this error message on other projects. Documenting below for myself to reference.

* ["Payment Required" to access garmin connect activities api? cyberjunky/python-garminconnect#44](https://github.com/cyberjunky/python-garminconnect/issues/44)
  
  * Fix was to change the Upload Url, latest versions of P2G have the latest url

* [Payment Required? pe-st/garmin-connect-export#54](https://github.com/pe-st/garmin-connect-export/issues/54)

* [Download: 402 Client Error tcgoetz/GarminDB#109 (comment)](https://github.com/tcgoetz/GarminDB/issues/109#issuecomment-785047086)
  
  * Fix was to add a special Header to the request, P2G already has this header

* [402 Client Error: Payment Required for url mkuthan/garmin-workouts#68](https://github.com/mkuthan/garmin-workouts/issues/68)
  
  * This issue implies the root cause is MFA, but I don't yet understand why they think its related

* [Error message 402 (Payment Required)  - any idea how to overcome this ? gsingers/garmin_golf#1](https://github.com/gsingers/garmin_golf/issues/1)

* [Payment required kjkjava/garmin-connect-export#50](https://github.com/kjkjava/garmin-connect-export/issues/50)

* [402 Error Downloading Files petergardfjall/garminexport#106](https://github.com/petergardfjall/garminexport/issues/106)
  
  * They are using a different [upload url](https://github.com/petergardfjall/garminexport/commit/9fcde6b7cde4a3ab4af80578c1a8cb6dd3f88950#diff-6093a27e9170fd8a464a72b64b04bca3857c7a77fab45b8dece651661fdfb129R498)

2023-11-17 08:31:20.405 +00:00 [VRB] HTTP Request: POST - "https://connect.garmin.com/modern/proxy/upload-service/upload/.fit" 
- Cookie: GARMIN-SSO=1; GarminNoCache=true; GARMIN-SSO-GUID=<redacted>; GARMIN-SSO-CUST-GUID=<redacted>; __cflb=<redacted>; SESSIONID=<redacted>; ADRUM_BTa=<redacted>; SameSite=None; ADRUM_BT1=<redacted>
NK: NT
Origin: https://sso.garmin.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/50.0
 - Flurl.Http.Content.CapturedMultipartContent

2023-11-17 08:31:20.469 +00:00 [INF] HTTP Response: "PaymentRequired" - POST - "https://connect.garmin.com/modern/proxy/upload-service/upload/.fit" - Date: Fri, 17 Nov 2023 08:31:20 GMT
Connection: keep-alive
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=<redacted>"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0.01,"report_to":"cf-nel","max_age":604800}
Vary: Accept-Encoding
Server: cloudflare
CF-RAY: 82769de89959ca54-HAM
 - 

Note: latest version of P2G uses https://connectapi.garmin.com/upload-service/upload url for uploading activities.

You've solved it!
Changing to:
philosowaffle/peloton-to-garmin:console-latest
was the key. I was using latest, unaware that tag was no longer pulling the most recent build.

Thank you for your time!