Add a context.WithTimeout for running git ls-remote
#2093
Conversation
There are cases that can result in `dep` trying to use `git ls-remote` to talk to a mercurial repository, which can have some really adverse effects due to poor connection handling characteristics of `git ls-remote`. This tries to improve that situation with a timeout. If the git command cannot connect to the remote SCM, it takes 2 minutes for the connection to time out if it's unable to establish the TCP connection. It then tries the next IP, which will also take 2 minutes to time out, and so on and so forth. If `dep` hands `git ls-remote` a Bitbucket repository that's `hg` backed, `git ls-remote` fails to establish the connection and retries all IPs until finally failing. If there are 6 IPs returned, this will take 12 minutes. If you're running a `dep` ensure that's trying to do a lot of different actions, this can add up very quickly. Looking in to the `git ls-remote` manpage there does not seem to be a straightforward way to configure it to fail a bit more aggressively (such as after 2 failed attempts with a 15 second timeout). This change wraps calls to `git ls-remote` in a `context.WithTimeout` that gives it 30 seconds to run. This is a completely arbitrary value and may need tuning, or even to become configurable. Updates #2092
|
We should definitely prevent this from ever getting a mercurial repo in the first place, but it seems like specifying some reasonable timeouts is likely a good idea. |
|
Might be good to hold on this for a little based on feedback from Sam. |
|
For the record, we get a different (better) error message now. |
|
Sam indicated we'd had issues with setting timeouts in the past... I don't know, though, this feels pretty specific, and doesn't require a big response. 2 minutes should be enough time? |
|
Can we set this to, say, 30 minutes or an hour? Even a very slow git checkout should complete in 30 minutes, and there are people who are reporting dep hangs for 4 hours or longer so it seems like even a crazy large default would be better than nothing. |
|
Dep was officially deprecated earlier this year, and the proposal to archive this repository was accepted. As such, I'm closing outstanding issues before archiving the repository. For any further comments, please use the proposal thread on the Go issue tracker. Thanks! |
There are cases that can result in
deptrying to usegit ls-remoteto talkto a mercurial repository, which can have some really adverse effects due to
poor connection handling characteristics of
git ls-remote. This tries toimprove that situation with a timeout.
If the git command cannot connect to the remote SCM, it takes 2 minutes for the
connection to time out if it's unable to establish the TCP connection. It then
tries the next IP, which will also take 2 minutes to time out, and so on and so
forth.
If
dephandsgit ls-remotea Bitbucket repository that'shgbacked,git ls-remotefails to establish the connection and retries all IPs until finallyfailing. If there are 6 IPs returned, this will take 12 minutes. If you're
running a
depensure that's trying to do a lot of different actions, this canadd up very quickly.
Looking in to the
git ls-remotemanpage there does not seem to be astraightforward way to configure it to fail a bit more aggressively (such as
after 2 failed attempts with a 15 second timeout).
This change wraps calls to
git ls-remotein acontext.WithTimeoutthat givesit 30 seconds to run. This is a completely arbitrary value and may need tuning,
or even to become configurable.
Updates #2092