bitbucket hangs on git query to hg repo declared ambiguously #2092
Comments
|
@kevinburke Is your suggested path to have this function return an error instead of a possibly bad remote SCM, or to wrap the Looking in to it more it seems like |
There are cases that can result in `dep` trying to use `git ls-remote` to talk to a mercurial repository, which can have some really adverse effects due to poor connection handling characteristics of `git ls-remote`. This tries to improve that situation with a timeout. If the git command cannot connect to the remote SCM, it takes 2 minutes for the connection to time out if it's unable to establish the TCP connection. It then tries the next IP, which will also take 2 minutes to time out, and so on and so forth. If `dep` hands `git ls-remote` a Bitbucket repository that's `hg` backed, `git ls-remote` fails to establish the connection and retries all IPs until finally failing. If there are 6 IPs returned, this will take 12 minutes. If you're running a `dep` ensure that's trying to do a lot of different actions, this can add up very quickly. Looking in to the `git ls-remote` manpage there does not seem to be a straightforward way to configure it to fail a bit more aggressively (such as after 2 failed attempts with a 15 second timeout). This change wraps calls to `git ls-remote` in a `context.WithTimeout` that gives it 30 seconds to run. This is a completely arbitrary value and may need tuning, or even to become configurable. Updates #2092
|
@kevinburke separate from that linked PR above (#2093), I'm thinking of tackling the TODO of hitting the Bitbucket REST API to just make this not make any guesses. I'm thinking of writing a I'd need to dig in to callers of What are your thoughts? |
|
I am also wondering what the real world consequences would be of just erroring if you specify this in a vague way. What percentage of BB repos are specified this way with no extra hints, and what percentage of them are Git vs Mercurial? |
|
What you described about a Bitbucket API seems fine... we can optimize it after the fact but just adding query support would be helpful. |
|
Going to close this out, I think it's fixed now. |
If you declare a repo source as eg "bitbucket.org/mattfarina/testhgrepo" it can be either a Git or Hg repo. We guess it is Git in the bitbucketDeducer in gps/deduce.go, but if we guess incorrectly, we can end up issuing a
git ls-remotecall to a Mercurial repository and hanging forever.There is this (wise) TODO in the code
In the meantime I think we should just return an error with a clear message instead of hanging forever.
This issue is the reason the test suite currently panics with a timeout.
Updates #2091
Updates #2071
The text was updated successfully, but these errors were encountered: