Skip to content
This repository has been archived by the owner on Sep 9, 2020. It is now read-only.

Commit

Permalink
Add a context.WithTimeout for running git ls-remote
Browse files Browse the repository at this point in the history
There are cases that can result in `dep` trying to use `git ls-remote` to talk
to a mercurial repository, which can have some really adverse effects due to
poor connection handling characteristics of `git ls-remote`. This tries to
improve that situation with a timeout.

If the git command cannot connect to the remote SCM, it takes 2 minutes for the
connection to time out if it's unable to establish the TCP connection. It then
tries the next IP, which will also take 2 minutes to time out, and so on and so
forth.

If `dep` hands `git ls-remote` a Bitbucket repository that's `hg` backed, `git
ls-remote` fails to establish the connection and retries all IPs until finally
failing. If there are 6 IPs returned, this will take 12 minutes. If you're
running a `dep` ensure that's trying to do a lot of different actions, this can
add up very quickly.

Looking in to the `git ls-remote` manpage there does not seem to be a
straightforward way to configure it to fail a bit more aggressively (such as
after 2 failed attempts with a 15 second timeout).

This change wraps calls to `git ls-remote` in a `context.WithTimeout` that gives
it 30 seconds to run. This is a completely arbitrary value and may need tuning,
or even to become configurable.

Updates #2092
  • Loading branch information
theckman committed Jan 22, 2019
1 parent a05c953 commit 4bf6f4b
Showing 1 changed file with 18 additions and 1 deletion.
19 changes: 18 additions & 1 deletion gps/vcs_source.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"path/filepath"
"regexp"
"strings"
"time"

"github.com/Masterminds/semver"
"github.com/golang/dep/gps/pkgtree"
Expand Down Expand Up @@ -217,7 +218,21 @@ func (*gitSource) existsCallsListVersions() bool {
func (s *gitSource) listVersions(ctx context.Context) (vlist []PairedVersion, err error) {
r := s.repo

cmd := commandContext(ctx, "git", "ls-remote", r.Remote())
// XXX(theckman): if git ls-remote is given a Bitbucket mercurial repository
// by mistake, this command can take a very long time to error. This is
// because it's hitting a connection timeout condition on each connection
// attempt, resulting in it taking 120s * n (connections) to return. The
// number of connection attempts seems to be tied to the number of IPs
// returned for the DNS record. If this is encountered during an ensure
// resolution, the command can run for a pretty long period of time as it
// tries to check multiple versions.
//
// There does not seem to be a way to configure its connection timeout or
// retry limit, so we need to set a shorter timeout so this doesn't hang
// forever.
cctx, cancel := context.WithTimeout(ctx, 30*time.Second)

cmd := commandContext(cctx, "git", "ls-remote", r.Remote())
// We want to invoke from a place where it's not possible for there to be a
// .git file instead of a .git directory, as git ls-remote will choke on the
// former and erroneously quit. However, we can't be sure that the repo
Expand All @@ -230,7 +245,9 @@ func (s *gitSource) listVersions(ctx context.Context) (vlist []PairedVersion, er
}
// Ensure no prompting for PWs
cmd.SetEnv(append([]string{"GIT_ASKPASS=", "GIT_TERMINAL_PROMPT=0"}, os.Environ()...))

out, err := cmd.CombinedOutput()
cancel()
if err != nil {
return nil, errors.Wrap(err, string(out))
}
Expand Down

0 comments on commit 4bf6f4b

Please sign in to comment.