feat: create DNS library (#141)

.github/workflows/test.yml

@@ -51,7 +51,9 @@ jobs:
         working-directory: ${{ github.workspace }}/x
 
       - name: Test SDK
-        run: go test -v -race -bench '.' ./... -benchtime=100ms
+        # Enable nettests, which executes external network requests.
+        run: go test -v -race -bench '.' ./... -benchtime=100ms -tags nettest
 
       - name: Test X
-        run: go test -C x -v -race -bench '.' ./... -benchtime=100ms
+        # Enable nettests, which executes external network requests.
+        run: go test -C x -v -race -bench '.' ./... -benchtime=100ms -tags nettest

CONTRIBUTING.md

@@ -42,25 +42,29 @@ In Go you can compile for other target operating system and architecture by spec
   <summary>Examples</summary>
 
 MacOS example:
-```
+
+```console
 % GOOS=darwin go build -C x -o ./bin/ ./examples/test-connectivity 
 % file ./x/bin/test-connectivity 
 ./x/bin/test-connectivity: Mach-O 64-bit executable x86_64
 ```
 
 Linux example:
-```
+
+```console
 % GOOS=linux go build -C x -o ./bin/ ./examples/test-connectivity 
 % file ./x/bin/test-connectivity                      
 ./x/bin/test-connectivity: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=n0WfUGLum4Y6OpYxZYuz/lbtEdv_kvyUCd3V_qOqb/CC_6GAQqdy_ebeYTdn99/Tk_G3WpBWi8vxqmIlIuU, with debug_info, not stripped
 ```
 
 Windows example:
-```
+
+```console
 % GOOS=windows go build -C x -o ./bin/ ./examples/test-connectivity 
 % file ./x/bin/test-connectivity.exe 
 ./x/bin/test-connectivity.exe: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows
 ```
+
 </details>
 
 ## Running Linux binaries
@@ -72,41 +76,49 @@ To run Linux binaries you can use a Linux container via [Podman](https://podman.
   <summary>Instructions</summary>
 
 [Install Podman](https://podman.io/docs/installation) (once). On macOS:
+
 ```sh
 brew install podman
 ```
 
 Create the podman service VM (once) with the [`podman machine init` command](https://docs.podman.io/en/latest/markdown/podman-machine-init.1.html):
+
 ```sh
 podman machine init
 ```
 
 Start the VM with the [`podman machine start` command](https://docs.podman.io/en/latest/markdown/podman-machine-start.1.html), after every time it is stopped:
+
 ```sh
 podman machine start
-``` 
+```
 
 You can see the VM running with the [`podman machine list` command](https://docs.podman.io/en/latest/markdown/podman-machine-list.1.html):
-```
+
+```console
 % podman machine list
 NAME                     VM TYPE     CREATED        LAST UP            CPUS        MEMORY      DISK SIZE
 podman-machine-default*  qemu        3 minutes ago  Currently running  1           2.147GB     107.4GB
 ```
 
 When you are done with development, you can stop the machine with the [`podman machine stop` command](https://docs.podman.io/en/latest/markdown/podman-machine-stop.1.html):
+
 ```sh
 podman machine stop
 ```
+
 </details>
 
 ### Run
 
 The easiest way is to run a binary is to use the [`go run` command](https://pkg.go.dev/cmd/go#hdr-Compile_and_run_Go_program) directly with the `-exec` flag and our convenience tool `run_on_podman.sh`:
+
 ```sh
 GOOS=linux go run -C x -exec "$(pwd)/run_on_podman.sh" ./examples/test-connectivity
 ```
 
 It also works with the [`go test` command](https://pkg.go.dev/cmd/go#hdr-Test_packages):
+
 ```sh
 GOOS=linux go test -exec "$(pwd)/run_on_podman.sh"  ./...
 ```
@@ -115,12 +127,14 @@ GOOS=linux go test -exec "$(pwd)/run_on_podman.sh"  ./...
   <summary>Details and direct invocation</summary>
 
 The `run_on_podman.sh` script uses the [`podman run` command](https://docs.podman.io/en/latest/markdown/podman-run.1.html) and a minimal ["distroless" container image](https://github.com/GoogleContainerTools/distroless) to run the binary you want:
+
 ```sh
 podman run --arch $(uname -m) --rm -it -v "${bin}":/outline/bin gcr.io/distroless/static-debian11 /outline/bin "$@"
 ```
 
 You can also use `podman run` directly to run a pre-built binary:
-```
+
+```console
 % podman run --rm -it -v ./x/bin:/outline gcr.io/distroless/static-debian11 /outline/test-connectivity
 Usage of /outline/test-connectivity:
   -domain string
@@ -153,14 +167,16 @@ This is not the same as a real Windows environment, so make sure you test on act
 
 Follow the instructions at https://wiki.winehq.org/Download.
 
-On macOS: 
-```
+On macOS:
+
+```sh
 brew tap homebrew/cask-versions
 brew install --cask --no-quarantine wine-stable
 ```
 
 After installation, `wine64` should be on your `PATH`. Check with `wine64 --version`:
-```
+
+```sh
 wine64 --version
 ```
 
@@ -177,6 +193,17 @@ GOOS=windows go run -C x -exec "wine64" ./examples/test-connectivity
 ```
 
 For tests:
+
 ```sh
 GOOS=windows go test -exec "wine64"  ./...
 ```
+
+# Tests with external network dependencies
+
+Some tests are implemented talking to external services. That's undesirable, but convenient.
+We started tagging them with the `nettest` tag, so they don't run by default. To run them, you need to specify `-tags nettest`, as done in our CI.
+For example:
+
+```sh
+go test -v -race -bench '.' ./... -benchtime=100ms -tags nettest
+```

README.md

@@ -22,12 +22,30 @@ The Outline SDK allows you to:
 |:-:|:-:|:-:|
 | Supports Android, iOS, Windows, macOS and Linux. | Field-tested in the Outline Client and Server, helping millions to access the internet under harsh conditions. | Designed for modularity and reuse, allowing you to craft custom transports. |
 
+### Interoperable and Reusable
+
+The Outline SDK is built upon a simple basic concepts, defined as interoperable interfaces that allow for composition and easy reuse.
+
+**Connections** enable communication between two endpoints over an abstract transport. There are two types of connections:
+  - `transport.StreamConn`: stream-based connection, like TCP and the `SOCK_STREAM` Posix socket type.
+  - `transport.PacketConn`: datagram-based connection, like UDP and the `SOCK_DGRAM` Posix socket type. We use "Packet" instead of "Datagram" because that is the convention in the Go standard library.
+
+Connections can be wrapped to create nested connections over a new transport. For example, a `StreamConn` could be over TCP, over TLS over TCP, over HTTP over TLS over TCP, over QUIC, among oter options.
+
+**Dialers** enable the creation of connections given a host:port address while encapsulating the underlying transport or proxy protocol. The `StreamDialer` and `PacketDialer` types create `StreamConn` and `PacketConn` connections, respectively, given an address. Dialers can also be nested. For example, a TLS Stream Dialer can use a TCP dialer to create a `StreamConn` backed by a TCP connection, then create a TLS `StreamConn` backed by the TCP `StreamConn`. A SOCKS5-over-TLS Dialer could use the TLS Dialer to create the TLS `StreamConn` to the proxy before doing the SOCKS5 connection to the target address.
+
+**Resolvers** (`dns.Resolver`) enable the answering of DNS questions while encapsulating the underlying algorithm or protocol. Resolvers are primarily used to map domain names to IP addresses.
+
+
 ### Bypass DNS-based Blocking
 
-We are working on a [new DNS library](https://github.com/Jigsaw-Code/outline-sdk/pull/141) that will let people bypass DNS-based blocking by using alternative
-resolvers and ports, and encrypted DNS (DNS-over-HTTPS and DNS-over-TLS).
+The Outline SDK offers two types of strategies for evading DNS-based blocking: resillient DNS or address override.
 
-Meanwhile, you can force a specific address resolution with an override, by implementing your own dialer, or leveraging the `override` config from [x/config](https://pkg.go.dev/github.com/Jigsaw-Code/outline-sdk/x/config).
+- The [dns](https://pkg.go.dev/github.com/Jigsaw-Code/outline-sdk/dns) package can replace the resolution based on the system resolver with more resillient options:
+  - Encrypted DNS over HTTPS (DoH) or TLS (DoT)
+  - Alternative hosts and ports for UDP and TCP resolvers, making it possible to use resolvers that are not blocked.
+- The `override` config from [x/config](https://pkg.go.dev/github.com/Jigsaw-Code/outline-sdk/x/config) with a `host` option can be used to force a specific address,
+  or you can implement your own Dialer that can map addresses.
 
 ### Bypass SNI-based Blocking
 

dns/doc.go

@@ -0,0 +1,41 @@
+// Copyright 2023 Jigsaw Operations LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     https://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+/*
+Package dns provides utilities to interact with the Domain Name System (DNS).
+
+The [Domain Name System] (DNS) is responsible for mapping domain names to IP addresses.
+Because domain resolution gatekeeps connections and is predominantly done in plaintext, it is [commonly used
+for network-level filtering].
+
+# Transports
+
+The main concept in this library is that of a [Resolver], which allows code to query the DNS. Different implementations are provided
+to perform DNS resolution over different transports:
+
+  - [DNS-over-UDP]: the standard mechanism of querying resolvers. Communication is done in plaintext, using port 53.
+  - [DNS-over-TCP]: alternative to UDP that allows for more reliable delivery and larger responses, but requires establishing a connection. Communication is done in plaintext, using port 53.
+  - [DNS-over-TLS] (DoT): uses the TCP protocol, but over a connection encrypted with TLS. Is uses port 853, which
+    makes it very easy to block using the port number, as no other protocol is assigned to that port.
+  - [DNS-over-HTTPS] (DoH): uses HTTP exchanges for querying the resolver and communicates over a connection encrypted with TLS. It uses
+    port 443. That makes the DoH traffic undistinguishable from web traffic, making it harder to block.
+
+[Domain Name System]: https://datatracker.ietf.org/doc/html/rfc1034
+[commonly used for network-level filtering]: https://datatracker.ietf.org/doc/html/rfc9505#section-5.1.1
+[DNS-over-UDP]: https://datatracker.ietf.org/doc/html/rfc1035#section-4.2.1
+[DNS-over-TCP]: https://datatracker.ietf.org/doc/html/rfc7766
+[DNS-over-TLS]: https://datatracker.ietf.org/doc/html/rfc7858
+[DNS-over-HTTPS]: https://datatracker.ietf.org/doc/html/rfc8484
+*/
+package dns