diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/assets/utreexo.png b/docs/assets/utreexo.png new file mode 100644 index 00000000..20d9e427 Binary files /dev/null and b/docs/assets/utreexo.png differ diff --git a/docs/contact.md b/docs/contact.md new file mode 100644 index 00000000..6851a062 --- /dev/null +++ b/docs/contact.md @@ -0,0 +1,15 @@ +# Contact + +## IRC + +- [irc.libera.chat](irc://irc.libera.chat), channel `#utreexo` + +## Discussions + +- [GitHub Discussions](https://github.com/mit-dci/utreexo/discussions) can be used to ask and answer questions and share updates. +- Weekly meetings on https://meet.jit.si/ + +## Issue Tracker + +The [integrated github issue tracker](https://github.com/mit-dci/utreexo/issues) +is used for this project. diff --git a/docs/readme.md b/docs/readme.md new file mode 100644 index 00000000..bfae97d5 --- /dev/null +++ b/docs/readme.md @@ -0,0 +1,16 @@ +# utreexo + +utreexo is a novel hash-based dynamic accumulator, which allows the millions of unspent outputs to be represented under a kilobyte -- small enough to be written on a sheet of paper. There is no trusted setup or loss of security; instead, the burden of keeping track of funds is shifted to the owner of those funds. + +Check out the ePrint paper here: https://eprint.iacr.org/2019/611 + +Currently, transactions specify inputs and outputs, and verifying an input requires you to know the whole state of the system. With utreexo, the holder of funds maintains a proof that the funds exist and provides that proof at spending time to the other nodes. These proofs represent the utreexo model’s main downside; they present an additional data transmission overhead that allows a much smaller state. + +utreexo pushes the costs of maintaining the network to the right place: an exchange creating millions of transactions may need to maintain millions of proofs, while a personal account with only a few unspent outputs will only need to maintain a few kilobytes of proof data. utreexo also provides a long-term scalability solution as the accumulator size grows very slowly with the increasing size of the underlying set (the accumulator size is logarithmic with the set size) + +# Documentation + +- [Installation](https://github.com/mit-dci/utreexo/blob/master/docs/installation.md) +- [Style Guidelines](https://github.com/mit-dci/utreexo/blob/master/docs/style.md) +- [Contributing Guidelines](https://github.com/mit-dci/utreexo/blob/master/docs/contributing.md) +- [Contact](https://github.com/mit-dci/utreexo/blob/master/docs/contact.md) diff --git a/readme.md b/readme.md index deb500ba..3fc8dfe8 100644 --- a/readme.md +++ b/readme.md @@ -1,139 +1,47 @@ -# Utreexo +# utreexo -A dynamic hash based accumulator designed for the Bitcoin UTXO set - -Check out the ePrint paper here: [https://eprint.iacr.org/2019/611](https://eprint.iacr.org/2019/611) +![utreexo cover image](/docs/assets/utreexo.png) -Currently under active development. If you're interested and have questions, checkout #utreexo on irc.libera.chat. +utreexo is a novel hash-based dynamic accumulator, which allows the millions of unspent outputs to be represented under a kilobyte -- small enough to be written on a sheet of paper. There is no trusted setup or loss of security; instead, the burden of keeping track of funds is shifted to the owner of those funds. -Logs for libera are [here](http://gnusha.org/utreexo/) +Check out the ePrint paper here: https://eprint.iacr.org/2019/611 ---- -## Importing Utreexo -The raw accumulator functions are in package accumulator. This is a general accumulator and is not bitcoin specific. For Bitcoin specific accumulator, look to the csn and bridgenode packages. +Currently, transactions specify inputs and outputs, and verifying an input requires you to know the whole state of the system. With utreexo, the holder of funds maintains a proof that the funds exist and provides that proof at spending time to the other nodes. These proofs represent the utreexo model’s main downside; they present an additional data transmission overhead that allows a much smaller state. -## Walkthrough for testing out Utreexo nodes - -Here's how to get utreexo running to test out what it can do. This currently is testing/research level code and should not be expected to be stable or secure. But it also should work, and if it doesn't please report bugs! - -To demonstrate utreexo we went with a client-server model. We have made prebuild binaries to run utreexo on Linux, Mac and Windows available here: https://github.com/mit-dci/utreexo/releases but you can also build from source. - -### Client - -#### Build from source -``` -$ go get github.com/mit-dci/utreexo -$ cd ~/go/src/github.com/mit-dci/utreexo/cmd/utreexoclient -$ go build -``` - -#### Run -Running the client can take a couple of hours (There are still lots of performance optimisations to be done to speed things up). -The client downloads blocks with inclusion proofs from the server and validates them. -``` -$ ./utreexoclient -[the client is able to resume from where it left off. Use ctrl+c to stop it.] -[To resume, just do `/utreexoclient` again] -``` - -*There is a `host` flag to specify a different server and a `watchaddr` flag to specify the address that you want to watch. To view all options use the `help` flag* - -If you pause the client it will create the `pollardFile` which holds the accumulator roots. As an experiment you can copy this file to a different machine and resume the client at the height it was paused. - -### Server -To try utreexo you must run the utreexo server. The instructions to run the server are given below. - -#### Build from source -``` -$ go get github.com/mit-dci/utreexo -$ cd ~/go/src/github.com/mit-dci/utreexo/cmd/utreexoserver -$ go build -``` - -#### Run - -If you want to run a server you will need the Bitcoin blockchain. Try testnet as it's smaller. (you can get Bitcoin Core from http://github.com/bitcoin/bitcoin or https://bitcoin.org/en/download) - -``` -[ ...install bitcoin core ] -$ echo "testnet=1" > ~/.bitcoin/bitcoin.conf -$ bitcoind --daemon -[wait for testnet to sync] -$ du -h ~/.bitcoin/testnet3/blocks/ - 214M ~/.bitcoin/testnet3/blocks/index - 24G ~/.bitcoin/testnet3/blocks/ -[OK looks like it's there] -$ bitcoin-cli stop -``` -**Note:** bitcoind has to be stopped before running the server. - -The server should take a few hours. It does two things. First, it goes through the blockchain, maintains the full merkle forest, and saves proofs for each block to disk. Second, it saves each TXO and height with LevelDB to make a TXO time-to-live (basically how long each TXO lasts until it is spent) for caching purposes. This is what the bridge node and archive node would do in a real node. - -``` -$ cd ~/.bitcoin/testnet3/blocks -$ ./utreexoserver #path probably differs on your system -[... takes time and builds block proofs] -[the server is able to resume from where it left off. Use ctrl+c to stop it.] -[To resume, just do `./cmd genproofs -net=testnet` again] -``` - -After the server has generated the proofs, it will start a local server to serve the blocks to clients. - -**Note**: your folders or filenames might be different, but this should give you the idea and work on default Linux/golang setups. If you've tried this and it doesn't work and you'd like to help out, you can either fix the code or documentation so that it works and make a pull request, or open an issue describing what doesn't work. - -### Windows walkthrough -
    -
  1. -To run Utreexo, download the Bitcoin core here (includes both testnet and main chain): https://bitcoin.org/en/download Open the respective application; the testnet application should appear as a green bitcoin and the main bitcoin core application should appear as an orange bitcoin. -From here, synchronize with the blockchain; this takes around 2-5 hours on the testnet and up to a day using Bitcoin core. -
    2. -
  2. -Install Go in your pc and get it working on your compiler/IDE. The guide below will refer to installing Go on VSCode. -
    3. -
  3. - -Get the Utreexo Code from ```github.com/mit-dci/utreexo``` -
    4. -
  4. - -Build and generate proofs (server) by running the following command where $USER is your username -``` -go build bridgeserver.go -bridgeserver -datadir=C:\Users\$USER\AppData\Roaming\Bitcoin\testnet3\blocks\ -``` - **If this fails, the command run was interupted or failed. To relaunch, delete the folders in the Utreexo\utreexoserver\utree folder: forestdata, offsetdata, pollarddata, proofdata, testnet-ttlbd** - -
    5. -
  5. - - For a debugging session, in launch.json in VSCode, create a configuration as follows. The - ```"-datadir"``` argument should point to the folder where Bitcoin was downloaded in step 1. - -``` -{ - "version": "0.2.0", - "configurations": [ - { - "name": "Launch", - "type": "go", - "request": "launch", - "mode": "auto", - "program": "${fileDirname}", - "env": {}, - "args": ["-datadir=C:\\Users\\$USER\\AppData\\Roaming\\Bitcoin\\testnet3\\blocks\\"] - } - ] -} -``` -
    6. - -
  6. - -Finally run Utreexo client from **command line** using the following. Make sure that the server has finished running before running this command. - -``` -go build csnclient.go -C:\utreexo\csnclient -datadir=C:\Users\admin\AppData\Roaming\Bitcoin\testnet3\blocks\ -``` -
    7. -
+utreexo pushes the costs of maintaining the network to the right place: an exchange creating millions of transactions may need to maintain millions of proofs, while a personal account with only a few unspent outputs will only need to maintain a few kilobytes of proof data. utreexo also provides a long-term scalability solution as the accumulator size grows very slowly with the increasing size of the underlying set (the accumulator size is logarithmic with the set size) + +## Development Process + +At the moment, this repository holds the code for the accumulator and proof-of-concept implementations for the compact state node and the bridge node. The accumulator package is currently used in [mit-dci/utcd](https://github.com/mit-dci/utcd) in an alternative full node bitcoin implementation written in Go. + +The c++ implementation of utreexo is at [mit-dci/libutreexo](https://github.com/mit-dci/libutreexo) and the package is currently being used in [dergoegge/bitcoin](https://github.com/dergoegge/bitcoin) in an full node bitcoin-core implementation. + +## Documentation + +It is located in the [docs](https://github.com/mit-dci/utreexo/docs) folder. + +## IRC + +Currently under active development. If you're interested and have questions, checkout #utreexo on irc.libera.chat. + +Logs for libera are [here](https://gnusha.org/utreexo/) + +## Contributions + +- Feel Free to Open a PR/Issue/Discussion for any features/bug(s)/question(s). +- Make sure you follow the contributing guidelines [here]()! + +## Resources + +### Blogs + +- https://dci.mit.edu/utreexo +- [Utreexo demonstration release](https://medium.com/mit-media-lab-digital-currency-initiative/utreexo-demonstration-release-a0d87506fd70) +- [Utreexo demo release 0.2](https://medium.com/mit-media-lab-digital-currency-initiative/utreexo-demo-release-0-2-ac40a1223a38) +- [Utreexo — A scaling solution](https://medium.com/@kcalvinalvinn/eli5-utreexo-a-scaling-solution-9531aee3d7ba) + +### Videos/Podcasts + +- [MIT Bitcoin Expo 2019 - Utreexo: Reducing Bitcoin Nodes to 1 Kilobyte](https://www.youtube.com/watch?v=edRun-6ubCc) +- [Tadge Dryja on Scaling Bitcoin With Utreexo](https://www.youtube.com/watch?v=2Kg1Cij3w20) +- [#1 Tadge Dryja, Digital Currency Initiative: uTreeXO and Bootstrapping Bitcoin Upgrades](https://www.youtube.com/watch?v=-MlKZ_bFLNk)