Simple, fast, standalone pastebin service.
Backwards compatible with mkaczanowski/pastebin. If migrating chmod 820:820 or remove USER from Dockerfile. If reverting, note that you will not be able to decrypt pastes that were encrypted with this fork.
- Dependency bumps (incl. Rocket 0.5) and as a result the build is fixed
- Various UI changes
- Dark theme (and a softer Light theme)
- Pastes are directly opened after creation
- Expiry is shown more discreetly and in human friendly form
- Text wrapping
- Diff highlighting
- Line linking
- Passwordless encryption (or optionally with a password)
- Encryption reworked to use Web Crypto API and URI fragments
- Pastes are cookie protected and can only be removed by the creator
- & smaller changes
Whenever you need to share a code snippet, diff, logs, or a secret with another human being, the Pastebin service is invaluable. However, using public services such as pastebin.com, privnote.com, etc. should be avoided when you're sharing data that should be available only for a selected audience (i.e., your company, private network). Instead of trusting external providers, you could host your own Pastebin service and take ownership of all your data!
There are numerous Pastebin implementations out there, why would you implement another one?
While the other implementations are great, I couldn't find one that would satisfy my requirements:
- no dependencies - one binary is all I want, no python libs, ruby runtime magic, no javascript or external databases to setup
- storage - fast, lightweight, self-hosted key-value storage able to hold a lot of data.
- speed - it must be fast. Once deployed in a mid-sized company you can expect high(er) traffic with low latency expectations from users
- reliability - no one wants to fix things that should just work (and are that simple!)
- cheap - low-cost service that would not steal too much CPU time, thus adding up to your bill
- CLI + GUI - it must be easy to interface from both ends (but still, no deps!)
- other features:
- on-demand encryption
- syntax highlighting
- destroy after reading
- destroy after expiration date
This Pastebin implementation satisfies all of the above requirements!
This is a rust version of Pastebin service with rocksdb database as storage. In addition to previously mentioned features it's worth to mention:
- all-in-one binary - all the data, including css/javascript files are compiled into the binary. This way you don't need to worry about external dependencies, it's all within. (see: std::include_bytes)
- REST endpoint - you can add/delete pastes via standard HTTP client (ie. curl)
- RocksDB compaction filter - the expired pastes will be automatically removed by custom compaction filter
- flatbuffers - data is serialized with flatbuffers (access to serialized data without parsing/unpacking)
- GUI - the UI is a plain HTML with Bootstrap JS, jQuery and prism.js
- Encryption - password-protected pastes are AES-GCM encrypted/decrypted in the browser using the Web Crypto API
The default configuration enables only one plugin, this is syntax highlighting through prism.js. This should be enough for p90 of the users but if you need extra features you might want to use the plugin system (src/plugins).
To enable additional plugins, pass:
--plugins prism <custom_plugin_name>
Currently supported:
Note that browsers will block the Encrypt feature if a secure connection is not used. It is most convenient to manage TLS certs with a reverse proxy such as nginx, but --tls-certs and --tls-key options are also available.
The rocksdb dependency requires the clang compiler library (and a few minutes of compilation time).
cargo build --release
cargo run
This fork is not published to Docker Hub. Clone the repo to build the Docker image manually.
docker build --pull -t local/pastebin .
mkdir /var/lib/pastebin.db && chown -R 820:820 /var/lib/pastebin.db
docker run --init -p 127.0.0.1:8000:8000 -v /var/lib/pastebin.db:/pastebin.db local/pastebin --ui-line-numbers --address=0.0.0.0 --uri=https://paste.example.com
This example would make Pastebin available on port 8000 on the local machine. Nginx could then be used to proxy to port 80 externally while adding TLS and compression.
alias pastebin="curl -w '\n' -q -L --data-binary @- -o - http://localhost:8000/"
echo "hello World" | pastebin
http://localhost:8000/T9kGrI5aN
See REST API doc
I used k6.io for benchmarking the read-by-id HTTP endoint. Details:
- CPU: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz (4 CPUs, 8 threads = 16 rocket workers)
- Mem: 24 GiB
- Storage: NVMe SSD Controller SM981/PM981/PM983
- both client (k6) and server (pastebin) running on the same machine
$ cargo run --release
$ echo "Hello world" | curl -q -L -d @- -o - http://localhost:8000/
http://localhost:8000/0FWc4aaZXzf6GZBsuW4nv
$ cat > script.js <<EOL
import http from "k6/http";
export default function() {
let response = http.get("http://localhost:8000/<ID>");
};
EOL
$ docker pull loadimpact/k6
$ docker run --network=host -i loadimpact/k6 run --vus 5 -d 15s - <script.js
data_received..............: 206 MB 14 MB/s
data_sent..................: 1.6 MB 108 kB/s
http_req_blocked...........: avg=203.98µs min=59.63µs med=97.34µs max=280.74ms p(90)=142.01µs p(95)=161.72µs
http_req_connecting........: avg=60.48µs min=0s med=54.22µs max=9.57ms p(90)=79.67µs p(95)=93.6µs
http_req_duration..........: avg=4.75ms min=2.87ms med=4.66ms max=27.25ms p(90)=6.02ms p(95)=6.59ms
http_req_receiving.........: avg=69.16µs min=18.54µs med=59µs max=12.94ms p(90)=103µs p(95)=128.14µs
http_req_sending...........: avg=53.21µs min=18.11µs med=33.01µs max=5.82ms p(90)=62.68µs p(95)=166.06µs
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=4.62ms min=2.8ms med=4.54ms max=20.25ms p(90)=5.87ms p(95)=6.36ms
http_reqs..................: 14986 999.062363/s
iteration_duration.........: avg=4.98ms min=2.96ms med=4.8ms max=299.92ms p(90)=6.18ms p(95)=6.77ms
iterations.................: 14986 999.062363/s
vus........................: 5 min=5 max=5
vus_max....................: 5 min=5 max=5
docker run --network=host -i loadimpact/k6 run --vus 2 --stage 15s:4,15s:8,15s:16,15s:32 - <script.js
data_received..............: 654 MB 11 MB/s
data_sent..................: 5.9 MB 98 kB/s
http_req_blocked...........: avg=175.61µs min=56.88µs med=133.4µs max=168.74ms p(90)=175.38µs p(95)=219.87µs
http_req_connecting........: avg=86.58µs min=0s med=67.93µs max=34.36ms p(90)=95.52µs p(95)=116.89µs
http_req_duration..........: avg=13.29ms min=2.64ms med=8.3ms max=129.12ms p(90)=30.32ms p(95)=38.67ms
http_req_receiving.........: avg=223.36µs min=18.63µs med=71.91µs max=39.84ms p(90)=143.88µs p(95)=217.81µs
http_req_sending...........: avg=461.61µs min=17.23µs med=46.8µs max=62.26ms p(90)=335.01µs p(95)=857.64µs
http_req_tls_handshaking...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
http_req_waiting...........: avg=12.6ms min=2.59ms med=8ms max=106.26ms p(90)=28.61ms p(95)=36.55ms
http_reqs..................: 47699 794.982442/s
iteration_duration.........: avg=13.48ms min=2.75ms med=8.47ms max=185.95ms p(90)=30.55ms p(95)=38.91ms
iterations.................: 47699 794.982442/s
vus........................: 31 min=2 max=31
vus_max....................: 32 min=32 max=32
At first glance, the performance is pretty good. In the simplest scenario (5 concurrent clients), we can get up to 1000 rps with the p95 response time at 6.59 ms (14986 total requests made).
As we add more concurrent clients, the rps drops a bit (794 rps) but still provides a good timing (p95 38.67ms) with high throughput at 47699 request made in 15s window (3x compared to Test 1).
The CPU utilization is at 100% on every core available and the memory usage is stable at ~13 Mb RSS.
