You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Apr 14, 2020. It is now read-only.
I've finished the Go port of SCV and have started the migration by merging into this repo. There's been about 100+ commits coming in all at once. The migration should take about 2-3 days to complete as I upgrade the CC's on AWS. I need to make some additional modifications on the CC as well. @mpiercy we might want to do this together.
Key features:
No longer dependent on redis/LUA to hack together custom queries. This was probably the biggest reason why I made the switch. You have to understand python's GIL at one point became the bane of my existence, and to allow multi-processing to work, you had to coordinate through an external service (such as redis) and fork your processes. As time went on, I needed more than the basic pipelining operations available in redis, and had to basically hack using LUA (think CUDA's equivalent of JIT).
Insanely parallelizable. The single-nature nature of redis meant that all memory-related queries were serialized. Before, this effectively meant that only one stream could be modified at any given time. The new ReadWrite mutex based Go code means that we had N-way parallelism, where N is the number of streams. This is an insane speed-up. It's really really fast.
Go server is effectively stateless, meaning we don't need to rely on the redis cache to persist across sessions. The server can self recover directly from the MongoDB and what ever binary data exists on disk.
Globally accessible stream information inside the DB. Originally, because of sharding and perhaps a bad design decision on my part, stream information was only accessible on SCVs. If an SCV goes off-line the state of the stream is lost. In this case, stream presistence is enabled through MongoDB, though in a slightly dirty state perhaps.
Go binaries are statically built, and will be made available soon. No more cmake, scons, make, pip or any of that garbage (for the SCV)! Go binaries are compiled to bytecode directly, cross-platform compilation is insanely easy.
Why Go and not java? Because of goroutines. Each HTTP request essentially exists in its own goroutine, and the overhead of goroutines in Go is a very very tiny fraction of that of an OS thread you'd find in C++/Java. Python's tornado is different, as it has its own built-in async event poll.
In short, Go is amazing.
The text was updated successfully, but these errors were encountered:
I should note that while the SCV is done, there is still some integration work to be done. I'm working on fixing the integration tests and siegetank tests.
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Sorry for the delay.
I've finished the Go port of SCV and have started the migration by merging into this repo. There's been about 100+ commits coming in all at once. The migration should take about 2-3 days to complete as I upgrade the CC's on AWS. I need to make some additional modifications on the CC as well. @mpiercy we might want to do this together.
Key features:
Why Go and not java? Because of goroutines. Each HTTP request essentially exists in its own goroutine, and the overhead of goroutines in Go is a very very tiny fraction of that of an OS thread you'd find in C++/Java. Python's tornado is different, as it has its own built-in async event poll.
In short, Go is amazing.
The text was updated successfully, but these errors were encountered: