Skip to content

Commit

Permalink
Update README.
Browse files Browse the repository at this point in the history
  • Loading branch information
kurtschelfthout committed Dec 9, 2023
1 parent 18cd283 commit 0f8b496
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
# Tensorken: A Fun, Hackable, GPU-Accelerated, Neural Network library in Rust, Written by an Idiot

(work in progress)

Understanding deep learning from the perspective of a programmer, by building a deep learning framework from the ground up, in the spirit of [tinygrad](https://github.com/geohot/tinygrad) and [micrograd](https://github.com/karpathy/micrograd).

- Fun and hackable: most importantly Tensorken doesn't take itself too seriously. It's meant to be small, hackable, easy to understand and change above all else. If you want something usable for real work, look elsewhere.
- GPU-Accelerated: For the moment Tensorken runs on the GPU via [wgpu](https://wgpu.rs/), Rust's implementation of WebGPU. Accelerated comes with a grain of salt: tensor operations are much faster than the bundled but very naïve CPU implementation.
- Neural network: Getting less aspirational every month. There are basic tensor operations that run on CPU and GPU and a prototype reverse-mode autodiff implementation with a JAX-style API.
- GPU-Accelerated: Tensorken runs on the GPU via [wgpu](https://wgpu.rs/), Rust's implementation of WebGPU. Accelerated comes with a grain of salt: tensor operations are much faster than the bundled but very naïve CPU implementation.
- Neural network: There are basic tensor operations that run on CPU and GPU and a pretty powerful autodiff implementation with a JAX-style API. It can do forward mode and reverse mode automatic differentiation, and you can calculate higher-order derivatives, combining forward and reverse however you wish. For example, Tensorken can calculate Hessians using forward-over-reverse or reverse-over-reverse.
- Rust: No particular reason other than that I'm learning Rust.
- Written by an idiot: Hi there! I know nothing about neural network or GPU programming. As a result, anything and everything in here may be slow, backward, wrong, or stupid, and that's not an exclusive or exhaustive list.

Expand All @@ -26,7 +24,9 @@ Some cool stuff I looked at while figuring out how to build this.

## Getting started

Just clone the repo and run `cargo run --example tour`. Then explore the code - it's not big!
Clone the repo and run `cargo run --example tour` for an overview of tensor operations. The `jax_autodiff_cookbook` is a partial adaptation of JAX's [autodiff cookbook](https://jax.readthedocs.io/en/latest/notebooks/autodiff_cookbook.html), to demonstrate Tensorken's AD capabilities. The `makemore` examples are a translation to Tensorken (from PyTorch) of the first part of Andrej Karpathy's [Zero to Hero](https://karpathy.ai/zero-to-hero.html) neural network course, in which he builds and trains a simple neural network to generate baby names.

Then explore the code - it's not big!

Emerging posts with more explanation of the various stages of development of Tensorken. Intended to be read in order.

Expand Down

0 comments on commit 0f8b496

Please sign in to comment.