Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrating torchsharp functionality into Bonsai #48

Open
wants to merge 71 commits into
base: main
Choose a base branch
from

Conversation

ncguilbeault
Copy link
Collaborator

@ncguilbeault ncguilbeault commented Jan 21, 2025

This pull request introduces a new project, Bonsai.ML.Torch, which adds functionalities from TorchSharp. It adds functions to perform basic tensor manipulations, linear algebra, inference with neural networks, and more. The changes include adding the project to the solution, defining the project file, and implementing several new classes and operations.

New Project:

  • The Bonsai.ML.Torch project was added to the solution file Bonsai.ML.sln.
  • Adds a reference to the TorchSharp package.

Tensor Operations:

  • Convert common .NET data types (double, float, arrays, etc.) to tensors using ToTensor.
  • ToTensor also has overloads for several OpenCV.Net data types (IplImage, Mat) and uses efficient wrapping.
  • Tensors can be initialized in various ways, including initializing tensors with Ones, Zeros, Arange, LinSpace, Empty, etc. Custom tensors can be defined using the CreateTensor class and specifying the values using Python-like syntax.
  • Tensors can be converted back to .NET arrays using the ToArray node (for flattening tensors) and the ToNDArray node (for multidimensional arrays).
  • Manipulate tensors in various ways, for example Reshape to change dimensions, Concat to concatenate tensors along a specified dimension, ConvertDataType to convert tensors to a specified scalar type, etc.
  • Adds GPU support by exposing InitializeDeviceType with CUDA-compatible GPUs. Tensors can be transferred to/from devices using ToDevice.
  • The primitive torch.Tensor contains many extension methods which can be accessed using ExpressionTransform (for example, it.sum() to sum a tensor, or it.T to transpose), and works with overloaded operators, for example, Zip -> Multiply.
  • Indexing tensors can be done using the Index class and specifying the indexes using Python-like syntax. It is also possible to define tensor indexes explicitly with nodes using, BooleanIndex, ColonIndex, etc., casting them to an array, and then combining them with a tensor to be used with the standard reactive Index node.

Neural Networks:

  • The package adds some initial functionality for working with torch modules.
  • Torch modules can be loaded from predefined architectures (AlexNet, MNIST, and MobileNet) and can optionally load precomputed weights, provided that the weights are saved from a model with the identical archiecture. Model weights can be saved using SaveModel.
  • Running inference can be done using the Forward class, and training the model can be done using the Backward class.
  • The package also provides an interface ITorchModule and an adapter class TorchModuleAdapter to allow custom modules to be created and used with the other modules.

Linear Algebra:

  • A couple of useful functions are added to allow for performing complex algebraic computations on tensors, such as calculating the determinant with Det, the eigens with Eig, matrix inversion with Inv, etc.

Vision:

  • Adds a single node, Normalize, for transforming image tensors using means and standard deviations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant