kron-torch 0.2.9

What's Changed

• merge memory improvement PR from Lucas Nestler @ClashLuke

kron-torch 0.2.6

What's Changed

• Get rid of trust region
• Add normalize grads layer-wise argument
• deterministically update preconditioners for stability
• TODO: update using Lucas Nestler's optimizations

kron-torch 0.2.5

What's Changed

• small improvements

kron-torch 0.2.4

What's Changed

• Efficiency improvements from ClashLuke
• New trust region clipping that needs less (maybe no) tuning

kron-torch 0.2.3

What's Changed

• triton install, 3.0.0

kron-torch 0.2.2

What's Changed

• Trust region clipping improved
• Get rid of max skew triangular and replace with memory_save_mode which can be either None to use default triangular preconditioners, 'one_diag' to use one diagonal per layer, or 'all_diag' to use all diagonal preconditioners (fastest/lowest mem but slower learning)

kron-torch 0.2.1

What's Changed

• Better compiling, work with @opooladz

kron-torch 0.2.0

What's Changed

• torch.compiled some functions
• Reorganized and simplified code

kron-torch 0.1.10

What's Changed

• print sizes of momentum and precond at start

kron-torch 0.1.9

What's Changed

• Better optimizer organization and handling of state
• docstring and readme changes