Skip to content

Added activation to MHAttention

Compare
Choose a tag to compare
@tatp22 tatp22 released this 20 Jun 16:14
· 73 commits to master since this release

Added both the RELU and GELU activation function options to the multihead attention block