Skip to content

Added intermediate dim change

Compare
Choose a tag to compare
@tatp22 tatp22 released this 04 Aug 16:03
· 18 commits to master since this release

Added intermediate ff dimension

Now, the model dimension can be different in the intermediate layers.
This change applies to the ff module, and only in the encoder. Now, if
the flag ff_intermediate is not None, the layers will look like this:

channels -> ff_dim -> ff_intermediate (For layer 1)
ff_intermediate -> ff_dim -> ff_intermediate (For layers 2 to depth-1)
ff_intermediate -> ff_dim -> channels (For layer depth)

As opposed to

channels -> ff_dim -> channels (For all layers)