Add New Transformer Backbone for TTS Models #11911
Conversation
Signed-off-by: Jason <[email protected]>
Signed-off-by: Jason <[email protected]>
Signed-off-by: blisc <[email protected]>
| padding=padding, | ||
| dilation=dilation, | ||
| bias=bias, | ||
| ) |
There was a problem hiding this comment.
Adding this initialization helps stabilize training (after this line) as per my observations torch.nn.init.normal_(self.conv.weight, mean=0.0, std=0.02)
This was there is experimentalt5tts
There was a problem hiding this comment.
So, we will not add this here. o_net is handled in Transformer init, but proj was not. I will add this init to the projection layer.
…ch.zeros. Signed-off-by: Xuesong Yang <[email protected]>
| cond: Optional[Union[torch.Tensor, List[torch.Tensor]]] = None, | ||
| cond_mask: Optional[Union[torch.Tensor, List[torch.Tensor]]] = None, |
There was a problem hiding this comment.
nit: we now support the types of cond either torch.Tensor or a list of torch.Tensor. Is it better to unify the two as only a list of torch.Tensor? in this case, a list of a single tensor corresponds to a single encoder, and a list of multipe tensors corresponds to multiple encoders?
There was a problem hiding this comment.
@paarthneekhara what are your thoughts on that?
Signed-off-by: Jason <[email protected]>
Signed-off-by: Jason <[email protected]>
Signed-off-by: blisc <[email protected]>
… passed after running, `pytest -s -vvv tests/collections/tts/modules/test_transformer_2501.py` Signed-off-by: Xuesong Yang <[email protected]>
Signed-off-by: XuesongYang <[email protected]>
|
beep boop 🤖: 🚨 The following files must be fixed before merge! Your code was analyzed with PyLint. The following annotations have been identified: Mitigation guide:
By applying these rules, we reduce the occurance of this message in future. Thank you for improving NeMo's documentation! |
|
[🤖]: Hi @blisc 👋, We wanted to let you know that a CICD pipeline for this PR just finished successfully So it might be time to merge this PR or get some approvals I'm just a bot so I'll leave it you what to do next. //cc @pablo-garay @ko3n1g |
There was a problem hiding this comment.
LGTM. Thank you guys!
@paarthneekhara there is a nit-pick regarding cond to discuss, but not a blocker to merge the code.
* add core functions and classes of Transformer blocks. * completed tests for all classes, and made bugfixes. Now all tests are passed after running, `pytest -s -vvv tests/collections/tts/modules/test_transformer_2501.py` --------- Signed-off-by: Jason <[email protected]> Signed-off-by: blisc <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: XuesongYang <[email protected]> Co-authored-by: blisc <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: XuesongYang <[email protected]> Signed-off-by: Parth Mannan <[email protected]>
* add core functions and classes of Transformer blocks. * completed tests for all classes, and made bugfixes. Now all tests are passed after running, `pytest -s -vvv tests/collections/tts/modules/test_transformer_2501.py` --------- Signed-off-by: Jason <[email protected]> Signed-off-by: blisc <[email protected]> Signed-off-by: Xuesong Yang <[email protected]> Signed-off-by: XuesongYang <[email protected]> Co-authored-by: blisc <[email protected]> Co-authored-by: Xuesong Yang <[email protected]> Co-authored-by: XuesongYang <[email protected]> Signed-off-by: Abhinav Garg <[email protected]>
What does this PR do ?
Add new transformer backbone for TTS models.
pytest -s -vvv tests/collections/tts/modules/test_transformer_2501.pyCollection: tts
Changelog
PR Type: