New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

网络初始化/Initializing neural networks #959

Open

junxnone opened this issue Apr 2, 2021 · 0 comments

Owner

junxnone commented Apr 2, 2021 •

edited

Loading

Reference

Brief

UseCase

任意常数初始化方法(包含 Zero) 阻止学习不同特征
太小/太大初始化会导致学习缓慢/发散(梯度消失/梯度爆炸)
合适的初始化值
- 激活值的均值应为零
- 每一层激活值的方差应该保持一致

This was referenced Apr 2, 2021

xavier 初始化 #574

Open

msra 初始化 #573

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment