Skip to content

Models to run

Jump to bottom
Hlib edited this page Jan 19, 2019 · 6 revisions

Lang model

  • nh=300

  • nh=600

  • nh=1200

  • nh=2000

  • bptt=150

  • bptt=350

  • bptt=990

  • em=150

  • em=300

  • bptt=150 nh=300 em=150

  • bptt=350 nh=300 em=150

  • bptt=150 nh=600 em=150

  • bptt=150 nh=300 em=300, BW (5_em_sz_300_nh_300)

  • bptt=150 nh=1200 em=150

  • bptt=350 nh=300 em=150, BW (5_baseline)

  • bptt=900 nh=300 em=150, BW (5_baseline__)

  • bptt=150 nh=2000 em=150

  • bptt=900 nh=2000 em=300 (bs=16)

  • bptt=350 nh=1200 em=300

Classification

Location

  • without pretraining bptt=990, nh=300, em=150, FW
  • without pretraining bptt=990, nh=600, em=300, FW
  • without pretraining bptt=990, nh=300, em=150, BW
  • without pretraining bptt=990, nh=600, em=300, BW

Level (categorical)

  • without pretraining bptt=990, nh=300, em=150, FW
  • without pretraining bptt=990, nh=600, em=300, FW
  • without pretraining bptt=990, nh=300, em=150, BW
  • without pretraining bptt=990, nh=600, em=300, BW

Level (binary: trace/debug/info vs warn/error/fatal)

  • without pretraining bptt=990, nh=300, em=150, FW
  • without pretraining bptt=990, nh=600, em=300, FW
  • without pretraining bptt=990, nh=300, em=150, BW
  • without pretraining bptt=990, nh=600, em=300, BW
  1. Introduction
  2. State of the art
  3. Background information
    1. Brief Introduction to language modeling
    2. Brief Introduction to Neural Networks and Deep Learning
    3. Using Recurrent Neural Networks for Language Modeling
    4. Long Short Term Memory Networks
  4. Vocab modeling
    1. Data Gathering and Tokenization
    2. Reducing Vocabulary Size
    3. Preprocess Configurations
    4. Vocabulary modeling evaluation
  5. Language modeling
    1. Neural network architecture
    2. Neural Network Training
    3. Subword aware accuracy and entropy
    4. Neural Cache
    5. Language modeling evaluation
  6. Building classifiers
    1. Generation of labeled data for classification
    2. Transfer learning
    3. Classification parameter hypersearch
    4. Classifier evaluation
  7. Discussion
  8. Future work
  9. Conclusion
  10. Appendix A. Preprocessing Configurations Examples
  11. Appendix B. Subword Representation Examples
Clone this wiki locally