Code

Add non-mcore fsdp2 strategy#11525

Code

Merged

ko3n1g merged 135 commits intomainNVIDIA/NeMo:mainfrom boxiangw/non-mcore-fsdp2NVIDIA/NeMo:boxiangw/non-mcore-fsdp2Copy head branch name to clipboardJan 8, 2025

+643-4

Commits

Commits on Dec 9, 2024

• Add fsdp2 strategy

  Show description for 3338e06
  BoxiangW

  committed
  3338e06

  Copy full SHA for 3338e06

• Apply isort and black reformatting

  Show description for 4c9b5df
  BoxiangW

  committed
  4c9b5df

  Copy full SHA for 4c9b5df

Commits on Dec 10, 2024

• Add imports

  Show description for 11a4637
  BoxiangW

  committed
  11a4637

  Copy full SHA for 11a4637

• Apply isort and black reformatting

  Show description for 5971cf4
  BoxiangW

  committed
  5971cf4

  Copy full SHA for 5971cf4

• Merge branch 'main' into boxiangw/non-mcore-fsdp2

  BoxiangW

  committed
  3533b89

  Copy full SHA for 3533b89

Commits on Dec 21, 2024

• Add init import

  Show description for 7c30f82
  BoxiangW

  committed
  7c30f82

  Copy full SHA for 7c30f82

• Apply isort and black reformatting

  Show description for ef11d67
  BoxiangW

  committed
  ef11d67

  Copy full SHA for ef11d67

Commits on Dec 23, 2024

• Fix mixtral export for NeMo 2.0 (#11532)

  Show description for 3d50f08
  

  2 peopleauthored and

  BoxiangW

  committed
  3d50f08

  Copy full SHA for 3d50f08

• Make HFDatasetDataModule a datasets.load_dataset wrapper (#11500)

  Show description for 6ea60de
  

  akoumpa

  authored and

  BoxiangW

  committed
  6ea60de

  Copy full SHA for 6ea60de

• ci: Bump release workflow (#11544)

  Show description for b81df9e
  

  ko3n1g

  authored and

  BoxiangW

  committed
  b81df9e

  Copy full SHA for b81df9e

• ci: Use SHA for cut-off (#11545)

  Show description for dfbb87f
  

  ko3n1g

  authored and

  BoxiangW

  committed
  dfbb87f

  Copy full SHA for dfbb87f

• link to mcore documentation (#11538)

  Show description for 1fe0310
  

  ashors1

  authored and

  BoxiangW

  committed
  1fe0310

  Copy full SHA for 1fe0310

• ci: Adjust inputs for code-freeze workflow (#11550)

  Show description for 02c2cdf
  

  ko3n1g

  authored and

  BoxiangW

  committed
  02c2cdf

  Copy full SHA for 02c2cdf

• ci: Bump release freeze (#11551)

  Show description for c37570a
  

  ko3n1g

  authored and

  BoxiangW

  committed
  c37570a

  Copy full SHA for c37570a

• Ko3n1g/ci/commit sha for cutoff (#11553)

  Show description for 37ee432
  

  ko3n1g

  authored and

  BoxiangW

  committed
  37ee432

  Copy full SHA for 37ee432

• ci: Bump code-freeze workflow (#11554)

  Show description for fc39d24
  

  ko3n1g

  authored and

  BoxiangW

  committed
  fc39d24

  Copy full SHA for fc39d24

• ci: Bump code freeze workflow (#11557)

  Show description for 2aff616
  

  ko3n1g

  authored and

  BoxiangW

  committed
  2aff616

  Copy full SHA for 2aff616

• Fix deploy conflicts in llm.api (#11367)

  Show description for 726e50a
  

  hemildesai

  authored and

  BoxiangW

  committed
  726e50a

  Copy full SHA for 726e50a

• perf summary docs link (#11262)

  Show description for f68208e
  

  2 peopleauthored and

  BoxiangW

  committed
  f68208e

  Copy full SHA for f68208e

• Add vlm nemo run scripts (#11394)

  Show description for 867dd0c
  

  2 peopleauthored and

  BoxiangW

  committed
  867dd0c

  Copy full SHA for 867dd0c

• Add from_dict to HFDatasetDataModule (#11559)

  Show description for 3867529
  

  akoumpa

  authored and

  BoxiangW

  committed
  3867529

  Copy full SHA for 3867529

• Prevent llama3.1 from using Linear interpolation (#11548)

  Show description for 8a44926
  

  suiyoubi

  authored and

  BoxiangW

  committed
  8a44926

  Copy full SHA for 8a44926

• [TTS] Add audio and mel codec HF models to docs (#11526)

  Show description for c1bb950
  

  rlangman

  authored and

  BoxiangW

  committed
  c1bb950

  Copy full SHA for c1bb950

• Update for NEST release (#11537)

  Show description for 4017f42
  

  stevehuang52

  authored and

  BoxiangW

  committed
  4017f42

  Copy full SHA for 4017f42

• Merging SpeechLLM development branch (#11462)

  Show description for 4b714b3
  

  4 peopleauthored and

  BoxiangW

  committed
  4b714b3

  Copy full SHA for 4b714b3

• Sync validation metrics for ASRModel (#11533)

  Show description for 9c264b7
  

  pzelasko

  authored and

  BoxiangW

  committed
  9c264b7

  Copy full SHA for 9c264b7

• NeMo 2.0 In-framework deployment support (#11523)

  Show description for 4972cc3
  

  2 peopleauthored and

  BoxiangW

  committed
  4972cc3

  Copy full SHA for 4972cc3

• Add SFT/PEFT HF tests (#11519)

  Show description for 73181b4
  

  akoumpa

  authored and

  BoxiangW

  committed
  73181b4

  Copy full SHA for 73181b4

• Fix typo: LocalNonpersitentObject -> LocalNonpersistentObject (#11546)

  Show description for e32ded1
  

  ananthsub

  authored and

  BoxiangW

  committed
  e32ded1

  Copy full SHA for e32ded1

• Adding documentation for packed dataset preparation with context para… (#11564)

  Show description for 81729e3
  

  tomlifu

  authored and

  BoxiangW

  committed
  81729e3

  Copy full SHA for 81729e3

• have micro_batch_size and global_batch_size as class attributes in mock datamodule (#11563)

  

  yashaswikarnati

  authored and

  BoxiangW

  committed
  729d2ee

  Copy full SHA for 729d2ee

• Revert "Fix the names of two sets of weight and bias in mcore_to_nemo_mapping" (#11560)

  Show description for bd6d6ff
  

  ashors1

  authored and

  BoxiangW

  committed
  bd6d6ff

  Copy full SHA for bd6d6ff

• add huggingface-based tokenizer support for mixtral HF -> .nemo (#11572)

  Show description for 820b3ec
  

  dimapihtar

  authored and

  BoxiangW

  committed
  820b3ec

  Copy full SHA for 820b3ec

• Github Actions tests for Llava Next and modify pretrain recipe to have language model path (#11424)

  Show description for ad1282e
  

  2 peopleauthored and

  BoxiangW

  committed
  ad1282e

  Copy full SHA for ad1282e

• Fix SingleDeviceStrategy support in Nsys callback (#11574)

  Show description for f11585e
  

  akoumpa

  authored and

  BoxiangW

  committed
  f11585e

  Copy full SHA for f11585e

• remove dialogue scripts and docs (#11577)

  Show description for fd4c302
  

  dimapihtar

  authored and

  BoxiangW

  committed
  fd4c302

  Copy full SHA for fd4c302

• add JitTransform (#11131)

  Show description for fc809b8
  

  akoumpa

  authored and

  BoxiangW

  committed
  fc809b8

  Copy full SHA for fc809b8

• NeMo 2.0 documentation upgrade (#11235)

  Show description for 36ee6f3
  

  dimapihtar

  authored and

  BoxiangW

  committed
  36ee6f3

  Copy full SHA for 36ee6f3

• Remove auto-import of lhotse when importing nemo.collections.common.data (#11578)

  Show description for a3c377c
  

  pzelasko

  authored and

  BoxiangW

  committed
  a3c377c

  Copy full SHA for a3c377c

• Fix example configs (#11571)

  Show description for 9d3d36e
  BoxiangW

  committed
  9d3d36e

  Copy full SHA for 9d3d36e

• fix (#11575)

  Show description for b8f9b0b
  

  ko3n1g

  authored and

  BoxiangW

  committed
  b8f9b0b

  Copy full SHA for b8f9b0b

• NIM supporting changes for nemo.export for NeMo 2.0 (#11488)

  Show description for 7bc32cd
  

  2 peopleauthored and

  BoxiangW

  committed
  7bc32cd

  Copy full SHA for 7bc32cd

• AED greedy confidence estimation (#11573)

  Show description for 929d643
  

  GNroy

  authored and

  BoxiangW

  committed
  929d643

  Copy full SHA for 929d643

• gemma fix (#11587)

  

  suiyoubi

  authored and

  BoxiangW

  committed
  82544df

  Copy full SHA for 82544df

• Update T5 DataModule regarding Pretrain/Finetune validate (#11584)

  Show description for 3483208
  

  2 peopleauthored and

  BoxiangW

  committed
  3483208

  Copy full SHA for 3483208

• fix llama3 (#11580)

  

  suiyoubi

  authored and

  BoxiangW

  committed
  6f084dd

  Copy full SHA for 6f084dd

• Add Hf nemorun tests (#11566)

  Show description for a3eb280
  

  HuiyingLi

  authored and

  BoxiangW

  committed
  a3eb280

  Copy full SHA for a3eb280

• [🤖]: Howdy folks, let's bump NeMo-Toolkit to 2.2.0rc0 ! (#11555)

  Show description for b98160c
  

  github-actions[bot]

  authored and

  BoxiangW

  committed
  b98160c

  Copy full SHA for b98160c

• Pass the number of experts to modelopt layer spec (#11607)

  Show description for 711176a
  

  janekl

  authored and

  BoxiangW

  committed
  711176a

  Copy full SHA for 711176a

• Adding changes to asr documentation (#11397)

  Show description for 7210212
  

  Ssofja

  authored and

  BoxiangW

  committed
  7210212

  Copy full SHA for 7210212

• Support Cosmos tokenizer TensorRT inference (#11472)

  Show description for 86393b5
  

  meatybobby

  authored and

  BoxiangW

  committed
  86393b5

  Copy full SHA for 86393b5

• Neva updates to latest mcore and some fixes (#11565)

  Show description for 5298ce3
  

  yaoyu-33

  authored and

  BoxiangW

  committed
  5298ce3

  Copy full SHA for 5298ce3

• add nemo2-sft-peft to readme (#11613)

  Show description for 08bf53c
  

  HuiyingLi

  authored and

  BoxiangW

  committed
  08bf53c

  Copy full SHA for 08bf53c

• Set Minitron width pruning batch size 1 (#11603)

  Show description for 75bc074
  

  kevalmorabia97

  authored and

  BoxiangW

  committed
  75bc074

  Copy full SHA for 75bc074

• Disable CP for running Inference using megatron_gpt_eval (#11547)

  Show description for 69d84cc
  

  2 peopleauthored and

  BoxiangW

  committed
  69d84cc

  Copy full SHA for 69d84cc

• ci: Add no-fail-fast mode (#11608)

  Show description for 0b14618
  

  ko3n1g

  authored and

  BoxiangW

  committed
  0b14618

  Copy full SHA for 0b14618

• Chat dataset support (#11423)

  Show description for b975aaa
  

  cuichenx

  authored and

  BoxiangW

  committed
  b975aaa

  Copy full SHA for b975aaa

• Sortformer Diarizer 4spk v1 model PR Part 2: Unit-tests for Sortformer Diarizer. (#11336)

  Show description for aa7a4e1
  

  tango4j

  authored and

  BoxiangW

  committed
  aa7a4e1

  Copy full SHA for aa7a4e1

• 2x more memory efficient Graph-based RNN-T (#11169)

  Show description for 4256580
  

  artbataev

  authored and

  BoxiangW

  committed
  4256580

  Copy full SHA for 4256580

• Use explicit subpaths in io for exporting a checkpoint (#11352)

  Show description for 2b6100d
  

  hemildesai

  authored and

  BoxiangW

  committed
  2b6100d

  Copy full SHA for 2b6100d

• Remove triton requirement (#11627)

  Show description for 1da9632
  

  2 peopleauthored and

  BoxiangW

  committed
  1da9632

  Copy full SHA for 1da9632

• ci: Remove comment if no changes required anymore (#11624)

  Show description for 2eb897b
  

  ko3n1g

  authored and

  BoxiangW

  committed
  2eb897b

  Copy full SHA for 2eb897b

• Jit with peft (#11586)

  Show description for e4afd2d
  

  akoumpa

  authored and

  BoxiangW

  committed
  e4afd2d

  Copy full SHA for e4afd2d

• NeMo-UX: add Hf's AutoModelForImageTextToText (#11321)

  Show description for 44689b1
  

  akoumpa

  authored and

  BoxiangW

  committed
  44689b1

  Copy full SHA for 44689b1

• ci: Bump release workflow (#11635)

  Show description for ff568cd
  

  ko3n1g

  authored and

  BoxiangW

  committed
  ff568cd

  Copy full SHA for ff568cd

• Add fix docstring for speech commands (#11638)

  Show description for b9457db
  

  titu1994

  authored and

  BoxiangW

  committed
  b9457db

  Copy full SHA for b9457db

• Fixing Multi_Task_Adapters.ipynb by replacing canary2 with canary_custom (#11641)

  Show description for 53b8eb4
  

  weiqingw4ng

  authored and

  BoxiangW

  committed
  53b8eb4

  Copy full SHA for 53b8eb4

• fixed config name in online augmentation tutorial (#11628)

  Show description for 8829106
  

  nasretdinovr

  authored and

  BoxiangW

  committed
  8829106

  Copy full SHA for 8829106

• fix default nodes (#11632)

  

  suiyoubi

  authored and

  BoxiangW

  committed
  faa04ed

  Copy full SHA for faa04ed

• add renormalize_blend_weights param (#11647)

  Show description for 276c075
  

  dimapihtar

  authored and

  BoxiangW

  committed
  276c075

  Copy full SHA for 276c075

• Sortformer Diarizer 4spk v1 model PR Part 3: Speaker Diarization Mixin (#11511)

  Show description for 368ed62
  

  2 peopleauthored and

  BoxiangW

  committed
  368ed62

  Copy full SHA for 368ed62

• Fix peft inference (#11568)

  Show description for 2b3b158
  

  cuichenx

  authored and

  BoxiangW

  committed
  2b3b158

  Copy full SHA for 2b3b158

• Add fix docstring for speech commands (#11659)

  Show description for 45f2a4c
  

  titu1994

  authored and

  BoxiangW

  committed
  45f2a4c

  Copy full SHA for 45f2a4c

• update nemo container version for notebooks (#11651)

  Show description for 90f6fb7
  

  HuiyingLi

  authored and

  BoxiangW

  committed
  90f6fb7

  Copy full SHA for 90f6fb7

• Fix Optimizer & LR scheduler & Consume Samples when Resuming in PEFT (#11631)

  Show description for 18448b9
  

  2 peopleauthored and

  BoxiangW

  committed
  18448b9

  Copy full SHA for 18448b9

• Add vlm generation function (#11063)

  Show description for ccb3dc1
  

  meatybobby

  authored and

  BoxiangW

  committed
  ccb3dc1

  Copy full SHA for ccb3dc1

• ci: Small pylint fix (#11667)

  Show description for db0a2d0
  

  ko3n1g

  authored and

  BoxiangW

  committed
  db0a2d0

  Copy full SHA for db0a2d0

• Add slimpajama example (#10671)

  Show description for 53c64ed
  

  hemildesai

  authored and

  BoxiangW

  committed
  53c64ed

  Copy full SHA for 53c64ed

• Remove NeMo 1 docs (#11670)

  Show description for 5cebc85
  

  cuichenx

  authored and

  BoxiangW

  committed
  5cebc85

  Copy full SHA for 5cebc85

• Change peft merged model to bf16 (#11663)

  Show description for c872739
  

  HuiyingLi

  authored and

  BoxiangW

  committed
  c872739

  Copy full SHA for c872739

• Add Minitron depth pruning (layer dropping) to megatron_gpt_prune.py (#11609)

  Show description for 72634f3
  

  kevalmorabia97

  authored and

  BoxiangW

  committed
  72634f3

  Copy full SHA for 72634f3

• add documentation for checkpoint averaging (#11594)

  Show description for 6dced1d
  

  2 peopleauthored and

  BoxiangW

  committed
  6dced1d

  Copy full SHA for 6dced1d

• Downgrading the 'datasets' package from 3.0.0 to 2.21.0 for Multilang_ASR.ipynb and ASR_CTC_Language_Finetuning.ipynb (#11675)

  Show description for 8078cd4
  

  weiqingw4ng

  authored and

  BoxiangW

  committed
  8078cd4

  Copy full SHA for 8078cd4

• Utilities to detect and drop deprecated arguments from NeMo 2.0 checkpoint context io.json (#11648)

  Show description for 054bd46
  

  janekl

  authored and

  BoxiangW

  committed
  054bd46

  Copy full SHA for 054bd46

• NIM supporting changes for nemo.export for NeMo 2.0 (part II) (#11669)

  Show description for fc54cee
  

  janekl

  authored and

  BoxiangW

  committed
  fc54cee

  Copy full SHA for fc54cee

• Add check for symlink in _safe_extract (#11611)

  Show description for 3c9c3f6
  

  athitten

  authored and

  BoxiangW

  committed
  3c9c3f6

  Copy full SHA for 3c9c3f6

• Fix baichuan export (#11640)

  Show description for 86c0f1a
  

  cuichenx

  authored and

  BoxiangW

  committed
  86c0f1a

  Copy full SHA for 86c0f1a

• Rename multimodal data module - EnergonMultiModalDataModule (#11654)

  Show description for 3a8e75d
  

  yashaswikarnati

  authored and

  BoxiangW

  committed
  3a8e75d

  Copy full SHA for 3a8e75d

• ci: Bump release workflow (#11686)

  Show description for e0b14e7
  

  ko3n1g

  authored and

  BoxiangW

  committed
  e0b14e7

  Copy full SHA for e0b14e7

• Fixing the device assignment issues during inference (test_batch) in Sortformer model (#11671)

  Show description for 6e3cccf
  

  tango4j

  authored and

  BoxiangW

  committed
  6e3cccf

  Copy full SHA for 6e3cccf

• add timestamp support (#11591)

  Show description for b4aecaf
  

  kevinhu-nv

  authored and

  BoxiangW

  committed
  b4aecaf

  Copy full SHA for b4aecaf

• Make LinearAdapter a nn.Linear child to maintain ckpt structure (#11642)

  Show description for 099bc80
  

  akoumpa

  authored and

  BoxiangW

  committed
  099bc80

  Copy full SHA for 099bc80

• Add multi images support for mllama generate (#11672)

  Show description for da10109
  

  meatybobby

  authored and

  BoxiangW

  committed
  da10109

  Copy full SHA for da10109

• Merge branch 'main' into boxiangw/non-mcore-fsdp2

  BoxiangW

  committed
  597b387

  Copy full SHA for 597b387

• Fix merge

  Show description for 7682cf2
  BoxiangW

  committed
  7682cf2

  Copy full SHA for 7682cf2

Commits on Dec 24, 2024

• Fix non-mcore fsdp2

  Show description for 0e694f7
  BoxiangW

  committed
  0e694f7

  Copy full SHA for 0e694f7

Commits on Dec 30, 2024

• Merge branch 'main' into boxiangw/non-mcore-fsdp2

  BoxiangW

  authored
  e50be32

  Copy full SHA for e50be32

• Add run code

  BoxiangW

  committed
  a2a67ef

  Copy full SHA for a2a67ef

• Apply isort and black reformatting

  Show description for 446c888
  BoxiangW

  committed
  446c888

  Copy full SHA for 446c888

Commits on Dec 31, 2024

• Add hooks

  BoxiangW

  committed
  4782c7e

  Copy full SHA for 4782c7e

• Apply isort and black reformatting

  Show description for d53ab15
  BoxiangW

  committed
  d53ab15

  Copy full SHA for d53ab15

Commits on Jan 2, 2025

• Add test

  BoxiangW

  committed
  a1f08a2

  Copy full SHA for a1f08a2

• Merge branch 'main' into boxiangw/non-mcore-fsdp2

  BoxiangW

  authored
  837035f

  Copy full SHA for 837035f

• Apply isort and black reformatting

  Show description for ae1104e
  BoxiangW

  committed
  ae1104e

  Copy full SHA for ae1104e

Commits on Jan 3, 2025

• Add fsdp2 support on hf_auto_model_for_causal_lm

  BoxiangW

  committed
  a31f367

  Copy full SHA for a31f367

• REvert model hooks for fsdp2 sharding

  BoxiangW

  committed
  b5014b8

  Copy full SHA for b5014b8

• REvert some test changes

  BoxiangW

  committed
  1d9257a

  Copy full SHA for 1d9257a

• REvert tesst

  BoxiangW

  committed
  27ffb82

  Copy full SHA for 27ffb82

• Add line

  BoxiangW

  committed
  346554d

  Copy full SHA for 346554d

• Fix test

  BoxiangW

  committed
  7cd8186

  Copy full SHA for 7cd8186

• fix test

  BoxiangW

  committed
  0a10868

  Copy full SHA for 0a10868

• Fix test

  BoxiangW

  committed
  03110a9

  Copy full SHA for 03110a9

• Apply isort and black reformatting

  Show description for bb52f17
  BoxiangW

  committed
  bb52f17

  Copy full SHA for bb52f17

• Add CI test

  BoxiangW

  committed
  648d830

  Copy full SHA for 648d830

• Revert test change

  BoxiangW

  committed
  752ff1a

  Copy full SHA for 752ff1a

• Fix test

  BoxiangW

  committed
  6650a1f

  Copy full SHA for 6650a1f

• Fix test

  BoxiangW

  committed
  6edd7fc

  Copy full SHA for 6edd7fc

• Apply isort and black reformatting

  Show description for 6f4cb64
  BoxiangW

  committed
  6f4cb64

  Copy full SHA for 6f4cb64

Commits on Jan 4, 2025

• Add check for parallel

  BoxiangW

  committed
  7e18cfe

  Copy full SHA for 7e18cfe

Commits on Jan 6, 2025

• Move function into nl.strategy

  BoxiangW

  committed
  70b9f7c

  Copy full SHA for 70b9f7c

• Apply isort and black reformatting

  Show description for 94d1f90
  BoxiangW

  committed
  94d1f90

  Copy full SHA for 94d1f90

• Merge branch 'main' into boxiangw/non-mcore-fsdp2

  BoxiangW

  authored
  1e18b9e

  Copy full SHA for 1e18b9e

• Add tests

  BoxiangW

  committed
  c175ca0

  Copy full SHA for c175ca0

• Remove test

  BoxiangW

  committed
  adeeffd

  Copy full SHA for adeeffd

• Apply isort and black reformatting

  Show description for 6dc57c4
  BoxiangW

  committed
  6dc57c4

  Copy full SHA for 6dc57c4

Commits on Jan 7, 2025

• Add test

  BoxiangW

  committed
  9c9b972

  Copy full SHA for 9c9b972

• Add fsdp2 ci test with memory check

  BoxiangW

  committed
  ff2c54c

  Copy full SHA for ff2c54c

• Apply isort and black reformatting

  Show description for db85277
  BoxiangW

  committed
  db85277

  Copy full SHA for db85277

• Fix import

  BoxiangW

  committed
  ca5ffe4

  Copy full SHA for ca5ffe4

• Merge branch 'main' into boxiangw/non-mcore-fsdp2

  BoxiangW

  authored
  42f4ee8

  Copy full SHA for 42f4ee8

• Merge branch 'main' into boxiangw/non-mcore-fsdp2

  BoxiangW

  authored
  1430226

  Copy full SHA for 1430226

• fix test

  BoxiangW

  committed
  b78205a

  Copy full SHA for b78205a

• Apply isort and black reformatting

  Show description for 1e069c3
  BoxiangW

  committed
  1e069c3

  Copy full SHA for 1e069c3

• Add copyright

  BoxiangW

  committed
  81aeb50

  Copy full SHA for 81aeb50

Commits on Jan 8, 2025

• include test list

  Show description for d8e7247
  akoumpa

  committed
  d8e7247

  Copy full SHA for d8e7247