[WC, PT] Store compression scale in f16 #2596
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
NNCF PT
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #2596 +/- ##
===========================================
- Coverage 91.16% 84.91% -6.26%
===========================================
Files 494 494
Lines 45350 45373 +23
===========================================
- Hits 41342 38527 -2815
- Misses 4008 6846 +2838
... and 59 files with indirect coverage changes
Flags with carried forward coverage won't be shown. Click here to find out more.
|
alexsu52
reviewed
Mar 24, 2024
nikita-savelyevv
added a commit
to KodiaqQ/nncf
that referenced
this pull request
Apr 19, 2024
- For weight compression, align quantization scales to always be saved in FP16 precision nevertheless the input model weights precision. (openvinotoolkit#2596, openvinotoolkit#2508) - (UX) Expose `OverflowFix`, `AdvancedSmoothQuantParameters` and `AdvancedBiasCorrectionParameters` classes to be available for import as `nncf.OverfloFix`, `nncf.AdvancedSmoothQuantParameters`, `nncf.AdvancedBiasCorrectionParameters`. (openvinotoolkit#2608, openvinotoolkit#2624)
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Below are the compression subgraphs for the first conv2d in mobilenet_v2 after conversion to OV, this is similar to the table presented in #2537 .
Compared to OV case, there is an additional Multiply node after the scale Multiply node. It seems to come from Batch Norm applied to the convolution. In case of PT weight compression it does not get merged into the weight as it does in OV case.
Reason for changes
Weight compression for PT backend fails when applied to model in half precision. The reason is that the scale is always in FP32, and hence decompression result is also in FP32, which conflicts with input type of FP16.
Related tickets
134063
Tests
Added test for half/full precision cases. Also added cases for different devices as it was thought that it may influence tracing in half precision.