Fuse reshapes on pointwise inputs for mlir output fusion

[pfultz2]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.20%. Comparing base (241e24e) to head (0ef4e55).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #3569   +/-   ##
========================================
  Coverage    92.20%   92.20%           
========================================
  Files          513      513           
  Lines        21658    21658           
========================================
  Hits         19970    19970           
  Misses        1688     1688           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[shivadbhavsar]

Approving, comment is probably a topic for another PR if its even an issue

[shivadbhavsar]

Might be tangential to this PR, but what happens if one of the reshape ops has multi outputs? ie. something like:

dot -> reshape -> pointwise -> .... -> dot
          |_____________________________|

[pfultz2]

The loop below checks that all the reshapes between dot and pointwise have only one output, and it will output both if any reshape is used more than once. For the aux inputs to pointwise, it doesnt really matter:

┌───┐┌─┐      
│dot││x│      
└┬──┘└┬┘      
 │┌───▽──────┐
 ││reshape   │
 │└┬───────┬─┘
┌▽─▽──────┐│  
│pointwise││  
└┬────────┘│  
┌▽─────────▽┐ 
│convolution│ 
└───────────┘ 

After mlir fusion it just becomes:

┌──────────┐                         
│x         │                         
└┬────────┬┘                         
┌▽──────┐┌▽─────────────────────────┐
│reshape││mlir_dot_reshape_pointwise│
└┬──────┘└┬─────────────────────────┘
┌▽────────▽─┐                        
│convolution│                        
└───────────┘ 

DCE wont remove the reshape since its used, and the duplicated reshape doesnt really matter since it is potentially an aliasing-like operator.

[migraphx-bot]

Test	Batch	Rate new b1579e	Rate old 879f30	Diff	Compare
torchvision-resnet50	64	3,235.54	3,237.49	-0.06%	✅
torchvision-resnet50_fp16	64	6,878.27	6,885.53	-0.11%	✅
torchvision-densenet121	32	2,437.85	2,438.04	-0.01%	✅
torchvision-densenet121_fp16	32	4,214.82	4,201.39	0.32%	✅
torchvision-inceptionv3	32	1,615.08	1,614.18	0.06%	✅
torchvision-inceptionv3_fp16	32	2,691.74	2,690.16	0.06%	✅
cadene-inceptionv4	16	750.75	750.10	0.09%	✅
cadene-resnext64x4	16	810.35	809.38	0.12%	✅
slim-mobilenet	64	6,668.09	6,664.81	0.05%	✅
slim-nasnetalarge	64	199.04	199.11	-0.03%	✅
slim-resnet50v2	64	3,430.59	3,426.01	0.13%	✅
bert-mrpc-onnx	8	1,146.10	1,145.27	0.07%	✅
bert-mrpc-tf	1	480.87	479.30	0.33%	✅
pytorch-examples-wlang-gru	1	462.96	470.08	-1.51%	✅
pytorch-examples-wlang-lstm	1	441.82	438.03	0.87%	✅
torchvision-resnet50_1	1	808.15	803.96	0.52%	✅
cadene-dpn92_1	1	431.76	433.48	-0.40%	✅
cadene-resnext101_1	1	391.26	389.71	0.40%	✅
onnx-taau-downsample	1	373.25	373.58	-0.09%	✅
dlrm-criteoterabyte	1	31.80	31.80	-0.01%	✅
dlrm-criteoterabyte_fp16	1	51.04	51.06	-0.03%	✅
agentmodel	1	8,558.05	8,764.93	-2.36%	✅
unet_fp16	2	57.92	58.01	-0.15%	✅
resnet50v1_fp16	1	1,058.83	1,021.79	3.63%	🔆
resnet50v1_int8	1	788.85	780.83	1.03%	✅
bert_base_cased_fp16	64	1,172.43	1,172.44	-0.00%	✅
bert_large_uncased_fp16	32	362.52	362.45	0.02%	✅
bert_large_fp16	1	202.19	201.10	0.54%	✅
distilgpt2_fp16	16	2,218.42	2,219.84	-0.06%	✅
yolov5s	1	525.33	521.90	0.66%	✅
tinyllama	1	43.59	43.62	-0.05%	✅
vicuna-fastchat	1	43.89	43.83	0.13%	✅
whisper-tiny-encoder	1	412.16	412.03	0.03%	✅
whisper-tiny-decoder	1	411.31	410.73	0.14%	✅

Check results before merge 🔆

[migraphx-bot]

     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     ✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

     ✅ unet: PASSED: MIGraphX meets tolerance

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance