Add int8 to gemm w/ addmatrix

[alexbaden]

Update the gemm addmatrix benchmark to support int8 inputs as well as bfloat16.

The int8 benchmark is pretty slow - not because Triton performance is bad (it is at least on par with bfloat16) but because PyTorch does not support int8 matmul on GPU, so we have to do the matmul on the GPU. This makes the benchmark something like 20x slower. To fix that, I changed the PyTorch accuracy check to only run for a few shapes instead of all the shapes - I tried to pick shapes that I thought were representative of different cases but am open to suggestions. Now the benchmark runs in reasonable time.

A few open items need to be addressed:

• for int8 we want a separate geomean (i.e. geomean of bfloat16, and geomean of int8). What's the best way to keep int8 and bfloat16 separate? I can introduce an environment variable and run the benchmark twice - once only bfloat16, once only int8. Open to other suggestions.
• for onednn comparison, there is no problem with bfloat16 but there is no support for GPU matmul w/ int8. I don't think we want to run the comparison vs CPU (it takes too long and gives us no info), so I might need to introduce the env variable anyway to do one run which is float16 w/ onednn, and another run which is int8 w/out onednn.

cc #3014

[Egor-Krivov]

I think we treat them as 2 separate benchmarks in terms of reporting (So we have 2 lines of --benchmark gemm-postop-addmatrix & --benchmark gemm-postop-addmatrix-int8). Then all geomeans will work as intended. Otherwise we'll have to introduce some geomean groups in our database.

Then we either run benchmark script twice with different dypes to generate 2 separate report files (I'd prefer this), or modify our report script to add some filtering capability.

About onednn and int8 support. Do we want to measure onednn? If not, and we run it only for validation, maybe we could run it in other precision, like fp32 or bf16 just for validation of the output.

[Egor-Krivov]

My only issue with this PR right now is that all charts and GeoMeans for addmatrix benchmark will be discontinued, due to new parameters. Hence, my suggestion to introduce separate benchmark for int8

[alexbaden]

I removed the onednn related bits because for onednn we only measure kernel time, and the add step is not fused into the main gemm kernel so the comparison would not be appropriate.

I modified the int8 code to only run as a separate benchmark step controllable by environment variable, so the existing bfloat16 time should not be affected. The default run mode is bfloat16, with an optional int8 mode or all dtypes mode for local runs.

[whitneywhtsang]

Should we have a env var for all benchmarks to control if we verify the result?
Don't think we should skip checking correctness for some shapes.

[alexbaden]

The correctness checks are very slow for int8 because they have to be done on CPU. I don't think it is worth doubling (at least) the benchmarks runtime to do correctness checks for all shapes. I tried to pick a small sampling of different shapes (batched and not batched) so we had some int8 correctness checks.

[whitneywhtsang]

In general, I agree is a good compromise due to torch int8 matmul not supported on GPU, but my concern is it is not obvious to users that some int8 shapes are not tested for correctness. I don't have a better suggestion anyways.

[alexbaden]

If we wanted to do a CI run I would add the env variable, but if a user is running the benchmarks they can just comment out the if (and unindent the line after it).