Add standardized extended floating point types with complete implementations

[jrhemstad]

As a CUDA user, I care about extended floating point types like __half or __nv_bfloat16. These types are deficient in a variety of ways as they may be lacking certain functionality one would expect from such a type.

We should add equivalent types to libcu++ that have both host and device implementations.

Ideally, these should provide seamless transition to a built-in compiler provided type such that if I write my code using cuda::float16 that it will use the built-in type when available and fall back to a library implementation if not.

CUTLASS has implementations that we should use as a starting point.

See also:

• Support for std::complex<__half>? libcudacxx#95
• Add cuda::std::numeric_limits<__half> #987

Tasks

Preview Give feedback

1. Identify and list features of existing extended floating point wrapper types #1451
   +0
   
2. Write up a table of features for each existing extended floating point type #1452
   +0
   

[cliffburdick]

In the MatX we had to create our own wrappers to support __half, and __nv_bfloat16 types with wrappers so all normal C++ math operations are supported. It would be very useful if we could drop our types and use a standard version.

[miscco]

@cliffburdick if you drop me a link I am happy to do the leg work of porting those

[jrhemstad]

@cliffburdick if you drop me a link I am happy to do the leg work of porting those

To be clear, we won't just be trivially porting the MatX types. We'd need to evaluate the diff between CUTLASS, MatX, and the various other solutions to design a standard solution and then build that.

[cliffburdick]

@miscco our half file is here: https://github.com/NVIDIA/MatX/blob/main/include/matx/core/half.h
half complex is here: https://github.com/NVIDIA/MatX/blob/main/include/matx/core/half_complex.h

I agree they should not be ported as-is. I don't know if the CUTLASS ones contain all the stuff we have, but I think the end goal would be once cuda:: has the types we can alias them using our old names and delete the files.

[jrhemstad]

Closing this as superseded by #1665