[EPIC] Extended Floating-Point Support

[jrhemstad]

We want to better support users in using extended floating-point types like __half and bfloat16 in our libraries.

One of the major projects would be adding new extended floating-point library types to libcu++ for things like __half and bfloat16. The main goals would be:

• Works in host/device code
• Seamlessly uses equivalent core language type if/when the compiler supports it
• Conversion to/from existing cudart types
• Eventually supersede the cudart types

CUTLASS already has implementations of these types we could use as a starting point.

Tasks for new types

• Identify and list features of existing extended floating point wrapper types #1451
• Create generic floating-point wrapping type for <NumExponentBits, NumMantissaBits> #1665
• Provide cmath for floating_point<M,E> #2161
• Identify MVP user for new floating_point<M,E> and try using the new type #2188
• numeric_limits for floating_point<M, E> #2186
• <atomic> specializations for floating_point<M, E> #2184
• <complex> specializations for floating_point<M,E> #2185
• Create cuda/type_traits for cuda::is_floating_point trait #2187

Tasks for existing __half/bfloat types

• Specialize relevant cuda::(std::) types for __half/bfloat16/fp8 #525

Tasks for extended floating-point vector types

• Provide extended floating-point vector types (e.g., half2)
• Provide cmath for extended floating-point vector types

[cmpfeil]

Would be eager to see this; in particular also specializations complex<__half> and complex<bfloat16>.
Are there any plans on whether or when this might be available?