Add IB verbs logging and enable traces through install.sh

[mustafabar]

Details

This PR will facilitate IB Verbs calls logging to help with network-level tracing of work requests at the IB Verbs level.

Work item: Internal.

What were the changes?

1. Enable logging of all ibv_post_sends and their underlying QP, source, and destination NIC, message length, etc.
2. Add VERBS as a type of NCCL_DEBUG_SUBSYS. Note that The NET subsystem level is broader in its coverage and adds a lot of unneeded info if used just to view IB verbs call traces
3. Add remote IBV device index to meta data struct ncclIbDevInfo so that it is possible to show both sender and receiver NIC index on the sender side

Why were the changes made?
Many want to understand how RCCL interacts with the network fabric in a fine-grain fashion. This will enable logging all other IB verbs interactions incremantally

How was the outcome achieved?
Through making TRACE calls for both FIFO and data ibv_post_sends, and adding a flag for enabling traces (--log-trace) to the install script

Additional Documentation:
What else should the reviewer know?

Approval Checklist

Do not approve until these items are satisfied.

• Verify the CHANGELOG has been updated, if
  • there are any NCCL API version changes,
  • any changes impact library users, and/or
  • any changes impact any other ROCm library.