"@verify_operation" lead to pretrain of multi-nodes hang

[sankexin]

System Info

python3.10
torch 2.3.0
accelerate 1.3.0

Information

• The official example scripts
• My own modified scripts

Tasks

• One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
• My own task or dataset (give details below)

Reproduction

/usr/local/lib/python3.10/site-packages/accelerate/utils/operations.py:

@verify_operation
def gather(tensor):

when I use "#@verify_operation" don`t hang，and the result of train is correct ！
so, there is some errors in verify_operation.

Expected behavior

don`t hang