Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#2387: Gather hashed trace user events at the end of run #2395

Draft
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

cwschilly
Copy link
Contributor

Fixes #2387

Work in progress

Copy link

github-actions bot commented Jan 10, 2025

Pipelines results

PR tests (gcc-12, ubuntu, mpich, verbose, kokkos)

Build for 2daccb6 (2025-01-14 17:25:33 UTC)

FAILED: src/CMakeFiles/vt-trace.dir/vt/trace/trace_lite.cc.o 
/usr/bin/ccache /usr/lib/ccache/g++  -I/vt/src -I/build/vt/release/vt-trace -isystem /vt/lib/fmt/include -isystem /vt/lib/EngFormat-Cpp/include -isystem /vt/lib/yaml-cpp/include -O3 -DNDEBUG -fdiagnostics-color=always -std=c++17 -MD -MT src/CMakeFiles/vt-trace.dir/vt/trace/trace_lite.cc.o -MF src/CMakeFiles/vt-trace.dir/vt/trace/trace_lite.cc.o.d -o src/CMakeFiles/vt-trace.dir/vt/trace/trace_lite.cc.o -c /vt/src/vt/trace/trace_lite.cc
/vt/src/vt/trace/trace_lite.cc: In member function 'void vt::trace::TraceLite::writeTracesFile(int, bool)':
/vt/src/vt/trace/trace_lite.cc:546:7: error: 'runInEpochCollective' is not a member of 'vt'
  546 |   vt::runInEpochCollective([&]{
      |       ^~~~~~~~~~~~~~~~~~~~
/vt/src/vt/trace/trace_lite.cc: In lambda function:
/vt/src/vt/trace/trace_lite.cc:547:5: error: 'proxy' was not declared in this scope
  547 |     proxy.reduce<vt::collective::PlusOp>(0, std::move(user_event_));
      |     ^~~~~
/vt/src/vt/trace/trace_lite.cc:547:34: error: 'PlusOp' is not a member of 'vt::collective'
  547 |     proxy.reduce<vt::collective::PlusOp>(0, std::move(user_event_));
      |                                  ^~~~~~
FAILED: src/CMakeFiles/vt.dir/Unity/unity_16_cxx.cxx.o 
/usr/bin/ccache /usr/lib/ccache/g++ -DJSON_USE_IMPLICIT_CONVERSIONS=1 -DKOKKOS_DEPENDENCE -DMAGISTRATE_KOKKOS_ENABLED=1 -DMAGISTRATE_KOKKOS_KERNELS_ENABLED=0 -DVT_NO_COLOR_ENABLED -I/vt/lib/CLI -I/build/vt/release -I/vt/src -I/vt/lib/json/include -I/vt/lib/brotli/c/include -I/vt/lib/libfort/lib -isystem /vt/lib/fmt/include -isystem /vt/lib/EngFormat-Cpp/include -isystem /build/checkpoint/install/include -isystem /build/kokkos/build/install/include -isystem /vt/lib/yaml-cpp/include -O3 -DNDEBUG -Wall -Wextra -Wno-unknown-pragmas -Wnon-virtual-dtor -Wshadow -Wsign-compare -Wsuggest-override -pedantic -ftemplate-backtrace-limit=100 -Werror -std=c++17 -MD -MT src/CMakeFiles/vt.dir/Unity/unity_16_cxx.cxx.o -MF src/CMakeFiles/vt.dir/Unity/unity_16_cxx.cxx.o.d -o src/CMakeFiles/vt.dir/Unity/unity_16_cxx.cxx.o -c /build/vt/src/CMakeFiles/vt.dir/Unity/unity_16_cxx.cxx
In file included from /build/vt/src/CMakeFiles/vt.dir/Unity/unity_16_cxx.cxx:9:
/vt/src/vt/trace/trace_lite.cc: In lambda function:
/vt/src/vt/trace/trace_lite.cc:547:5: error: 'proxy' was not declared in this scope; did you mean 'vt::objgroup::proxy'?
  547 |     proxy.reduce<vt::collective::PlusOp>(0, std::move(user_event_));
      |     ^~~~~
      |     vt::objgroup::proxy
In file included from /vt/src/vt/objgroup/manager.static.h:49,
                 from /vt/src/vt/objgroup/manager.fwd.h:86,
                 from /vt/src/vt/serialization/messaging/serialized_messenger.impl.h:56,
                 from /vt/src/vt/serialization/messaging/serialized_messenger.h:114,
                 from %0D%0A%0D%0A%0D%0A ==> And there is more. Read log. <==

Build log


@cwschilly cwschilly changed the title #2387: Allreduce hashed trace user events at the end of run #2387: Gather hashed trace user events at the end of run Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Hashed trace user events should do an all-reduce at the end instead of sending pt2pts to rank 0 during the run
1 participant