[OSRM] New Binary

[jeremiahpslewis]

Release plan:

• Once oneTBB PR [oneTBB] Version bump #6115 is merged, reactivate windows
• Get a new OSRM release tagged New Release Tag Project-OSRM/osrm-backend#6516
• Merge this PR and ship binary for linux
• Once mac builds are debugged, then add and ship them

[jeremiahpslewis]

@mattwigway Hey! Once this PR gets to ✅ and OSRM tags a new release, this should be the OSRM_jll.jl package you seem to have been looking for. :)

(note, at the moment only packaging the executables, do you also need the lib's?)

[mattwigway]

@jeremiahpslewis that's great! Yes, I need the libs as well. Let me know if there's anything I can do to help. I've been working on creating a JLL myself that's nearly done, perhaps we can combine efforts.

[jeremiahpslewis]

@jeremiahpslewis that's great! Yes, I need the libs as well. Let me know if there's anything I can do to help. I've been working on creating a JLL myself that's nearly done, perhaps we can combine efforts.

Gladly, perhaps this PR is a good starting point for a collaboration? Do you have any idea how to get the osrm build to output dynamically linked libraries?

[jeremiahpslewis]

@mattwigway Just gave you access to the branch / fork so you can make changes.

[mattwigway]

@jeremiahpslewis thanks, I'll take a look. Allegedly -DBUILD_SHARED_LIBS=ON should do the trick, but I'm getting some linker errors when I try that.

[mattwigway]

I also found that using the toolchain file for CMake avoided me needing to specify all the library locations: https://github.com/JuliaPackaging/BinaryBuilder.jl/blob/master/docs/src/build_tips.md#cmake-builds

[mattwigway]

Attempting a build with the shared libraries now.

[jeremiahpslewis]

I just ran it locally with your -D command, seems to work, so I've added the library products as well. :)

[mattwigway]

It's possible the error I was running into is apple M1 specific - will see momentarily

[mattwigway]

I was thinking it might also make sense to add the profiles (car.lua and friends) as FileProducts since OSRM is not very useful without these.

[jeremiahpslewis]

@mattwigway Apple M1 is brilliant for Julia but not so good for BinaryBuild-ing. Have you tried using Yggdrasil via GitHub codespaces? It's all wired up so relatively painless. :)

[jeremiahpslewis]

I was thinking it might also make sense to add the profiles (car.lua and friends) as FileProducts since OSRM is not very useful without these.

Sure, fine by me. Want to add it to the PR yourself?

[mattwigway]

Will do.

[mattwigway]

I'm building on x86, but the aarch64 mac target is failing. I think this is an upstream issue though. I'm trying to build OSRM outside of BinaryBuilder on an actual M1 Mac, and if that fails too I'll file an issue with the OSRM project. I think if that's the case we should just remove aarch64-apple-darwin as a supported platform from the jll for now.

[mattwigway]

Yep, same error when building natively on M1.

[jeremiahpslewis]

@mattwigway ok, so we are ✅ for linux, but windows and Mac seem to be blocked (upstream?)

Release plan:

• Get a new OSRM release tagged New Release Tag Project-OSRM/osrm-backend#6516
• Merge this PR and ship binary for linux
• Once mac / windows builds are debugged, then add and ship them

[mattwigway]

Thanks! I'll look quickly at the Windows build situation as it would be ideal if I had a Windows binary available (I know, I know, I hate Windows too... but unfortunately for the project I'm working on my hands are tied).

…

On Fri, Jan 20, 2023 at 7:30 AM Jeremiah ***@***.***> wrote: @mattwigway <https://github.com/mattwigway> ok, so we are ✅ for linux, but windows and Mac seem to be blocked (upstream?) Release plan: - Get a new OSRM release tagged Project-OSRM/osrm-backend#6516 <Project-OSRM/osrm-backend#6516> - Merge this PR and ship binary for linux - Once mac / windows builds are debugged, then add and ship them — Reply to this email directly, view it on GitHub <#6106 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEKNLR4RNMN7SE2XNXEI6TWTKAPNANCNFSM6AAAAAAT7UWXBM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mattwigway]

No luck getting the Windows binaries built - endless errors with mingw redefining GCC internal functions, and TBB not finding things from float.h. I'm giving up on Windows for now.

[jeremiahpslewis]

@mattwigway the latest oneTBB is also a disaster for windows, so until someone upstream makes this happen, think we‘ll have to stay content with Linux (and Mac someday?)

do you know anyone you could ping to get a new version tagged of osrm?

[giordano]

and TBB not finding things from float.h

#6115 (comment)

[jeremiahpslewis]

Mac (aarch64 error); x86_64 apple has the same error when the SDK is upgraded to 10.15:

Current OS X Error:

Entering directory '/workspace/srcdir/osrm-backend/build'

[ 86%] Linking CXX shared library libosrm_guidance.dylib

/usr/bin/cmake -E cmake_link_script CMakeFiles/osrm_guidance.dir/link.txt --verbose=1
/opt/bin/aarch64-apple-darwin20-libgfortran5-cxx11/aarch64-apple-darwin20-clang++ --sysroot=/opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root  -Wall -Wextra -Wpedantic -Werror -Wstrict-overflow=2 -Wsuggest-override -Wsuggest-destructor-override -Wunused -Wunreachable-code -Wdelete-incomplete -Wdisabled-optimization -Winit-self -Wlogical-not-parentheses -Wmisleading-indentation -Wodr -Wpointer-arith -Wredundant-decls -Wreorder -Wshift-negative-value -Wsizeof-array-argument -Wswitch-bool -Wtautological-compare -Wno-c++17-extensions -Wno-implicit-int-conversion -Wno-implicit-float-conversion -Wno-unused-member-function -Wno-old-style-cast -Wno-non-virtual-dtor -Wno-float-conversion -Wno-sign-conversion -Wno-shorten-64-to-32 -Wno-padded -Wno-missing-noreturn -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fPIC -fcolor-diagnostics -ftemplate-depth=1024 -O3 -DNDEBUG -mmacosx-version-min=11.0 -dynamiclib -Wl,-headerpad_max_install_names -o libosrm_guidance.dylib -install_name @rpath/libosrm_guidance.dylib CMakeFiles/GUIDANCE.dir/src/extractor/intersection/coordinate_extractor.cpp.o CMakeFiles/GUIDANCE.dir/src/extractor/intersection/have_identical_names.cpp.o CMakeFiles/GUIDANCE.dir/src/extractor/intersection/intersection.cpp.o CMakeFiles/GUIDANCE.dir/src/extractor/intersection/intersection_analysis.cpp.o CMakeFiles/GUIDANCE.dir/src/extractor/intersection/mergable_road_detector.cpp.o CMakeFiles/GUIDANCE.dir/src/extractor/intersection/node_based_graph_walker.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/driveway_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/guidance_processing.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/intersection_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/motorway_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/roundabout_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/segregated_intersection_classification.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/sliproad_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/suppress_mode_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_analysis.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_classification.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_discovery.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_lane_augmentation.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_lane_data.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_lane_handler.cpp.o CMakeFiles/GUIDANCE.dir/src/guidance/turn_lane_matcher.cpp.o CMakeFiles/UTIL.dir/src/util/assert.cpp.o CMakeFiles/UTIL.dir/src/util/conditional_restrictions.cpp.o CMakeFiles/UTIL.dir/src/util/coordinate.cpp.o CMakeFiles/UTIL.dir/src/util/coordinate_calculation.cpp.o CMakeFiles/UTIL.dir/src/util/exception.cpp.o CMakeFiles/UTIL.dir/src/util/fingerprint.cpp.o CMakeFiles/UTIL.dir/src/util/geojson_debug_policies.cpp.o CMakeFiles/UTIL.dir/src/util/guidance/bearing_class.cpp.o CMakeFiles/UTIL.dir/src/util/guidance/entry_class.cpp.o CMakeFiles/UTIL.dir/src/util/guidance/turn_lanes.cpp.o CMakeFiles/UTIL.dir/src/util/log.cpp.o CMakeFiles/UTIL.dir/src/util/opening_hours.cpp.o CMakeFiles/UTIL.dir/src/util/timed_histogram.cpp.o CMakeFiles/UTIL.dir/src/util/timezones.cpp.o  -Wl,-rpath,@loader_path 

...

[31m  "_crc32_combine", referenced from:�[39m

[31m      __ZN3tbb6detail2d115concrete_filterINSt3__110shared_ptrIZN4osrm8guidance13annotateTurnsERKNS5_4util12DynamicGraphINS7_17NodeBasedEdgeDataEEERKNS5_9extractor6detail30EdgeBasedNodeDataContainerImplILNS5_7storage9OwnershipE0EEERKNS3_6vectorINS7_10CoordinateENS3_9allocatorISM_EEEERKNSD_23CompressedEdgeContainerERKNS3_13unordered_setIjNS3_4hashIjEENS3_8equal_toIjEENSN_IjEEEERKNSD_18NodeRestrictionMapINSD_17UnconditionalOnlyEEERKNSD_17WayRestrictionMapERKNSE_13NameTableImplILSH_0EEERKNSD_11SuffixTableERKNS3_5tupleIJNSL_IjS10_EENSL_ItNSN_ItEEEEEEERNS7_15ConcurrentIDMapIS1M_tNSD_24TurnLaneDescription_hashEEERNS1Q_INS7_8guidance15LaneTupleIdPairEtN5boost4hashIS1V_EEEERNS6_6detail21TurnDataContainerImplILSH_2EEERS1K_RNS1Q_INS1U_12BearingClassEjNSW_IS26_EEEERNS1Q_INS1U_10EntryClassEtNSW_IS2A_EEEERjE19TurnsPipelineBufferEEvZNS6_13annotateTurnsESC_SK_SR_SU_S13_S18_S1B_S1F_S1I_S1P_S1T_S20_S24_S25_S29_S2D_S2E_E3$_2EclEPv in guidance_processing.cpp.o�[39m

[31mld: symbol(s) not found for architecture arm64�[39m

[giordano]

#5425 (comment)

If you want to make everybody's life easier, when you show linking errors you also want to show the compiler invocation line. Because the undefined reference to error means that the linker can't find the mentioned symbol in any of the linked libraries, which often simply means that the library which is supposed to implement the missing symbol wasn't passed as argument. But if I don't see the compiler invocation I can't tell you "see, that's the problem".

[jeremiahpslewis]

#5425 (comment)

If you want to make everybody's life easier, when you show linking errors you also want to show the compiler invocation line. Because the undefined reference to error means that the linker can't find the mentioned symbol in any of the linked libraries, which often simply means that the library which is supposed to implement the missing symbol wasn't passed as argument. But if I don't see the compiler invocation I can't tell you "see, that's the problem".

Thanks, will update!

[jeremiahpslewis]

Mingw error (x86-64-w64-mingw32-cxx03), which also pops up on apple-darwin; adding -fext-numeric-literals to CMAKE_CXX_FLAGS didn't seem to help.

/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/usr/local/include/boost/math/constants/constants.hpp:246:3: error: unable to find numeric literal operator ‘operator""Q’�[39m

  246 |   BOOST_DEFINE_MATH_CONSTANT(half, 5.000000000000000000000000000000000000e-01, "5.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e-01")�[39m

      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~�[39m

/opt/x86_64-w64-mingw32/x86_64-w64-mingw32/sys-root/usr/local/include/boost/math/constants/constants.hpp:246:3: note: use ‘-fext-numeric-literals’ to enable more built-in suffixes