r5r not building network.dat file; unable to detect error but believe coming from GTFS files

[tripwright]

Brief description of the problem:

Hi there! I want to preface that I'm aware others have been experiencing issues with r5r returning

Finished building network.dat at filepath/network.dat

but no network.dat file exists in the r5r data path directory.

I've gone through multiple threads on this Issues page to try and diagnose my code issue with no luck. I'm working with the Los Angeles metropolitan area (12238.08 km²), which is much smaller than the R5 bbox limit of the limit of 975000 km².

I do believe this issue to be coming from my GTFS files because the network does build when I remove the GTFS files from my r5r data path. I'm working with 52 (I know, quite a lot) GTFS files, and I have run each one through the Canonical GTFS Schedule Validator from MobilityData, making sure that the calendar dates were valid (many were not, and I corrected those issues when applicable).

I'm attaching a copy of the verbose text from when I run setup_r5 in hopes of getting some clarity as to what may be causing the code to break. I'm not able to make out any obvious errors from the verbose code, and therefore, am looking to you experts.

End of verbose = T output

Any tips or thoughts as to what may be occuring or what potential errors/warnings I should be looking for in my GTFS files would be greatly appreciated.

Reproducible code:

Data

library(r5r)

# build transport network
LA_r5r_path <- "LA_Urbanr5rData"

LA_r5r_core <- setup_r5(data_path = LA_r5r_path,
                        elevation = "TOBLER",
                        verbose = T)

Situation report:

$r5r_package_version
[1] ‘2.0’

$r5_jar_version
[1] "7.1"

$java_version
[1] "21.0.2"

$set_memory
[1] "-Xmx50G"

$session_info
R version 4.4.2 (2024-10-31)
Platform: aarch64-apple-darwin20
Running under: macOS Sequoia 15.1.1

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] r5r_2.0         sf_1.0-19       lubridate_1.9.3 forcats_1.0.0   stringr_1.5.1   dplyr_1.1.4    
 [7] purrr_1.0.2     readr_2.1.5     tidyr_1.3.1     tibble_3.2.1    ggplot2_3.5.1   tidyverse_2.0.0
[13] devtools_2.4.5  usethis_3.0.0  

loaded via a namespace (and not attached):
 [1] gtable_0.3.6       htmlwidgets_1.6.4  remotes_2.5.0      rJava_1.0-11       tzdb_0.4.0        
 [6] vctrs_0.6.5        tools_4.4.2        generics_0.1.3     proxy_0.4-27       fansi_1.0.6       
[11] pkgconfig_2.0.3    KernSmooth_2.23-24 data.table_1.16.2  checkmate_2.3.2    lifecycle_1.0.4   
[16] compiler_4.4.2     farver_2.1.2       munsell_0.5.1      httpuv_1.6.15      htmltools_0.5.8.1 
[21] class_7.3-22       later_1.3.2        pillar_1.9.0       urlchecker_1.0.1   ellipsis_0.3.2    
[26] classInt_0.4-10    cachem_1.1.0       wk_0.9.4           sessioninfo_1.2.2  mime_0.12         
[31] tidyselect_1.2.1   digest_0.6.37      stringi_1.8.4      fastmap_1.2.0      grid_4.4.2        
[36] colorspace_2.1-1   cli_3.6.3          magrittr_2.0.3     pkgbuild_1.4.5     utf8_1.2.4        
[41] e1071_1.7-16       withr_3.0.2        scales_1.3.0       promises_1.3.0     backports_1.5.0   
[46] timechange_0.3.0   hms_1.1.3          memoise_2.0.1      shiny_1.9.1        miniUI_0.1.1.1    
[51] s2_1.1.7           profvis_0.4.0      rlang_1.1.4        Rcpp_1.0.13-1      xtable_1.8-4      
[56] glue_1.8.0         DBI_1.2.3          pkgload_1.4.0      rstudioapi_0.17.1  R6_2.5.1          
[61] fs_1.6.5           units_0.8-5  

[rafapereirabr]

Hi @tripwright , thanks for the detailed issue and apologies for my slow response. @mvpsaraiva and I are still trying to detect the source of the problem. In the meantime, could you please try building the network using only the OSM .pbf file in yout data_path ?

on a side note, the log of the verbose code included this message that you might wan't to pay attention to:

2024-11-26 08:50:25,267 [main] ERROR o.i.r.N.TransitLayerWithShapes - agency time zone America/New_York differs from TransportNetwork time zone: America/Los_Angeles. This will be problematic.

[tripwright]

Hi @rafapereirabr, thanks for the response...no worries! And thank you for the LinkedIn comment the other day too. I was successful in building the network using just the OSM.pbf file in my data_path. Here's what the last part of the verbose code looks like verbose2.txt.

Regarding your other comment, yes, I noticed that time zone error too. How does one make sure they are aligned? I looked for an argument to specify a time zone but didn't find one. I presume the "agency time zone" is pulled from the GTFS files and the "TransportNetwork time zone" from the OSM.pbf files? Help here would be greatly appreciated.

[rafapereirabr]

I'm glad to hear it works with the OSM data only. This is a sign the cause of the bug is in the GTFS data or in how R5 is dealing with them. We're making progress in our investigation and I hope we have some updates soon.

As for the time zone question, I understand you're right. R5 gets the "agency time zone" from the GTFS files and the "TransportNetwork time zone" from the OSM file. In this case, the two time zones are different and this seems to be causing the error. Perhaps one more of your GTFS feeds are in the time zone America/New_York . Have you checked this?

[tripwright]

Awesome! Thanks for the clarification about the time zones. I just went through all 52 GTFS files for the LA Metro area and found a pesky one (Gardena GTrans) where the time zone was set to America/New_York...I updated it to America/Los_Angeles. I removed the old file and added the updated Gardena GTrans GTFS file to the data.

[rafapereirabr]

Hi @tripwright , thanks for sharing the data. I managed to build the transport network (the network.dat file was saved successfully) using the data you shared. Is it working for you as well ?

[tripwright]

@rafapereirabr I'm glad to hear that, but I still have no luck. I just tried it three separate times to no avail. Here's what the verbose text looks like when it breaks verbose3.txt. I'm using the same data attached above with the GTFS file with the fixed time zone. Any idea what could be causing it to break on my end? I'm using a device with 64 GiB of memory, so I don't suspect that to be the issue.

[rafapereirabr]

Hmmm. Strange. Because it works fine for me on a Windows machine. I noticed you're using a MacOS with an arm64 processor, which seems to have a compatibility issues with Java. See this issue.

Can you please try running the code with these data on another machine ?

[tripwright]

Yes, so initially I was using my personal MacBook, but the memory on the machine was not sufficient to run the code. I ended up switching to a Linux machine, running the code remotely with AWS. The error that I’ve shared with you has occurred on both the MacOS machine and Linux machine.

As well, I’ve been able to successfully build network.dat files for other cities for this project like Miami, Philadelphia, Houston, Dallas, etc. with no problem.

[rafapereirabr]

Hmm. Strange. You said you can build the network.dat for LA when you keep only the OSM files in the directory, right? If this is the case, the source of the error comes from one or more GTFS feeds.

But as I said, I get the code to run correctly with all the data files you shared. I cannot reproduce your new error on my windows machine

[tripwright]

Yes, that's right. The network.dat file builds without GTFS files in the r5r directory. Again, glad you're able to have it build on your machine. Frustrated that it won't build on mine...haha!