v4.0.0

[isabelle-dr]

Overview & impact

This is tagged as a major release because the addition of the rules for Fares v2 base implementation is affecting the validity of existing datasets. We started to evaluate the impact of releases on production data with the acceptance tests, which make this validator run on all datasets from the Mobility Database, flagging additional errors with every code addition.

New rules

GTFS-Fares v2 base implementation, added in google/transit/pull/286

• fare_transfer_rule_duration_limit_type_without_duration_limit 🔴 ERROR 🔗
• fare_transfer_rule_duration_limit_without_type 🔴 ERROR 🔗
• fare_transfer_rule_invalid_transfer_count 🔴 ERROR 🔗
• fare_transfer_rule_missing_transfer_count 🔴 ERROR 🔗
• fare_transfer_rule_with_forbidden_transfer_count 🔴 ERROR 🔗
• invalid_currency_amount 🔴 ERROR 🔗

Transfers, changes in google/transit/pull/303 and google/transit/pull/284

• transfer_with_invalid_stop_location_type 🔴 ERROR 🔗
• transfer_with_invalid_trip_and_route 🔴 ERROR 🔗
• transfer_with_invalid_trip_and_stop 🔴 ERROR 🔗
• transfer_with_suspicious_mid_trip_in_seat 🟠 WARNING 🔗

Best Practice rules

• missing_recommended_file (applies to feed_info.txt) 🟠 WARNING 🔗
• missing_recommended_field (applies to feed_info.txt start_date, end_date, email, URL, version (🟠 WARNING) 🔗
• feed_expiration_date_7_days (replacing feed_expiration_date) 🟠 WARNING 🔗
• feed_expiration_date_30_days (replacing feed_expiration_date) 🟠 WARNING 🔗

What it means for existing data

On the approx. 1200 sources that are available to us, we found that 73 sources are affected by this release. The datasets affected mostly contain earlier versions of Fares v2 that don't match what was adopted in the spec, and uncertain cases, future versions.
The detailed analysis is in the Fares v2 base implementation for the validator document.

Producers affected

• Trillium Transit
• Maryland Department of Transportation
• San Diego Metropolitan Transit System

Complete list

Complete list of datasets affected

URL	Notices
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-norwalk-transit-system-nts-gtfs-102.zip?alt=media	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-victor-valley-transit-authority-vvta-gtfs-99.zip?alt=media	duplicate_key,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-city-of-tracy-tracer-gtfs-877.zip?alt=media	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-downeylink-gtfs-383.zip?alt=media	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-trinity-transit-gtfs-115.zip?alt=media	duplicate_key,foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-lake-transit-authority-lta-gtfs-68.zip?alt=media	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-auburn-transit-gtfs-217.zip?alt=media	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-ridgerunner-gtfs-311.zip?alt=media	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-san-diego-international-airport-metropolitan-transit-system-mts-gtfs-13.zip?alt=media	duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-nevada-county-connects-gold-country-stage-gtfs-84.zip?alt=media	duplicate_key,fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-santa-cruz-metro-scmtd-gtfs-1211.zip?alt=media	duplicate_key,invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-redding-area-bus-authority-raba-gtfs-114.zip?alt=media	duplicate_key,foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-corona-cruiser-gtfs-104.zip?alt=media	fare_transfer_rule_missing_transfer_count,foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-gtrans-gtfs-616.zip?alt=media	fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-ventura-county-transportation-commission-gtfs-31.zip?alt=media	fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-san-benito-county-express-sbce-gtfs-58.zip?alt=media	fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-redwood-coast-transit-rct-gtfs-20.zip?alt=media	fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-plumas-transit-gtfs-119.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-sage-stage-gtfs-113.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-mountain-transit-gtfs-107.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-merced-county-transit-the-bus-gtfs-86.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-thousand-oaks-transit-gtfs-33.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-lassen-rural-bus-lrb-gtfs-118.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-city-of-banning-transit-gtfs-220.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-morongo-basin-transit-authority-gtfs-579.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-amador-transit-gtfs-92.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-glenn-transit-service-gtfs-835.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-mendocino-transit-authority-gtfs-69.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-arcata-mad-river-transit-system-gtfs-19.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-yosemite-area-regional-transportation-system-yarts-gtfs-88.zip?alt=media	foreign_key_violation,invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-el-monte-transit-gtfs-103.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-avalon-transit-gtfs-12.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-city-of-madera-gtfs-91.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-mission-bay-transportation-management-association-tma-gtfs-51.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-porterville-transit-gtfs-48.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-delano-area-rapid-transit-gtfs-320.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-la-go-bus-gtfs-28.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-yubasutter-transit-gtfs-79.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-huntington-park-express-gtfs-558.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-colorado-el-dorado-transit-gtfs-75.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-santa-barbara-metropolitan-transit-district-mtd-gtfs-1246.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-clovis-transit-gtfs-89.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-simi-valley-transit-gtfs-40.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-kern-transit-gtfs-47.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-moorpark-city-transit-gtfs-578.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-guadalupe-flyer-gtfs-617.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-blossom-express-gtfs-236.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-etrans-gtfs-65.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-nevada-tahoe-transportation-district-gtfs-897.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-tahoe-area-regional-transit-tart-north-lake-tahoe-express-gtfs-94.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-camarillo-area-transit-gtfs-297.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-glendora-transportation-division-gtfs-609.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-pass-transit-gtfs-106.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-spirit-bus-gtfs-39.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-turlock-transit-gtfs-619.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-compton-renaissance-gtfs-314.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-south-county-transit-link-gtfs-817.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-tehama-rural-area-express-trax-susanville-indian-rancheria-public-transportation-program-gtfs-116.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-palos-verdes-peninsula-transit-authority-pvpta-gtfs-35.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-calaveras-transit-gtfs-93.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-eastern-sierra-transit-gtfs-25.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-santa-maria-area-transit-gtfs-26.zip?alt=media	invalid_currency_amount,missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-laguna-beach-transit-lbt-gtfs-16.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-siskiyou-county-gtfs-117.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-mountain-view-transportation-management-association-gtfs-60.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-needles-area-transit-gtfs-583.zip?alt=media	invalid_currency_amount
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-unitrans-gtfs-82.zip?alt=media	missing_required_column
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-tulare-county-area-transit-gtfs-646.zip?alt=media	missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-bellflower-bus-gtfs-227.zip?alt=media	missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-bell-gardens-trolley-gtfs-228.zip?alt=media	missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-lynwood-trolley-breeze-gtfs-570.zip?alt=media	missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-rosemead-explorer-gtfs-806.zip?alt=media	missing_required_field
https://storage.googleapis.com/storage/v1/b/mdb-latest/o/us-california-get-around-town-express-gtfs-606.zip?alt=media	missing_required_field

What's changed

Bug fixes

• fix: Sign and Notarize MacOS App by @bdferris-v2 in fix: Sign and Notarize MacOS App #1250
• fix: "Validate" button disappears after app font-size change by @bdferris-v2 in fix: "Validate" button disappears after app font-size change #1255
• fix: Do not return non-zero exit code when exporting schema via CLI. by @bdferris-v2 in fix: Do not return non-zero exit code when exporting schema via CLI. #1272
• fix: Bump queue_runner.sh max heap size to 10Gb by @bdferris-v2 in fix: Bump queue_runner.sh max heap size to 10Gb #1227
• fix(docker): update path to CLI jar by @themightychris in fix(docker): update path to CLI jar #1241

Rules

• feat: recommended file notice by @KClough in feat: recommended file notice #1151
• feat: apply recommended file notice to feed_info by @KClough in feat: apply recommended file notice to feed_info #1197
• feat: add recommended field notice by @KClough in feat: add recommended field notice #1157
• feat: feed_info should include start_date, end_date, version, email, url by @KClough in feat: feed_info should include start_date, end_date, version, email, url #1156
• feat: add trip-to-trip and route-to-route fields to transfers by @KClough in feat: add trip-to-trip and route-to-route fields to transfers #1208
• feat: Initial support for multiple @PrimaryKey annotations. by @bdferris-v2 in feat: Initial support for multiple @PrimaryKey annotations. #1190
• feat: Apply multiple @PrimaryKey annotations to Transfers by @KClough in feat: Apply multiple @PrimaryKey annotations to Transfers  #1220
• feat: Add @CurrencyAmount annotation for currency amount validation by @bdferris-v2 in feat: Add @CurrencyAmount annotation for currency amount validation #1248
• Replace feed_expiration_date with rules for 7 and 30 days by @KClough in Replace feed_expiration_date with rules for 7 and 30 days #1245
• feat: Initial support for Fares v2 validation. by @bdferris-v2 in feat: Initial support for Fares v2 validation. #1228
• feat: transfers.txt trip=>route reference validation. by @bdferris-v2 in feat: transfers.txt trip=>route reference validation. #1267
• feat: Add validation support for transfers.txt transfer_type 4 and 5. by @bdferris-v2 in feat: Add validation support for transfers.txt transfer_type 4 and 5. #1266

Documentation updates

• Docs: Add new contribution guideline by @isabelle-dr in Docs: Add new contribution guideline #1223
• docs: fix typos in RULES.md by @isabelle-dr in docs: fix typos in RULES.md #1209
• Docs: Add point near pole by @isabelle-dr in Docs: Add point near pole #1211
• docs: fix typo in RULES.md by @isabelle-dr in docs: fix typo in RULES.md #1217
• docs: Add example of validation report by @isabelle-dr in docs: Add example of validation report #1219
• Docs: Update README by @isabelle-dr in Docs: Update README #1222
• Docs: Documentation updates by @isabelle-dr in Docs: Documentation updates #1257
• Docs: Update README.md by @isabelle-dr in Docs: Update README.md #1259
• docs: fix typo in RULES.md by @isabelle-dr in docs: fix typo in RULES.md #1242
• fix(docs): remove duplicate prefix in example commits by @themightychris in fix(docs): remove duplicate prefix in example commits #1240

Other great stuff + general maintenance

• build(deps): bump numpy from 1.21.2 to 1.22.0 in /scripts/mobility-database-harvester by @dependabot in build(deps): bump numpy from 1.21.2 to 1.22.0 in /scripts/mobility-database-harvester #1192
• feat: Updates acceptance tests to use the Mobility Database Catalogs by @maximearmstrong in feat: Updates acceptance tests to use the Mobility Database Catalogs #1204
• feat(ci): handle gradle build in Dockerfile by @themightychris in feat(ci): handle gradle build in Dockerfile #1120
• feat: Add unit-tests for @PrimaryKey and @FirstKey/@SequenceKey by @bdferris-v2 in feat: Add unit-tests for @PrimaryKey and @FirstKey/@SequenceKey #1189
• feat: Add config.yml for welcome bot by @isabelle-dr in feat: Add config.yml for welcome bot #1212
• Fix: Update config.yml by @isabelle-dr in Fix: Update config.yml #1216
• Fix: Update config.yml by @isabelle-dr in Fix: Update config.yml #1221
• feat: Move gtfsvalidator.cli package into new cli sub-module. by @bdferris-v2 in feat: Move gtfsvalidator.cli package into new cli sub-module.  #1229
• feat: Switch gtfs-validator master to use cli/ module by @bdferris-v2 in feat: Switch gtfs-validator master to use cli/ module #1239
• feat: Refactor the output comparator to consider dropped errors by @bdferris-v2 in feat: Refactor the output comparator to consider dropped errors #1238
• feat: Lighten end_to_end.yml workflow by @maximearmstrong in feat: Lighten end_to_end.yml workflow #1243
• build(deps): bump protobuf from 3.17.3 to 3.18.3 in /scripts/mobility-database-harvester by @dependabot in build(deps): bump protobuf from 3.17.3 to 3.18.3 in /scripts/mobility-database-harvester #1260
• feat: Check validator subclasses are annotated with @GtfsValidator by @bdferris-v2 in feat: Check validator subclasses are annotated with @GtfsValidator #1263
• Move GTFS field names from table loader to entity classes by @aababilov in Move GTFS field names from table loader to entity classes #1274

New Contributors

• @themightychris made their first contribution in feat(ci): handle gradle build in Dockerfile #1120 🥳

Full Changelog: v3.1.1...v4.0.0

---

This discussion was created from the release v4.0.0.