feat: GTFS-RT feed validation & E012 compliance fix#64
feat: GTFS-RT feed validation & E012 compliance fix#64diveshpatil9104 wants to merge 3 commits intoOneBusAway:mainfrom
Conversation
aaronbrethorst
left a comment
There was a problem hiding this comment.
Divesh, this is a well-structured approach to GTFS-RT compliance — the E012 header timestamp fix is exactly right, the validateFeedCompliance test helper is a great reusable asset for catching regressions, and the test quality improvements (replacing Timestamp: 100 with time.Now().Unix() and time.Sleep with assert.Eventually) are welcome cleanups. The 21-test validation suite organized by MobilityData rule number makes it easy to verify coverage at a glance.
Before this can merge, there are a few things to address.
Critical Issues (2 found)
-
Merge conflicts with
main. GitHub reports this PR as CONFLICTING. You'll need to rebase againstmainto resolve. -
Branch is stale against PR #45 (
*float64pointer migration) — code will not compile after rebase. PR #45 changedBearing,Speed, andAccuracyfromfloat64to*float64on bothLocationReportandVehicleState. This PR uses barefloat64throughout:-
Validation (
handlers.go:67-71):r.Bearing < 0 || r.Bearing > 360andr.Speed < 0won't compile when these are*float64. After rebase, these checks must be conditional on non-nil — a nil bearing/speed (field not provided) should be allowed, not rejected. The validation should be:if r.Bearing != nil && (*r.Bearing < 0 || *r.Bearing > 360) { return fmt.Errorf("bearing must be between 0 and 360 (inclusive)") } if r.Speed != nil && *r.Speed < 0 { return fmt.Errorf("speed must be non-negative") }
-
buildFeed()(handlers.go:201-202):proto.Float32(float32(v.Bearing))andproto.Float32(float32(v.Speed))use bare value access, but onmainthese are pointers with conditional emission (see PR #45'sbuildFeedchanges that use nil guards). After rebase, you'll need to adopt the same nil-guard pattern. -
feed_validation_test.go: AllVehicleStateliterals use barefloat64fields (e.g.,Bearing: 180,Speed: 8.5) which won't compile against*float64. These need to use thefloat64ptr()helper. -
handlers_test.go: Same issue — the new validation test cases useBearing: -1,Speed: -5, etc. onLocationReport, which will needfloat64ptr().
-
Important Issues (1 found)
- E050 tension with ingest window is documented but may need a follow-up. The ingest window allows timestamps up to
now+300s, but E050 rejects>now+60s. The PR correctly documents this inTestFeedValidation_E050_TimestampsNotFarFutureand in the "Known Limitations" section. However, this means a vehicle with a timestamp 2 minutes in the future will pass ingest validation, appear in the feed, and fail the MobilityData validator for E050. Since the goal is to pass the validator without errors, this gap should be tracked for follow-up — either tighten the ingest window to 60s (breaking change) or clamp entity timestamps inbuildFeed()tomin(ts, now+60).
Suggestions (1 found)
- Consider
float64ptr()boundary tests for the pointer migration. Once you rebase and adopt*float64, you'll want test cases that verify nil bearing/speed passes validation (no rejection), explicit zero passes (bearing=0 is north, speed=0 is stationary), and out-of-range values are rejected. The current test cases cover the range checks well but will need the nil-is-valid case added.
Strengths
- E012 fix is clean and correct:
headerTimestamp = max(now, maxEntityTimestamp)is a simple, correct solution - Zero-timestamp vehicle skip prevents W001 violations in the feed with an appropriate
slog.Warn validateFeedCompliancehelper is a great reusable asset — it testsbuildFeed()through the lens of every applicable MobilityData rule- Test quality improvements: replacing hardcoded
Timestamp: 100with realistictime.Now().Unix()and flakytime.Sleepwithassert.Eventually - 21 well-organized tests covering E001, E012, E026, E027, E038, E039, E048, E049, E050, E052, W001, W002
- Protobuf round-trip test (
TestFeedValidation_FeedSerializesCleanly) verifies the feed survives marshal/unmarshal
Recommended Action
- Rebase against
mainto resolve merge conflicts - Adapt all bearing/speed code to the
*float64pointer types from PR #45 - Add nil-bearing/speed validation test cases
- Consider tracking the E050/ingest-window tension for follow-up
- Re-run review after fixes
9c83b54 to
a14bfd5
Compare
|
hii aaron! I Rebased against main and resolved all conflicts. Changes:
|
Summary
Why
Milestone 1 exit criteria requires the feed to pass the https://github.com/MobilityData/gtfs-realtime-validator without errors. An audit found two violations:
What Changed
Known Limitations