Skip to content

96 optionally support gtfs rt data#114

Open
ankoure wants to merge 23 commits intomainfrom
96-optionally-support-gtfs-rt-data
Open

96 optionally support gtfs rt data#114
ankoure wants to merge 23 commits intomainfrom
96-optionally-support-gtfs-rt-data

Conversation

@ankoure
Copy link
Contributor

@ankoure ankoure commented Aug 30, 2025

I remembered I had a project from a few months back where I was trying to convert GTFS_RT data to Moving Feature JSON.

https://anitagraser.com/2024/07/08/new-movingpandas-tutorial-taking-ogc-moving-features-full-circle-with-mf-json/

I was able to merge in most of the code that I had previously written and "gobblified" it as much as possible.

The main missing components for this one are:

  1. Conventions surrounding supporting data from other agencies GTFS-RT feeds. For example adding another tier of folder in "data" to separate "MTBA" from any other.
  2. Integrating proper routing of Static GTFS References
  3. Integrating the GTFS-RT processing into it's own thread for each feed
  4. Cleaning up the added code to make it more readable/ fix linting issues
  5. Unit Testing

Minor concerns

  1. Adding support for agencies such as the MTA that use custom protobufs to add additional fields

@ankoure ankoure linked an issue Aug 30, 2025 that may be closed by this pull request
@ankoure
Copy link
Contributor Author

ankoure commented Sep 3, 2025

This one currently tracks each of the GTFS entities using 3 different classes. It currently works and have tested it against data from Cleveland RTA, SEPTA, and the Pioneer Valley Transit Authority.

Wanted to jot this thought down before I forgot it:
This process could probably be "gobblified" if each GTFS entity was morphed into the same JSON structure as MBTA V3 API and then passed as an "update" through the Trip State Management Class. If gobble already handles de-duplication then this will work directly, otherwise trimming down the VehicleEntity class to just track a unique identifier, direction id, and time of last update would help to only pass true "updates" to the data.

Some agencies use a JSON rest API instead of GTFS-RT for vehicle positions. Therefore it would be nice to lump all APIs that require polling together would be nice. For the custom JSON, a mapping of the fields to standard field names would need to be put in place before passing it into standard gobble logic.

The more labor intensive alternative would be to create an Server-Sent Event API that would consume GTFS-RT or custom JSON and output the same schema as the MBTA.

@ankoure ankoure marked this pull request as ready for review September 25, 2025 13:23
@ankoure
Copy link
Contributor Author

ankoure commented Sep 25, 2025

@devinmatte I have most of the parsing logic done I just need some help with the threading logic and the trips state manager, hence the ready for review.

@devinmatte
Copy link
Member

How do we run to test this? Normally we just run src/gobble.py

@ankoure
Copy link
Contributor Author

ankoure commented Sep 25, 2025

I set it up so we can run poetry run python src/gtfs_rt.py to test just the GTFS-RT aspect

@ankoure ankoure mentioned this pull request Nov 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optionally support GTFS-RT data

2 participants