<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" version="2.0"><channel><title><![CDATA[511.org SF Bay Regional Feed — Interline Technologies]]></title><description><![CDATA[Interline operates the 511.org Regional GTFS Feed, Regional GTFS Realtime Feed, and associated data services for the Metropolitan Transportation Commission and the 30+ transit agencies of the San Francisco Bay Area.]]></description><link>https://www.interline.io/</link><image><url>https://www.interline.io/favicon.ico</url><title>511.org SF Bay Regional Feed — Interline Technologies</title><link>https://www.interline.io/</link></image><lastBuildDate>Tue, 04 Oct 2022 07:00:00 GMT</lastBuildDate><atom:link href="https://www.interline.io/blog/tag/511-sf-bay-regional-feed/rss.xml" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title><![CDATA[Updated GTFS-Fares v2 for the SF Bay Area]]></title><description><![CDATA[MTC and Interline have released updated fares and transfer discounts for over 30 Bay Area transit agencies using the adopted GTFS-Fares v2 "base implementation" and a series of proposed extensions. ]]></description><link>https://www.interline.io/blog/mtc-regional-gtfs-feed-fares-updates/</link><guid isPermaLink="false">672550a2b62841000136ed3b</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Tue, 04 Oct 2022 07:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/ios-directions-with-fares-2.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/ios-directions-with-fares-2.png" alt="Updated GTFS-Fares v2 for the SF Bay Area"><blockquote><p><strong>April 20, 2023</strong>: The GTFS-Fares v2 extension originally named <code>fare_containers.txt</code> has been renamed to <code>fare_media.txt</code>, as part its recent <a href="https://github.com/google/transit/pull/355">adoption in the GTFS specification</a>.</p></blockquote><p>In 2020, Interline and the Metropolitan Transportation Commission <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#fares-and-transfer-discounts">released</a> a beta version of fares and transfer discounts for eight SF Bay Area transit agencies. Since then, we’ve been working together to improve data coverage and its schema. As of this August, the Regional GTFS Feed now includes fares and transfer discounts for over 30 Bay Area transit agencies. The updated dataset uses the adopted GTFS-Fares v2 “base implementation,” as well a few proposed additions to the GTFS-Fares v2 data schema. This data is now available for use, and our team is updating it on an ongoing basis.</p><p>In this blog post we’ll cover:</p><ul><li><a href="#gtfs-fares-v2-in-action">GTFS-Fares v2 in action</a></li><li><a href="#gtfs-fares-v2-base-implementation-and-future-additions-to-the-specification">GTFS-Fares v2 base implementation and future additions to the specification</a></li><li><a href="#sf-bay-area-transit-agencies-with-gtfs-fares-v2-data">SF Bay Area transit agencies with GTFS-Fares v2 data</a></li><li><a href="#using-gtfs-fares-v2-data-for-the-bay-area">Using GTFS-Fares v2 data for the Bay Area</a></li></ul><h2 id="gtfs-fares-v2-in-action">GTFS-Fares v2 in action</h2><p>If you have an Apple iPhone running <a href="https://www.apple.com/ios/ios-16/">iOS 16</a>, you can see the estimated costs of transit journeys. This data is sourced from the the Regional Feed.</p><p>All of the above fares information is defined in the Regional GTFS Feed using the GTFS-Fares v2 specification. Here are extracts of the relevant GTFS files from the feeds for the itinerary displayed on the iPhone:</p><h3 id="fare_productstxt">fare_products.txt</h3><table><thead><tr><th>fare_product_id</th><th>fare_product_name</th><th>amount</th><th>currency</th><th>rider_category_id</th><th>fare_media_id</th></tr></thead><tbody><tr><td><code>AC:local:single</code></td><td>AC Transit local fare</td><td>2.25</td><td>USD</td><td><code>adult</code></td><td><code>clipper</code></td></tr><tr><td><code>BA:matrix:FTVL-EMBR</code></td><td>generated</td><td>2.15</td><td>USD</td><td><code>adult</code></td><td><code>clipper</code></td></tr></tbody></table><h3 id="fare_mediatxt-updated">fare_media.txt Updated</h3><table><thead><tr><th>fare_media_id</th><th>fare_media_name</th><th>fare_media_type</th></tr></thead><tbody><tr><td><code>clipper</code></td><td>Clipper</td><td>2</td></tr></tbody></table><h3 id="fare_leg_rulestxt">fare_leg_rules.txt</h3><table><thead><tr><th>leg_group_id</th><th>from_area_id</th><th>to_area_id</th><th>network_id</th><th>fare_product_id</th></tr></thead><tbody><tr><td><code>AC</code></td><td><code>AC:local</code></td><td><code>AC:local</code></td><td><code>AC</code></td><td><code>AC:local:single</code></td></tr><tr><td><code>BA</code></td><td><code>FTVL</code></td><td><code>EMBR</code></td><td><code>BA</code></td><td><code>BA:matrix:FTVL-EMBR</code></td></tr></tbody></table><h3 id="stop_areastxt">stop_areas.txt</h3><table><thead><tr><th>area_id</th><th>stop_id</th></tr></thead><tbody><tr><td><code>AC:local</code></td><td><code>55227</code></td></tr><tr><td><code>AC:local</code></td><td><code>55550</code></td></tr><tr><td><code>FTVL</code></td><td><code>FTVL</code></td></tr><tr><td><code>EMBR</code></td><td><code>EMBR</code></td></tr></tbody></table><p>Note that this data schema has changed since the examples in <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#an-example-of-fares-v2-in-practice">our first release blog post</a>. This new schema was adopted as part of the GTFS-Fares v2 “base implementation,” which we will now discuss in more detail.</p><h2 id="gtfs-fares-v2-base-implementation-and-future-additions-to-the-specification">GTFS-Fares v2 base implementation and future additions to the specification</h2><p>Since the <a href="https://bit.ly/gtfs-fares">original GTFS-Fares v2 proposal</a>, many stakeholders in GTFS have provided additional input. In May, stakeholders voted to adopt the core of the refined GTFS-Fares v2 proposal as a <a href="https://github.com/google/transit/pull/286">“base implementation”</a>.</p><p>The “base implementation” simplifies the organization of <code>fare_leg_rules.txt</code>, <code>fare_transfer_rules.txt</code>, and <code>fare_products.txt</code>. This is a breaking change, so consumers who implemented their GTFS processing software against the original proposal will need to update their tools.</p><p>To fully capture the complexity of Bay Area transit fares and transfer discounts requires additional data tables and fields that were included in the original proposal but not in the “base implementation.”</p><p>Here is an overview of all the GTFS-Fares v2 tables and columns/fields included in the Regional GTFS Feed:</p><table><thead><tr><th>GTFS file</th><th>Spec version(s)</th><th>Columns added for extensions</th><th>Documentation</th></tr></thead><tbody><tr><td><code>fare_products.txt</code></td><td>&quot;base&quot; with additional columns for extensions</td><td><ul><li><code>rider_category_id</code></li></ul></td><td><a href="https://gtfs.org/schedule/reference/#fare_productstxt">See gtfs.org reference</a></td></tr><tr><td><code>fare_leg_rules.txt</code></td><td>&quot;base&quot; with additional columns for extensions</td><td><ul><li><code>transfer_only</code>*</li></ul></td><td><a href="https://gtfs.org/schedule/reference/#fare_leg_rulestxt">See gtfs.org reference</a></td></tr><tr><td><code>fare_transfer_rules.txt</code></td><td>&quot;base&quot; with additional columns for extensions</td><td><ul><li><code>filter_fare_product_id</code>*</li></ul></td><td><a href="https://gtfs.org/schedule/reference/#fare_transfer_rulestxt">See gtfs.org reference</a></td></tr><tr><td><code>areas.txt</code></td><td>&quot;base&quot;</td><td></td><td><a href="https://gtfs.org/schedule/reference/#areastxt">See gtfs.org reference</a></td></tr><tr><td><code>stop_areas.txt</code></td><td>&quot;base&quot;</td><td></td><td><a href="https://gtfs.org/schedule/reference/#stop_areastxt">See gtfs.org reference</a></td></tr><tr></tr><tr><td><code>fare_media.txt</code></td><td>&quot;base&quot; as of March 14, 2023 Updated</td><td></td><td><a href="https://gtfs.org/schedule/reference/#fare_mediatxt">See gtfs.org reference</a></td></tr><tr><td><code>rider_categories.txt</code></td><td>extension</td><td><ul><li><code>rider_category_id</code></li><li><code>rider_category_name</code></li><li><code>min_age</code></li><li><code>max_age</code></li><li><code>eligibility_url</code></li></ul></td><td><a href="https://docs.google.com/document/d/19j-f-wZ5C_kYXmkLBye1g42U-kvfSVgYLkkG5oyBauY/edit#heading=h.9gqyl4w7f6x1">See proposal Google Doc</a></td></tr><tr><td><code>routes.txt</code></td><td>extension</td><td><ul><li><code>as_route</code></li></ul></td><td><a href="https://docs.google.com/document/d/19j-f-wZ5C_kYXmkLBye1g42U-kvfSVgYLkkG5oyBauY/edit#heading=h.v160xvt5oen5">See proposal Google Doc</a></td></tr></tbody></table><p>* <code>fare_leg_rules.txt transfer_only</code> and <code>fare_transfer_rules.txt filter_fare_product_id</code> are two fields proposed for describing partial credit on inter-agency transfers. Thanks to the <a href="https://transitapp.com/">Transit</a> app team for proposing these additions to make some of the transfer discounts in the Regional GTFS Feed more explicit for consuming software to parse.</p><p>Interline and MTC will continue to produce these “extensions” to the “base” data schema, and we’ll continue to work with data consumers, <a href="https://mobilitydata.org/">MobilityData</a>, and other stakeholders to finalize their adoption into the GTFS spec.</p><h2 id="sf-bay-area-transit-agencies-with-gtfs-fares-v2-data">SF Bay Area transit agencies with GTFS-Fares v2 data</h2><p>Interline and MTC now produce fares, intra-agency transfer discounts, and inter-agency transfer discounts for the following agencies:</p><table><thead><tr><th>511 operator code</th><th>Operator full name</th></tr></thead><tbody><tr><td><code>3D</code></td><td>Tri Delta Transit</td></tr><tr><td><code>AC</code></td><td>AC Transit (including transbay routes)</td></tr><tr><td><code>AF</code></td><td>Angel Island Tiburon Ferry</td></tr><tr><td><code>AM</code></td><td>Capitol Corridor</td></tr><tr><td><code>BA</code></td><td>Bay Area Rapid Transit (BART)</td></tr><tr><td><code>CC</code></td><td>County Connection</td></tr><tr><td><code>CE</code></td><td>Altamont Corridor Express (ACE)</td></tr><tr><td><code>CM</code></td><td>Commute.org Shuttles</td></tr><tr><td><code>CT</code></td><td>Caltrain</td></tr><tr><td><code>DE</code></td><td>Dumbarton Express Consortium</td></tr><tr><td><code>EM</code></td><td>Emery Go-Round</td></tr><tr><td><code>FS</code></td><td>Fairfield and Suisun Transit</td></tr><tr><td><code>GF</code></td><td>Golden Gate Ferry</td></tr><tr><td><code>GG</code></td><td>Golden Gate Transit</td></tr><tr><td><code>MA</code></td><td>Marin Transit</td></tr><tr><td><code>MV</code></td><td>MVgo Mountain View</td></tr><tr><td><code>PE</code></td><td>Petaluma Transit</td></tr><tr><td><code>RV</code></td><td>Rio Vista Delta Breeze</td></tr><tr><td><code>SA</code></td><td>Sonoma Marin Area Rail Transit</td></tr><tr><td><code>SB</code></td><td>San Francisco Bay Ferry</td></tr><tr><td><code>SC</code></td><td>Valley Transportation Authority (VTA)</td></tr><tr><td><code>SF</code></td><td>San Francisco Municipal Transportation Agency (SFMTA)</td></tr><tr><td><code>SI</code></td><td>San Francisco International Airport (SFO)</td></tr><tr><td><code>SM</code></td><td>SamTrans</td></tr><tr><td><code>SO</code></td><td>Sonoma County Transit</td></tr><tr><td><code>SR</code></td><td>Santa Rosa CityBus</td></tr><tr><td><code>SS</code></td><td>City of South San Francisco</td></tr><tr><td><code>ST</code></td><td>SolTrans</td></tr><tr><td><code>TD</code></td><td>Tideline Water Taxi</td></tr><tr><td><code>UC</code></td><td>Union City Transit</td></tr><tr><td><code>VC</code></td><td>Vacaville City Coach</td></tr><tr><td><code>VN</code></td><td>VINE Transit</td></tr><tr><td><code>WC</code></td><td>Western Contra Costa (WestCat)</td></tr><tr><td><code>WH</code></td><td>Livermore Amador Valley Transit Authority (LAVTA)</td></tr></tbody></table><p>Fares are included for Clipper Card, cash, and a wide variety of passes. (<a href="https://www.clippercard.com/">Clipper Card</a> is the contact-less payment card and system operated by MTC and available on buses, trains, and ferries throughout the Bay Area.) Transfer discounts between routes operated by the same agency (intra-agency transfers) are included for all methods of payment. Transfer discounts between routes operated by different agencies (inter-agency transfers) are only included for Clipper Card.</p><p>Our team is updating these fares and transfer discounts on a monthly basis.</p><h2 id="using-gtfs-fares-v2-data-for-the-bay-area">Using GTFS-Fares v2 data for the Bay Area</h2><p>Download the daily Regional Feed like so:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li></ol><p>We welcome questions sent to the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources mailing list</a>.</p><h2 id="acknowledgements">Acknowledgements</h2><p>Credit and many thanks to project team members including Ian Rees (Interline), Nome Dickerson (Garnet Consulting), Nisar Kapeel and Kapeel Daryani (MTC), and our partners at Bay Area transit agencies.</p>]]></content:encoded></item><item><title><![CDATA[Observed stop arrivals and departures through the SF Bay Area]]></title><description><![CDATA[Interline and the Metropolitan Transportation Commission are now releasing monthly "stop observation" datasets derived from the Regional GTFS Realtime Feed.]]></description><link>https://www.interline.io/blog/mtc-regional-stop-observations/</link><guid isPermaLink="false">672554e4b62841000136ed97</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Fri, 15 Jul 2022 22:25:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/trip-update-selection.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/trip-update-selection.png" alt="Observed stop arrivals and departures through the SF Bay Area"><p>When did the bus arrived at that stop yesterday? This isn’t a question that transit riders ask often — but it is a question that analysts and planners often do ask.</p><p>Interline and the Metropolitan Transportation Commission have been archiving and processing the <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">Regional GTFS Realtime</a> feed in order to answer these questions.</p><h2 id="what-are-stop-observations">What are stop observations?</h2><p>“Stop observations” describes transit service as was delivered to riders: the arrival time of a vehicle at a designated stop, along with any additional contextual information available from GTFS and GTFS Realtime feeds.</p><p>Stop observations are in contrast with the <code>stop_times.txt</code> file in the <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/">daily Regional Feed</a>, which uses static GTFS data to capture transit service as it is planned and scheduled. This is also in contrast with the <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#historical-feeds">monthly Historic Regional Feed</a>, which uses static GTFS data to capture transit service schedules, retrospectively, as they changed over an entire month (with a different schedule for each agency for each day).</p><p>Stop observations can be used to better understand the real-world realities of transit service as it fluctuates from day to day and location to location. Stop observations can be used to estimate on-time performance (also known as schedule adherence or punctuality) and to inform potential schedule adjustments by service planners. Stop observations can also be used by navigation app developers to provide their users with contextual information, such as an estimated range of times when a route is likely to depart from a given stop.</p><h2 id="measurement-error-and-imprecision">Measurement error and (im)precision</h2><p>Stop observations are inferred from public GTFS Realtime data feeds (trip updates and vehicle positions) and therefore, there are many potential sources for measurement error. Potential sources of error include GPS error on transit vehicles, misassignment of vehicles to “runs” of a given trip and route, imprecision or data errors in the transit agency’s computer-aided dispatch/automatic vehicle location (CAD/AVL) system, issues in the transit agency’s GTFS Realtime emitting system, issues in 511.org’s GTFS Realtime aggregation system, or issues in the <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">Regional Realtime GTFS Feed</a>.</p><p>We’ll discuss how stop observations are calculated later in this blog post, to help users better understand stop observations as measurements. When considering potential applications for stop observations, we ask users to keep the limitations of this data source in mind.</p><h2 id="what-information-is-contained-in-a-stop-observation">What information is contained in a stop observation?</h2><p>We are distributing stop observations in the Historic Regional Feed. When users request a Historic Regional Feed with stop observations for a given year and month, a <code>stop_observations.txt</code> file will be included with all stop observations for that month.</p><p>The <code>stop_observations.txt</code> file is a CSV file based on the <a href="https://bit.ly/gtfs-performance">GTFS-Performance draft proposal</a>. GTFS-Performance is an extension to the GTFS specification prepared by Remix, Swiftly, and their partners.</p><p>The following table is based on the GTFS-Performance draft proposal and covers columns included in our implementation of <code>stops_observations.txt</code>, as well as columns that are in the proposal but not in our implementation:</p><table><thead><tr><th>Field Name</th><th>Details</th><th>Regional GTFS stop observations support</th></tr></thead><tbody><tr><td><code>trip_id</code></td><td><p>Identifies a trip.</p><p> When <code>schedule_relationship=DUPLICATED</code>, then this references the trip that is copied, like GTFS Realtime <code>TripUpdate.trip.trip_id</code>. </p></td><td><p>✅</p></td></tr><tr><td><code>trip_start_time</code></td><td><p> See GTFS Realtime <code>TripDescriptor.start_time</code>. </p><p>Required to specify the start time of a trip that’s not defined in Static GTFS which occurs when using frequency-based service defined by <code>frequencies.txt</code> or when an ad-hoc added service based on an existing trip via <code>TripUpdate.tripschedule_relationship=DUPLICATED</code>.</p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>stop_id</code></td><td><p>Identifies a stop</p></td><td>✅ (added by Interline for developer convenience)</td></tr><tr><td><code>stop_sequence</code></td><td><p>Identifies the order of stops for a particular trip.</p></td><td><p>✅</p></td></tr><tr><td><code>schedule_relationship</code></td><td><p>Valid options are:</p><ul><li> 0 - SCHEDULED </li><li> 1 - SKIPPED </li><li> 2 - NO_DATA </li><li> 3 - UNSCHEDULED </li><li> 4 - CANCELED </li><li> 5 - DUPLICATED </li><li>6 - MODIFIED</li></ul></td><td><p>✅</p></td></tr><tr><td><code>schedule_adherence_stop</code></td><td><p>Indicates if this observation is used for reporting purposes when a subset of stops are utilized, instead of all stops.</p></td><td><p>No</p></td></tr><tr><td><code>trip_start_date</code></td><td><p>The start date of this trip instance in <code>YYYYMMDD</code> format to specify the service date of the corresponding trip. This should match the value in <code>TripDescriptor.start_date</code>.</p></td><td><p>✅</p></td></tr><tr><td><code>vehicle_id</code></td><td><p> Unique identifier for the vehicle that served this stop. This should match the value provided in real-time via <code>VehicleDescriptor.id</code>. </p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>scheduled_arrival_time</code></td><td><p> GTFS static scheduled arrival time </p></td><td><p>✅ (added by Interline for developer convenience)</p></td></tr><tr><td><code>observed_arrival_time</code></td><td><p>Observed arrival time at a specific stop for a specific trip on a route. For times occurring after midnight on the service day, enter the time as a value greater than <code>24:00:00</code> in <code>HH:MM:SS</code> local time for the day on which the trip schedule begins.</p></td><td><p>✅</p></td></tr><tr><td><code>observed_departure_time</code></td><td><p>Observed departure time at a specific stop for a specific trip on a route. For times occurring after midnight on the service day, enter the time as a value greater than <code>24:00:00</code> in <code>HH:MM:SS</code> local time for the day on which the trip schedule begins.</p></td><td><p>✅ (when available)</p></td></tr><tr><td><code>new_departure_time</code></td><td><p>Required when <code>schedule_relationship=DEPARTURE_TIME_MODIFIED</code>.</p></td><td><p>No</p></td></tr><tr><td><code>dwell_time_secs</code></td><td><p>Seconds spend boarding/alighting riders at the stop.</p></td><td><p>No</p></td></tr><tr><td><code>scheduled_dwell_time_secs</code></td><td><p>Seconds scheduled for boarding/alighting riders at the stop (when provided in agency source RT feed).</p></td><td><p>✅</p></td></tr><tr><td><code>boardings</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s board_alight.boardings</p></td><td><p>No</p></td></tr><tr><td><code>alightings</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s board_alight.alightings</p></td><td><p>No</p></td></tr><tr><td><code>source</code></td><td><p>Valid options are:</p><ul><li>0 - Manual</li><li>1 - Door Sensor</li><li>2 - Estimated using GPS/AVL data</li></ul></td><td><p>No</p></td></tr><tr><td><code>uncertainty</code></td><td><p>See <code>StopTimeEvent.uncertainty</code>.</p></td><td><p>✅</p></td></tr><tr><td><code>occupancy_status</code></td><td><p>See <code>VehiclePosition.occupancy_status</code>.</p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>occupancy_percentage</code></td><td><p>See <code>VehiclePosition.occupancy_percentage</code>.</p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>occupancy_type</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s <code>board_alight.load_type</code>.</p></td><td><p>No</p></td></tr><tr><td><code>occupancy_source</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s <code>board_alight.source</code>.</p></td><td><p>No</p></td></tr></tbody></table><h2 id="how-are-stop-observations-calculated">How are stop observations calculated?</h2><p>Stop observations are derived retrospectively using an archive of the Regional GTFS Realtime Feed (updated every 15 seconds). Once a day, our workflow processes GTFS Realtime trip updates and vehicle positions from the archive, combined together with the day’s static GTFS daily regional feed.</p><p><strong>Deriving stop observations from trip updates:</strong> When an agency supplies GTFS Realtime trip updates, our workflow identifies the final <code>TripUpdate</code> sent before a vehicle reaches a given stop. From that record, our workflow extracts the most recent <code>StopTimeUpdate</code>. Here is an example showing three trip updates fetched from BART. The first two trip updates are before the train reaches Montgomery Station; the third trip update is after the train departs Montgomery Station. The portion of all of the records that is use to create a stop observation is marked with the orange box:</p><figure><img src="https://www.interline.io/content/images/2024/11/trip-update-selection-1.png" alt="illustration of three GTFS Realtime feed trip updates, showing the portion of stop time updates retained to create a stop observation record" width="1167" height="943"></figure><p><strong>Deriving stop observations from vehicle positions</strong>: When an agency supplies vehicle positions for a given route, our workflow also identifies the last update sent before the stop location. To do this, we need to consider the geometry of the route alignment from <code>shapes.txt</code> in the static RG feed. The workflow snaps the latitude/longitude point from each vehicle position to the route alignment, enabling it to determine from which update to take the <code>StopTimeUpdate</code> portion. Like many geometric algorithms, this approach can be stymied by complicated paths, such as looping routes that visit the same station multiple times. Our workflow can take advantage of stop sequence, when specified. Still, the algorithm can only do its best to estimate when the route alignment is complicated and/or the update frequency is infrequent.</p><p>By being able to derive stop observations from two different parts of GTFS Realtime feeds, we have also been able to test and tune our workflow, to ensure that the two approaches agree when source feeds are of high quality.</p><h2 id="related-data-specifications">Related data specifications</h2><p>Our stop observations data model is based on the <a href="http://bit.ly/gtfs-performance">GTFS-Performance</a> proposal. The GTFS-Performance proposal is, in turn, inspired by the larger <a href="https://apps.trb.org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=4687">TCRP G-18</a> project. The GTFS-Performance <code>stop_visits.txt</code> is supposed to represent a subset of the TCRP G-18 <code>stop_visits.txt</code> file. Thank you to the creators of GTFS-Performance and to the G-18 research team.</p><p>GTFS-Performance and TCRP G-18 go well beyond the functionality that we have implemented in our system for producing stop observations, to also address more internal operating concerns of transit agencies. We have intentionally implemented our system to create stop observations using only publicly available GTFS and GTFS Realtime inputs.</p><h2 id="accessing-stop-observations">Accessing stop observations</h2><p>Stop observations are now included in historical monthly RGs. To download a historical monthly RG including stop observations, make an HTTP request like so:</p><pre><code>wget https://api.511.org/transit/datafeeds?operator_id=RG&amp;historic=2022-03-so&amp;api_key=xxx
</code></pre><p>Replace <code>2022-03</code> with a given year and month. Note the addition of <code>-so</code> to request the inclusion of stop observations. Note that <code>wget</code> is a Unix command that can be replaced by any other utility to download the URL; also note that <code>xxx</code> should be replaced by your own <a href="https://511.org/open-data/token">511.org Open Data API key</a>.</p><p>Monthly RGs without stop observations are roughly 50Mb in size; with stop observations, the monthly RG will be roughly 500Mb to download as a compressed ZIP archive and then roughly 2Gb of uncompressed CSV files.</p><p>Stop observations are available on a more frequent basis to staff at MTC and Bay Area transit agencies. Transit agencies may contact the 511 SF Bay Data Portal at <a href="mailto:transitdata@511.org">transitdata@511.org</a> for more information.</p><h2 id="using-stop-observations-to-understand-transit">Using stop observations to understand transit</h2><p>Public-transit agencies can be hesitant to release this type of data. In part, it’s a challenge to assemble operations data in an ongoing and consistent manner. In part, it’s a risk that the general public will take operations data out of context.</p><p>While we began this blog post by asking <em>When did the bus arrived at that stop yesterday?</em>, we hope that users of stop observations will ask an even wider range of questions of this dataset, especially questions that can explore the complexity of daily operations for over 20 agencies operating buses, trains, subways, and ferries across the Bay Area and questions that can help to improve the experience for riders using navigation apps and similar software.</p>]]></content:encoded></item><item><title><![CDATA[SF Bay Area's Regional GTFS Realtime Feed]]></title><description><![CDATA[The Metropolitan Transportation Commission and Interline are pleased to announce a single consolidated Regional GTFS Realtime feed. Developers can now get a comprehensive and up-to-date views of the Bay Area's bus, train, and ferry services.]]></description><link>https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/</link><guid isPermaLink="false">6725541ab62841000136ed7f</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Fri, 10 Sep 2021 07:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/list-of-rt-enabled-agencies.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/list-of-rt-enabled-agencies.png" alt="SF Bay Area's Regional GTFS Realtime Feed"><p>Last year, <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/">Interline and the Metropolitan Transportation Commission released the Regional GTFS Feed for the San Francisco Bay Area</a> and also a <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/">series of improvements to the static data including station pathways and fares</a>. We’re now pleased to announce a single consolidated Regional GTFS Realtime feed.</p><p>Developers can combine the static Regional GTFS Feed and the Regional GTFS Realtime endpoints for comprehensive and up-to-date views of the Bay Area’s bus, train, and ferry services. Combining feeds is straightforward, as the same ID schemes are used for data entities across all feeds.</p><h2 id="_21-agencies-included">21 agencies included</h2><p>The Regional GTFS Realtime Feed currently aggregates data for the following Bay Area transit agencies:</p><table><thead><tr><th>Operator Name</th><th>Service Alerts</th><th>Trip Updates</th><th>Vehicle Positions</th></tr></thead><tbody><tr><td>AC Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Bay Area Rapid Transit (BART)</td><td>✅</td><td>✅</td><td>-</td></tr><tr><td>Caltrain</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>County Connection</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Dumbarton Express Consortium</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Emery Go-Round</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Fairfield and Suisun Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Golden Gate Transit (GGT)</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Livermore Amador Valley Transit Authority (LAVA)</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Marin Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Petaluma Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>SamTrans</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>San Francisco Municipal Transportation Agency (SFMTA)</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Santa Rosa CityBus</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>SolTrans</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Sonoma County Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Sonoma Marin Area Rail Transit (SMART)</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Tri Delta Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Union City Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>VINE Transit</td><td>✅</td><td>✅</td><td>✅</td></tr><tr><td>Valley Transportation Authority (VTA)</td><td>✅</td><td>✅</td><td>✅</td></tr></tbody></table><p>MTC will be announcing additional agencies providing GTFS Realtime data in the future.</p><h2 id="built-using-shared-tooling-and-validation-processes">Built using shared tooling and validation processes</h2><p>Like the static Regional GTFS Feed, the Regional GTFS Realtime Feed is build using Interline’s open-source tooling, including the <a href="https://github.com/interline-io/transitland-lib">transitland-lib software library</a>.</p><p>While developing the Regional GTFS Realtime Feed, our teams wanted to ensure that the process of merging together all the individual input agency feeds into the single set of output endpoints did not introduce any additional data errors. We made us of the same GTFS Realtime validation tooling and test process as we recently described in a <a href="https://www.interline.io/blog/trb-transit-idea-gtfs-realtime-report/">Transportation Research Board report by Interline and our partners at University of South Florida</a>.</p><h2 id="how-to-consume-the-regional-gtfs-realtime-feed">How to consume the Regional GTFS Realtime Feed</h2><p>The Regional GTFS Realtime Feed is now available to all developers through the 511 Open Transit Data API. For instructions on how to access the endpoints and to sign up for an API key, see this <a href="https://groups.google.com/g/511sfbaydeveloperresources/c/VlHlYyj4vRA/m/sLdfHXhdAgAJ">post to the 511 SF Bay Developer Resources mailing list</a>.</p>]]></content:encoded></item><item><title><![CDATA[SF Bay Area's Regional GTFS Feed Expanded]]></title><link>https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/</link><guid isPermaLink="false">672547ebb62841000136ed0a</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Mon, 10 Aug 2020 07:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/headway-july-only.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/headway-july-only.png" alt="SF Bay Area's Regional GTFS Feed Expanded"><blockquote><p><strong>October 4, 2022</strong>: MTC and Interline have released updated fares and transfer discounts for all Bay Area transit agencies using the adopted GTFS-Fares v2 &quot;base implementation.&quot; See <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-fares-updates/">this blog post for more information</a>.</p></blockquote><blockquote><p><strong>September 10, 2021</strong>: MTC and Interline have now released the Regional GTFS <em>Realtime</em> Feed, which provides up-to-the-minute service alerts, trip updates, and vehicle positions using the same ID scheme as the static Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">this blog post for an overview</a>.</p></blockquote><p>Earlier this year, <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/">Interline and the Metropolitan Transportation Commission released the Regional GTFS Feed for the San Francisco Bay Area</a>. The Regional GTFS Feed is produced on a daily basis and made available through <a href="https://511.org/open-data">511 SF Bay’s Open Data Portal</a> and its Open Transit Data API.</p><p>We’re pleased to now share a series of additions to the Regional GTFS Feed that we’ve released together over recent months:</p><ul><li><a href="#historical-feeds"><strong>Historical Feeds</strong></a> which provide a retrospective look at an entire month of service</li><li><a href="#station-pathways-and-levels"><strong>Station Pathways and Levels</strong></a> to provide richer wayfinding for all types of riders</li><li><a href="#fares-and-transfer-discounts"><strong>Fares and Transfer Discounts</strong></a> [beta] to calculate the cost of transit journeys on and across the eight largest agencies</li></ul><h2 id="historical-feeds">Historical Feeds</h2><p>Since we released the Regional GTFS Feed in January, much has changed across the Bay Area. Transit agencies and their dedicated staff have rapidly reduced and re-targeted transit service for those riders who perform essential work or depend upon transit for their necessary journeys.</p><p>Using the newly released Historical Feeds component of the Regional GTFS Feed, we can visualize how transit service has changed throughout the Bay Area from January to July of 2020:</p><figure><img src="https://www.interline.io/content/images/2024/11/headway-animation.jpg" alt="" width="2000" height="1582"><figcaption><p>Buses, trains, and ferries throughout the San Francisco Bay Area color-coded by frequency: the reddest route lines are routes that operate at least every 10 minutes; medium hues of red are 10 - 20 minute headways; the lightest hue of red are routes with service less than once every 20 minutes. This animation is created by using the Historical Feeds for the months of January through July of 2020.</p></figcaption></figure><p>Historical Regional Feed products are fully valid GTFS feeds, but they differ somewhat in their contents from the daily Regional Feed products. Read on to understand the process used to produce the Historical Regional Feed products and their key differences, or skip to the end of this section to <a href="#download-historical-feed-products">download Historical Feed products</a>.</p><h3 id="slicing-regional-feeds">Slicing regional feeds</h3><p>Each day, the Regional Feed is produced from the versions of agency feeds on 511.org that provide the best view of service on that day. Each month, the Historical Regional Feed creation process takes these Regional Feeds and combines them together, taking one day of service from each feed, which we are calling a “slice.”</p><p>For example:</p><table><thead><tr><th>Feed filename</th><th>Published</th><th>Contributes service slice for</th></tr></thead><tbody><tr><td>mtc-regional-feed-2020-04-24.zip</td><td>2020-04-24</td><td>2020-04-24</td></tr><tr><td>mtc-regional-feed-2020-04-23.zip</td><td>2020-04-23</td><td>2020-04-23</td></tr><tr><td>mtc-regional-feed-2020-04-22.zip</td><td>2020-04-22</td><td>2020-04-22</td></tr><tr><td>mtc-regional-feed-2020-04-21.zip</td><td>2020-04-21</td><td>2020-04-21</td></tr><tr><td>mtc-regional-feed-2020-04-20.zip</td><td>2020-04-20</td><td>2020-04-20</td></tr></tbody></table><p>If the Regional Feed for a given day is missing, the closest previous day provides service. For instance, if 2020-04-22 was missing, the 2020-04-21 feed slice would cover both 2020-04-21 and 2020-04-22.</p><h3 id="global-entity-copying">Global entity copying</h3><p>Agencies, stops, and routes are considered “global”, and are handled using a simple ID-based merge with the most recent version winning. For example, if BART has a route with ID “OR-S” that is called “Richmond - Warm Springs”, but then later renames it to “Richmond to Warm Springs”, then the latter version will be used.</p><h3 id="trip-hashing-comparison-and-copying">Trip hashing, comparison, and copying</h3><p>Trips are more complicated and handled separately. A simple combining of all the trips and stop_times in all of the input files can easily create a GTFS feed that is too large for practical use, especially given that programs like <a href="https://www.interline.io/routing-platform/">OpenTripPlanner</a> need to hold the entire schedule in memory. Therefore, duplicate copies of trips are detected using a hash based approach and only copied to the output once. This reduces the output size by approximately 90%.</p><p>For example, here are three hypothetical versions of Trip ID “BA:2210503” from three consecutive days of input regional feeds.</p><table><thead><tr><th>Feed filename</th><th>Trip ID</th><th>Route ID</th><th>Service ID</th><th>Headsign</th><th>1st stop</th><th>2nd stop</th><th>n stops</th><th>Hash</th></tr></thead><tbody><tr><td><code>2020-04-24.zip</code></td><td><code>BA:2210503</code></td><td><code>BA:OR-S</code></td><td><code>BA:Wkd_BASE-Weekday-07</code></td><td>Warm Springs/South Fremont</td><td><code>RICH 5:03</code></td><td><code>DELN 5:07</code></td><td><code>PLZA 5:10...</code></td><td><code>8c4ecb</code></td></tr><tr><td><code>2020-04-23.zip</code></td><td><code>BA:2210503</code></td><td><code>BA:OR-S</code></td><td><code>BA:Wkd_BASE-Weekday-07</code></td><td>Warm Springs/South Fremont</td><td><code>RICH 5:03</code></td><td><code>DELN 5:07</code></td><td><code>PLZA 5:10...</code></td><td><code>8c4ecb</code></td></tr><tr><td><code>2020-04-22.zip</code></td><td><code>BA:2210503</code></td><td><code>BA:OR-S</code></td><td><code>BA:Wkd_BASE-Weekday-07</code></td><td>Warm Springs to South Fremont</td><td><code>RICH 5:04</code></td><td><code>DELN 5:06</code></td><td><code>PLZA 5:12...</code></td><td><code>a4bf1a</code></td></tr></tbody></table><p>For each of these, the hashing function takes into account all trip attributes, all the calendar attributes for that trip, and the full details of each entry in <code>stop_times.txt</code>. Any change in any field will result in a different hash. This allows us to directly compare trips between versions of the input feed. Above, all details and schedule for 2020-04-23 and 2020-04-24 match exactly, so these trips two will be considered identical. The trip for 2020-04-22 has some minor differences in name and schedule, so will generate a different hash, and be considered a different trip.</p><p>As the historical feed merging program processes each input feed, it calculates the hash of each trip in the feed. If it has not seen a trip before, it copies it to the output and notes the hash for future use. If it has been seen before, it is not copied again. To prevent clashes, the original Trip IDs are appended with the trip hash (e.g. <code>BA:2210503</code> -&gt; <code>BA:2210503:8c4ecb</code>). The merging program then takes all trips in the input feed (both seen and unseen) and examines the calendars to see which are active for each day in this slice, and then creates <code>calendar_dates.txt</code> entries for each trip on each day where that trip is scheduled to run. The original service IDs are changed to be the same as the hash appended Trip ID, and the calendar is unrolled into a day-by-day format, but it works reliably. This hashing approach is resource efficient and allows us to create historical feeds of arbitrary duration while minimizing the output size.</p><p>Example output <code>calendar_dates.txt</code>:</p><table><thead><tr><th>Service ID</th><th>Date</th><th>Exception Type</th></tr></thead><tbody><tr><td><code>BA:2210503:8c4ecb</code></td><td>2020-04-24</td><td>1 (Added)</td></tr><tr><td><code>BA:2210503:8c4ecb</code></td><td>2020-04-23</td><td>1 (Added)</td></tr><tr><td><code>BA:2210503:a4bf1a</code></td><td>2020-04-22</td><td>1 (Added)</td></tr></tbody></table><p>In this way, the <code>8c4ecb</code> version of the trip is scheduled to run on the two days of input data where it was seen, and the <code>a4bf1a</code> version is scheduled to run on the other day.</p><h3 id="differences-between-regional-and-historic-feeds">Differences between Regional and Historic feeds</h3><p>Historic Regional Feeds are equivalent to the original daily Regional Feeds in the stops, routes, and scheduled services they contain. Using a Historic will produce the same output in a routing engine or another type of analysis.</p><p>Historic Feeds are different from Regional Feeds in their specific GTFS structure:</p><ul><li><code>calendars.txt</code> records are removed and rewritten in <code>calendar_dates.txt</code></li><li><code>trips.txt</code> records are hashed and compared (as described above)</li><li>IDs for global records are namespaced (as described above)</li></ul><p>These differences should not affect routing engine or similar types of analysis. However, keep these differences in mind if you are trying to use historical feeds to understand changes in GTFS data and its practices over time at Bay Area agencies.</p><h3 id="download-historical-feed-products">Download Historical Feed products</h3><p>To use the Historical Feed products:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG&amp;historic=YYYY-MM</code> (for example, to request May 2020: <code>historic=2020-05</code>)</li><li>At the start of each month, the Historic Feed for the last month is created and posted. You may download as many months as you wish and combine them to analyze as many months/quarters/years as you wish at once.</li></ol><p>As of this blog post, the following months are available for download (that is, you can you any of these values for the <code>historic</code> query parameter):</p><ul><li><code>2020-01</code></li><li><code>2020-02</code></li><li><code>2020-03</code></li><li><code>2020-04</code></li><li><code>2020-05</code></li><li><code>2020-06</code></li><li><code>2020-07</code></li></ul><h2 id="station-pathways-and-levels">Station Pathways and Levels</h2><p>The Regional Feed exists to both merge together individual agency GTFS feeds and to serve as a home for new GTFS data that describes cross-agency conditions. The daily Regional Feed now comes with layouts for 35 key transit stations across the Bay Area, each of which serves multiple agencies. Using the newly added <code>pathways.txt</code> and <code>levels.txt</code> files in GTFS, Interline and MTC are now able to provide more detailed information about how to transfer from one agency to another. This includes information on elevators, escalators, and routes that may not be accessible to those in wheelchairs. Pathways and levels information will equip trip planning apps to provide more helpful wayfinding information to their users, particular those who are new to the Bay Area or who have limited vision.</p><figure><img src="https://www.interline.io/content/images/2024/11/university-ave-pathways.jpg" alt="" width="2000" height="948"><figcaption><p>Pathways and levels throughout the University Ave/Downtown Palo Alto station served by Caltrain regional rail on two platforms and VTA and SamTrans buses at bus bays.</p></figcaption></figure><h3 id="_35-regional-transit-hubs">35 Regional Transit Hubs</h3><p>Here is a list of the 35 stations currently in the Regional Feed:</p><table><thead><tr><th>Transit station/hub</th><th>County</th></tr></thead><tbody><tr><td>Embarcadero BART</td><td>San Francisco</td></tr><tr><td>Montgomery BART</td><td>San Francisco</td></tr><tr><td>Caltrain Station 4th &amp; King</td><td>San Francisco</td></tr><tr><td>Salesforce Transit Center</td><td>San Francisco</td></tr><tr><td>12th St Oakland City Center BART</td><td>Alameda</td></tr><tr><td>El Cerrito Del Norte BART</td><td>Contra Costa</td></tr><tr><td>19TH St Oakland BART</td><td>Alameda</td></tr><tr><td>Powell ST BART</td><td>San Francisco</td></tr><tr><td>Civic Center BART</td><td>San Francisco</td></tr><tr><td>Walnut Creek BART</td><td>Contra Costa</td></tr><tr><td>Richmond BART/Amtrak</td><td>Contra Costa</td></tr><tr><td>San Jose Diridon Station</td><td>Santa Clara</td></tr><tr><td>Palo Alto Station</td><td>Santa Clara</td></tr><tr><td>San Rafael Transit Center</td><td>Marin</td></tr><tr><td>Millbrae BART</td><td>San Mateo</td></tr><tr><td>Pleasant Hill BART</td><td>Contra Costa</td></tr><tr><td>San Francisco Ferry Terminal</td><td>San Francisco</td></tr><tr><td>Daly City BART</td><td>San Francisco</td></tr><tr><td>Santa Rosa Transit Mall</td><td>Sonoma</td></tr><tr><td>Union City BART</td><td>Alameda</td></tr><tr><td>MacArthur BART</td><td>Alameda</td></tr><tr><td>Dublin/ Pleasanton BART</td><td>Alameda</td></tr><tr><td>Warm Springs/ South Fremont BART</td><td>Alameda</td></tr><tr><td>Santa Clara Caltrain</td><td>Santa Clara</td></tr><tr><td>Oakland Coliseum BART</td><td>Alameda</td></tr><tr><td>SFO</td><td>San Francisco</td></tr><tr><td>Fairfield Transportation Center</td><td>Solano</td></tr><tr><td>OAK</td><td>Alameda</td></tr><tr><td>Petaluma Transit Mall</td><td>Marin</td></tr><tr><td>Vallejo Ferry Terminal</td><td>Solano</td></tr><tr><td>Mountain View Station</td><td>Santa Clara</td></tr><tr><td>Great America</td><td>Santa Clara</td></tr><tr><td>Napa Intermodal</td><td>Napa</td></tr><tr><td>SJC</td><td>Santa Clara</td></tr><tr><td>Great Mall/Milpitas BART</td><td>Santa Clara</td></tr></tbody></table><p>Note that just as the ongoing pandemic has made our work on the Historic Feed all the more relevant, our work on Station Pathways and Levels has had to adapt to current circumstances. Our staff have been working from home using aerial imagery, architectural drawings, station maps, and other materials. Interline’s Station Editor tool also works on tablets, and we look forward to again going out into the field to all of these stations to correct and improve details.</p><p>We welcome questions and corrections sent to the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources mailing list</a>.</p><h3 id="using-station-pathways-and-levels">Using Station Pathways and Levels</h3><p>Download the daily Regional Feed like so:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li><li>Look for the <code>pathways.txt</code> and <code>levels.txt</code> files inside the zip archive.</li><li>For more information, see the <a href="https://gtfs.org/reference/static#pathwaystxt">static GTFS documentation</a> and the <a href="https://bit.ly/gtfs-pathways">GTFS-Pathways extension proposal document</a>.</li></ol><h2 id="fares-and-transfer-discounts">Fares and Transfer Discounts</h2><p>To date, the Regional GTFS Feed has focused on how to plan a journey by transit. Now we’re curating and adding additional data to help riders understand how to pay for their journeys. The Bay Area’s Regional GTFS Feed is now the first in the world to include data using the GTFS Fares-v2 specification. This <a href="https://bit.ly/gtfs-fares">newly proposed specification</a> allows us to model the many different fare products and discounts that are available to the Bay Area’s transit riders. For example, we can now account for how some agencies provide riders a discount when they pay by Clipper Card rather than by cash. (<a href="https://www.clippercard.com/">Clipper Card</a> is the contact-less payment card and system operated by MTC and available on buses, trains, and ferries throughout the Bay Area.) We can also capture how riders can receive a discount when transferring from certain agencies to other agencies. Read on for a detailed example of how Fares-v2 work in practice, or skip ahead to <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#using-fares-and-transfer-discounts">use the Fares-v2 beta data</a>.</p><h3 id="an-example-of-fares-v2-in-practice">An Example of Fares-v2 in Practice</h3><figure><img src="https://www.interline.io/content/images/2024/11/bart-to-ac-transit-route.png" alt="map of a journey by BART and AC Transit originating in San Francisco, transferring in Oakland, and ending in Alameda" width="1606" height="1042"><figcaption><p>A journey by BART and AC Transit, from San Francisco to Oakland to Alameda, which we will use in the following example of how to calculate the total cost. (Credit: BART Trip Planner)</p></figcaption></figure><p>Fares-v2 expands the traditional route- and zone-based GTFS fares model with several additional files, each focused on modeling a different part of a complex fare scheme. Imagine a rider making a transfer from BART to AC Transit; both BART and AC Transit have different prices for adult fares, eligible discount fares, and different prices when paying with cash or when using a Clipper Card; additionally, the discount applied when transferring from BART to AC Transit has different values and rules when using Clipper. Calculating the individual fares for each leg of this trip requires a description of any zones that apply to routes and stops (<code>fare_networks.txt</code> and <code>fare_areas.txt</code>), the costs and requirements of the base fare for each leg (<code>fare_leg_rules.txt</code>), any discount categories that might include the rider (<code>fare_profiles.txt</code>), and the type of payment used by the rider (<code>fare_containers.txt</code>). All of these factors are considered and then used to search for any transfer discounts that may apply for the trip (<code>fare_transfer_rules.txt</code>).</p><p>Let’s work through the hypothetical trip above, first taking BART from Embarcadero Station to 12th St. in Oakland, and then riding an AC Local bus, 51A, to Alameda.</p><p>The following is an except from <code>fare_leg_rules.txt</code> with some columns removed for brevity. This file contains sets of rules that are matched against a single leg of a trip, without regard to transfers. The columns <code>from_area_id</code>, <code>to_area_id</code>, <code>fare_container_id</code> and <code>fare_category_id</code> references values defined in the other files mentioned above.</p><table><thead><tr><th>order</th><th>leg_group_id</th><th>from_area_id</th><th>to_area_id</th><th>amount</th><th>fare_container_id</th><th>fare_category_id</th><th>notes</th></tr></thead><tbody><tr><td>90</td><td>BA:</td><td>EMBR</td><td>12TH</td><td>1.35</td><td>clipper</td><td>BA:Senior/Disabled Clipper</td><td>Embarcadero to 12th with Clipper and discount</td></tr><tr><td>100</td><td>BA:</td><td>EMBR</td><td>12TH</td><td>3.70</td><td>clipper</td><td></td><td>Embarcadero to 12th with Clipper</td></tr><tr><td>110</td><td>BA:</td><td>EMBR</td><td>12TH</td><td>4.20</td><td></td><td></td><td>Embarcadero to 12th with cash surcharge</td></tr></tbody></table><p>The BART leg would match the first the rules above based on <code>from_area_id</code> and <code>to_area_id</code>, with fares of $1.35, $3.70, and $4.20. However, the first two rules are only available when using a Clipper Card (<code>fare_container_id</code>), and the first of these is only available to riders eligible for a discounted fare (<code>fare_category_id</code>). The <code>order</code> field is used to select the correct fare when multiple rules match the trip leg: any applicable discounted Clipper fare would match first, followed by adult Clipper fare, then finally the fare including additional charge for cash riders. In the event two (or more) rules match with the same order value, both are considered valid options to present to the rider.</p><table><thead><tr><th>order</th><th>leg_group_id</th><th>from_area_id</th><th>to_area_id</th><th>amount</th><th>fare_container_id</th><th>fare_product_id</th><th>fare_category_id</th><th>notes</th></tr></thead><tbody><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>2.25</td><td>clipper</td><td></td><td></td><td>Adult local fare with clipper $2.25</td></tr><tr><td>110</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>2.5</td><td></td><td></td><td></td><td>Adult local fare $2.5</td></tr><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>1.12</td><td></td><td></td><td>AC:senior</td><td>Discounted local fare $1.12 with Clipper</td></tr><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>1.25</td><td></td><td></td><td>AC:senior</td><td>Discounted local fare $1.25</td></tr><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>0</td><td></td><td>AC:local:monthly</td><td></td><td>Free local fare with monthly pass</td></tr></tbody></table><p>The AC Transit fare rules are similar, with the addition of a few extra columns. The 51A is an East Bay local only route (<code>to_area_id</code> is assumed), and matches each rule, and the rules for discounts and cash are applied as above. The <code>fare_product_id</code> field describes additional rules that are available to users of certain prepaid products, such as day passes and monthly passes. In this case, a rider with an Adult Local 31 Day Pass ($84, described in <code>fare_products.txt</code>) would enjoy a free ride. More complex AC Transit fares use additional columns not pictured here (<code>fare_network_id</code>) and handle cases such as riding a Transbay bus and the different prices applied when riding within the East Bay vs. taking the trip all the way to the Salesforce Transit Center in downtown San Francisco.</p><p>Any applicable transfers are calculated by reading the rules in <code>fare_transfer_rules.txt</code> and matching each leg of the trip against each subsequent leg of the trip. This can create quite complicated models, but let’s start by looking at our BART and AC Transit legs. The clever bit is that many <code>fare_leg_rules.txt</code> rules can share the same <code>leg_group_id</code>, which simplifies the number and types of transfer rules that must be defined. For example, all of the BART rules above use <code>BA:</code> and all of the AC Transit rules use <code>AC:local</code>. The table below is an except of <code>fare_transfer_rules.txt</code>.</p><table><thead><tr><th>order</th><th>from_leg_group_id</th><th>to_leg_group_id</th><th>fare_container_id</th><th>amount</th><th>duration_limit</th><th>duration_limit_type</th><th>fare_transfer_type</th><th>spanning_limit</th><th>notes</th></tr></thead><tbody><tr><td>100</td><td>BA:</td><td>AC:local</td><td>clipper</td><td>-0.5</td><td>90</td><td>2</td><td>1</td><td>2</td><td>1 credit of $-0.50 to AC w/in 90 minutes</td></tr><tr><td>110</td><td>BA:</td><td>AC:local</td><td></td><td>-0.25</td><td></td><td></td><td>1</td><td>3</td><td>Cash: 2 fare credits of $0.25 each</td></tr></tbody></table><p>Two different discounts apply when taking an AC Transit leg after a BART leg. When using Clipper, a $0.50 (<code>amount=-0.5</code>, <code>fare_transfer_type=1</code>) discount is applied to the first AC Transit leg (<code>spanning_limit=2</code>, which means it can only match a subjourney of two legs). Alternatively, a paper transfer slip given to cash users contains two tabs, each of which applies a $0.25 discount to the AC Transit cash fare (<code>spanning_limit=3</code>, or a subjourney of up to three legs). Additionally, the Clipper discount expires within 90 minutes after tagging off BART (<code>duration_limit=90</code>, <code>duration_limit_type=2</code>). Because the <code>order</code> value is different, these two transfer options are mutually exclusive to Clipper and cash riders respectively.</p><p>The combination of matching fare rules and fare transfer rules allows describing even complicated transfers, such as discounts to SamTrans local bus riders who hold a Caltrain 2 zone or higher monthly pass, the several categories of upgrade charges when transfering from an AC Transit Local to an AC Transit Transbay bus, and the many varied products and transfers that apply full or partial fare to trips on the Dumbarton Express.</p><h3 id="agencies-in-the-beta-release">Agencies in the Beta Release</h3><p>Note that this is a “beta” release. The GTFS Fares-v2 specification is still being finalized. It does not fully capture some functionality, like how AC Transit and VTA use “fare capping” so that riders never have to pay more in one day than the cost of a day pass. We also expect that as trip planners and other apps begin to consume the Regional Feed’s fare data that we may need to revise some of the data or our schema.</p><p>To our knowledge, Interline is the only organization to have a “rules engine” that can calculate journey costs using GTFS Fares-v2 data. We expect that as other organizations adopt the Interline fares rules engine or build their own equivalents, we may need to revisit some of the assumptions built into the engine.</p><p>The beta release of data includes fares and transfer discounts within and between eight agencies:</p><ul><li>BART</li><li>SFMTA</li><li>AC Transit</li><li>SamTrans</li><li>VTA</li><li>Caltrain</li><li>Golden Gate Transit (bus only; not ferries)</li><li>Dumbarton Express</li></ul><p>This list includes the seven largest agencies and a key connector across San Francisco Bay (Dumbarton Express). It should provide a broad enough sample of fares and transfer discounts to power a wide range of potential applications. This sample is also deep enough to inform analyses of fares, the cost of riding transit, and ideally even ways to reform fares and transfer discounts to produce more equitable outcomes for Bay Area transit riders.</p><h3 id="using-fares-and-transfer-discounts">Using Fares and Transfer Discounts</h3><p>Download the daily Regional Feed like so:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li><li>For more information about the fares and transfer discount files in the GTFS feed, see the <a href="https://bit.ly/gtfs-fares">GTFS-Fares v2 proposal document</a>.</li></ol><p>Interline and MTC expect to improve both the Fares-v2 beta data and schemas as we learn more. We welcome questions and corrections sent to the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources mailing list</a>.</p><h2 id="public-transit-in-2020">Public Transit in 2020</h2><p>A final note that behind every piece of transit data and every multi-modal trip plan are real people who drive vehicles, serve riders, and maintain equipment and facilities. Thank you to all the front-line workers keeping the Bay Area’s public transit systems and riders moving safely!</p><h2 id="acknowledgements">Acknowledgements</h2><p>Credit and many thanks to project team members including Ian Rees and Ruth Miller (Interline), Nisar Kapeel and Kapeel Daryani (MTC), and our partners at Bay Area transit agencies.</p>]]></content:encoded></item><item><title><![CDATA[A Regional GTFS Feed for the San Francisco Bay Area]]></title><description><![CDATA[Last year, Interline began a "startup in residence" at the Metropolitan Transportation Commission. Now we're pleased to publicly release our first creation: the San Francisco Bay Area's Regional GTFS Feed.]]></description><link>https://www.interline.io/blog/mtc-regional-gtfs-feed-release/</link><guid isPermaLink="false">6725531fb62841000136ed64</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Wed, 08 Jan 2020 08:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/mtc-regional-feed-all-routes.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/mtc-regional-feed-all-routes.png" alt="A Regional GTFS Feed for the San Francisco Bay Area"><blockquote><p><strong>September 10, 2021</strong>: MTC and Interline have now released the Regional GTFS <em>Realtime</em> Feed, which provides up-to-the-minute service alerts, trip updates, and vehicle positions using the same ID scheme as the static Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">this blog post for an overview</a>.</p></blockquote><blockquote><p><strong>August 10, 2020</strong>: MTC and Interline have released additional functionality in the Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/">this more recent blog post</a>.</p></blockquote><p>Last year, <a href="https://www.interline.io/blog/metropolitan-transportation-commission-selects-interline/">Interline began a “startup in residence” at the Metropolitan Transportation Commission</a>. Now we’re pleased to publicly release our first creation together: the San Francisco Bay Area’s Regional GTFS Feed.</p><p>Every day, Interline’s workflow processes 31 agency feeds and creates a single unified static GTFS feed.</p><figure><img src="https://www.interline.io/content/images/2024/11/mtc-regional-feed-animation.jpg" alt="" width="1530" height="1990"><figcaption><p>Transit routes around the Bay Area, first by agency feed, then from the aggregated Regional GTFS Feed.</p></figcaption></figure><h2 id="why-a-regional-feed">Why a Regional Feed?</h2><p>People don’t stop their commutes or their travels at transit agency boundaries. Similarly, many applications require combining multiple agency feeds: trip planners, visualizations, data analyses, travel demand models, etc. <strong><em>The Regional GTFS Feed will provide data consumers an easier option.</em></strong> Less effort to ingest multiple feeds, more time toward your actual goals.</p><p><strong><em>The Regional GTFS Feed enables the addition of cross-agency data</em></strong>: fares, transfer discounts, other transfer information, and transit hub layouts and accessibility information. These types of data reference IDs and entities from multiple agencies and are a natural fit within a consolidated feed.</p><p>The workflow that creates the Regional GTFS Feed also identifies conflicts between agency feeds, data entities, and IDs. <strong><em>By validating agency feeds together, the workflow identifies data quality issues that are only apparent once individual feeds are combined.</em></strong> We’re sharing this information with MTC and agency staff to help further improve all feeds.</p><p>Data consumers are still able to access individual feeds from agency websites and <a href="https://511.org/developers/list/apis/">511.org developer APIs</a>. The Regional GTFS Feed is produced using a “conservative” approach: data transformation rules are targeted and constrained. Our motto is <em>Merge. Don’t lose.</em></p><h2 id="future-plans">Future Plans</h2><p>In 2020, we’re continuing to work with MTC to further enrich the Regional GTFS Feed:</p><ul><li>We’re <strong><em>mapping key transit hubs</em></strong> around the Bay Area, creating GTFS-Pathways data using Interline’s new Gotransit editing tools.<a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/#footnote-1">*</a> This will help trip planning websites and apps to provide better directions to those who use wheelchairs or have other mobility limitations.</li><li>We’re <strong><em>cataloging fares, tickets, and transfer discounts around the Bay Area</em></strong>, creating GTFS-Fares v2 data, also using Interline’s Gotransit editing tools.<a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/#footnote-2">**</a> Transit fares and discounts across agencies is a problem of combinatorial complexity. The Regional GTFS Feed will not be able to precisely capture every possible fare, pass, and transfer discount. The goal is to capture sufficient detail to create realistic fare estimates for a variety of rider types riding and transferring between the agencies of highest ridership.</li><li>To start, the Regional GTFS Feed is updated once per day. In the future, we’ll also <strong>create historical versions of the Regional GTFS Feed</strong> that combine “slices in time” from different agency feed versions. While trip planning is best powered by the latest GTFS feed, many types of analyses and travel models instead require archival feed versions from the past.</li></ul><p>Please watch 511.org’s <a href="https://511.org/open-data/transit">Open Transit Data</a> section for updates as these additions are released.</p><h2 id="powered-by-transitland">Powered by Transitland</h2><p>The Regional GTFS Feed has much in common with <a href="https://www.transit.land/">Transitland</a>, our open data platform that aggregates GTFS data for over 2,500 agencies around the world. We’ve used many of the lessons we’ve learned from Transitland to create the Regional GTFS Feed workflow. In fact, the workflow is build using the core, open-source ingredients of <a href="https://www.transit.land/news/2019/10/17/tlv2.html">Transitland version 2</a> (also known as “Tlv2”) The 30+ agency feeds are represented as a DMFR (a Distributed Mobility Feed Registry) and processed using the Gotransit library. The Bay Area’s Regional GTFS Feed workflow is, in effect, a Transitland of its own, which is exposed using a static GTFS export.</p><h2 id="to-use-the-feed-and-ask-questions">To Use the Feed and Ask Questions</h2><p>To use the new Regional GTFS Feed:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li><li>The feed is updated daily; download it as frequently as your application needs updates.</li></ol><p>To provide feedback or ask questions, please visit the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources</a> Google group.</p><h2 id="acknowledgements">Acknowledgements</h2><figure><img src="https://www.interline.io/content/images/2024/11/mtc-logo.png" alt="Metropolitan Transportation Commission logo" width="516" height="128"></figure><p>Thanks to our partners in MTC’s Technology Services division. They’ve been serving transit data through 511.org since before GTFS and GTFS Realtime specifications. It’s a pleasure learning from their experience and working together to take advantage of the latest advances in transit data.</p><figure><img src="https://www.interline.io/content/images/2024/11/stir-banner-1.png" alt="Startup in Residence logo" width="231" height="128"></figure><p>Thanks to also to <a href="https://www.cityinnovate.com/">City Innovate</a> for managing the Startup in Residence (STIR) program. We encourage procurement officers and civic-oriented startups to learn more about the program.</p><h2 id="notes">Notes</h2><p><a id="footnote-1"></a></p><p><sup>*</sup> For more on the GTFS-Pathways additions, see <a href="http://bit.ly/gtfs-pathways">this Google doc</a>.</p><p><a id="footnote-2"></a></p><p><sup>**</sup> For more on the GTFS-Fares-v2 proposal, see <a href="http://bit.ly/gtfs-fares">this Google doc</a>.</p>]]></content:encoded></item><item><title><![CDATA[Metropolitan Transportation Commission and Interline to tackle regional transit data challenges]]></title><description><![CDATA[We're excited to share that the Metropolitan Transportation Commission has selected Interline through a competitive process to participate in its 2019 Startup in Residence (STIR) program. Interline and MTC will collaborate on the problem of regional transit data.]]></description><link>https://www.interline.io/blog/metropolitan-transportation-commission-selects-interline/</link><guid isPermaLink="false">67254697b62841000136ece8</guid><dc:creator><![CDATA[Interline Technologies]]></dc:creator><pubDate>Thu, 10 Jan 2019 08:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/stir-banner.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/stir-banner.png" alt="Metropolitan Transportation Commission and Interline to tackle regional transit data challenges"><blockquote><p><strong>January 8, 2020</strong>: MTC and Interline have released the Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/">this more recent blog post</a>.</p></blockquote><blockquote><p><strong>August 10, 2020</strong>: MTC and Interline have released additional functionality in the Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/">this more recent blog post</a>.</p></blockquote><p>We’re excited to share that the <a href="https://mtc.ca.gov/whats-happening/news/mtc-work-startups-tackle-transportation-challenges">Metropolitan Transportation Commission has selected Interline</a> through a competitive process to participate in its 2019 Startup in Residence (STIR) program.</p><h2 id="regional-transit-data-for-the-bay-area">Regional Transit Data for the Bay Area</h2><p>Interline and MTC will collaborate on the problem of regional transit data. The Bay Area has over 30 public-transit operators. Most agencies provide open-data in the <a href="http://gtfs.org/">GTFS format</a>, which is in turn used by many companies and developers to power navigation and transportation apps. However, to use all of these GTFS feeds together presents many technical challenges to both public agencies and private consumers. During the STIR residency, Interline will use the <a href="https://www.transit.land/">Transitland</a> platform to help the MTC improve regional transit data distribution.</p><p>Transitland currently aggregates GTFS feeds from approximately 2,500 transit operators around the world. Transitland users — including startups, large corporations, advocacy groups, and government agencies — currently query Transitland’s open APIs five million times per month. Transitland is powered by open-source software, and can be customized by Interline for particular agency needs.</p><p>Transitland currently aggregates transit data for the San Francisco Bay Area from 36 operators:</p><figure><img src="https://www.interline.io/content/images/2024/11/feed-registry-bay-area.png" alt="Transitland Feed Registry listing 36 operators in the San Francisco Bay Area" width="1197" height="3857"></figure><p>For a current list of Bay Area transit agencies that provide open GTFS data, see the <a href="https://www.transit.land/feed-registry/?metro=San+Francisco+Bay+Area">Transitland Feed Registry</a>.</p><h2 id="startup-in-residence-program">Startup in Residence Program</h2><figure><img src="https://www.interline.io/content/images/2024/11/stir-banner-2.png" alt="Start up in Residence program logo" width="231" height="128"></figure><p>The national STIR program is supported by the <a href="https://www.cityinnovate.com/">City Innovate Foundation</a>.</p><p>“The Startup in Residence program is a model for civic innovation and national collaboration,” says Jay Nath, former Chief Innovation Officer for San Francisco and Executive Director for City Innovate. “This program is a unique opportunity for government agencies and startups to think creatively about how we can all work together to modernize government to benefit residents.”</p><p>For more on STIR, see <a href="https://medium.com/@CityInnovate/700-startups-compete-to-work-on-urban-challenges-through-startup-in-residence-55052fd781ea">City Innovate’s announcement of the 2019 program</a>.</p><h2 id="metropolitan-transportation-commission">Metropolitan Transportation Commission</h2><figure><img src="https://www.interline.io/content/images/2024/11/mtc-logo-1.png" alt="Metropolitan Transportation Commission (MTC) logo" width="516" height="128"></figure><p>The <a href="https://mtc.ca.gov/">Metropolitan Transportation Commission</a> (MTC) is the transportation planning, financing, and coordinating agency for the nine-county San Francisco Bay Area. MTC is based out of downtown San Francisco.</p><p>MTC provides <a href="https://511.org/">511.org</a> with transportation information for travelers around the Bay Area and <a href="https://511.org/developers/list/resources/">511 Developer Resources</a> to power third-party apps and websites.</p><h2 id="transitland-and-interline">Transitland and Interline</h2><figure><img src="https://www.interline.io/content/images/2024/11/transitland_logo-1.png" alt="Transitland logo" width="620" height="290"></figure><p>Transitland is powered by open-source software and open data. Started in 2014 by Mapzen, <a href="https://www.transit.land/news/2019/01/08/transitland-continues.html">Transitland is now supported by Interline and a growing list of partners</a>.</p><p>To follow Interline’s progress on this project, follow this blog or <a href="http://eepurl.com/dmWHln">subscribe to our newsletter</a>.</p><p>Interline’s team of principals and specialists all live in the Bay Area and ride public transit regularly. We’re pleased to have this opportunity to collaborate with the MTC on improvements to our region and its transportation systems!</p>]]></content:encoded></item></channel></rss>