<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" version="2.0"><channel><title><![CDATA[GTFS — Interline Technologies]]></title><description><![CDATA[GTFS is the core data specification for public transit stops, schedules, routes, and related information for riders. The Interline team has been involved in GTFS's development and governance for a decade.]]></description><link>https://www.interline.io/</link><image><url>https://www.interline.io/favicon.ico</url><title>GTFS — Interline Technologies</title><link>https://www.interline.io/</link></image><lastBuildDate>Thu, 05 Mar 2026 04:48:03 GMT</lastBuildDate><atom:link href="https://www.interline.io/blog/tag/gtfs/rss.xml" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title><![CDATA[Explore GTFS-Pathways station layouts using Transitland]]></title><description><![CDATA[Visualize subway station topologies. Inspect wheelchair-accessible navigation. Transitland's free GTFS-Pathways viewer is powered by Interline Station Editor.]]></description><link>https://www.interline.io/blog/explore-gtfs-pathways-station-layouts-using-transitland/</link><guid isPermaLink="false">69a8b8bb24682d0001f25803</guid><dc:creator><![CDATA[Interline Technologies]]></dc:creator><pubDate>Thu, 05 Mar 2026 04:48:03 GMT</pubDate><media:content url="https://www.interline.io/content/images/unsplash/photo-1617727553230-35c143f84010.jpg" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/unsplash/photo-1617727553230-35c143f84010.jpg" alt="Explore GTFS-Pathways station layouts using Transitland"><p><a href="https://www.interline.io/saas-tools/station-editor/">Interline Station Editor</a> has helped transit agencies and consultants build GTFS-Pathways data for major transit hubs across North America. Now we're also bringing the core of that functionality to Transitland, so anyone can explore and inspect pathways data, wherever it exists and however it was originally created.</p><blockquote><p>GTFS-Pathways data is useful to provide wheelchair-accessible navigation and to enrich wayfinding instructions for riders using smartphone apps.</p><p>The GTFS-Pathways extension adds <code>pathways.txt</code> and <code>levels.txt</code> files, as well as stops with <code>location_type={3,4}</code> so GTFS can represent the physical layout of a transit station.</p><p>In contrast with other indoor mapping data formats, GTFS-Pathways represents a topological &quot;skeleton&quot; of how riders can travel through a multi-level subway, train, or bus station.</p></blockquote><p>Find a pathways-enabled station in Transitland and open the Transitland Pathways Explorer to use the interactive Map View tab to view the structure of multi-level stations. For example, <a href="https://www.transit.land/stops/s-dqcjr1jyg8-metrocenterredlinetrack1platform/pathways">WMATA's Metro Center</a> underground subway station:</p><figure><img src="https://www.interline.io/content/images/2026/03/CleanShot-2026-03-04-at-15.04.22.jpg" alt="" width="1200" height="1089"></figure><p>Use the Network Diagram tab to understand the topological organization of a station's levels and nodes. For example, the San Francisco Bay Area's <a href="https://www.transit.land/stops/s-9q8vzhbs2k-millbraebart/pathways">Millbrae Intermodal Station</a>, which is served by BART and Caltrain and bus agencies:</p><figure><img src="https://www.interline.io/content/images/2026/03/CleanShot-2026-03-04-at-20.23.16@2x.png" alt="" width="2000" height="1439"></figure><p>Use the Route Finder tab to test a journey through a station. For example, <a href="https://www.transit.land/stops/s-drt2yyxdb8-downtowncrossing/pathways">MBTA's Downtown Crossing</a> underground subway station:</p><figure><img src="https://www.interline.io/content/images/2026/03/image-2.png" alt="using Transitland to find a route through MBTA's Downtown Crossing subway station in Boston" width="1490" height="1814"></figure><p>The Validation tab provides a high-level summary of potential issues to debug and improve within a station's GTFS-Pathways data.</p><blockquote><p>The Transitland Pathways Explorer is great for understanding what's publicly available. When you need to build it, fix it, or validate it to detailed production standards, that's what <a href="https://www.interline.io/saas-tools/station-editor/">Interline Station Editor</a> is for.</p></blockquote><p>To find pathways-enabled stations, use the Transitland global transit map and look for the notice that &quot;<em>Station pathway points are in view</em>&quot;</p><figure><img src="https://www.interline.io/content/images/2026/03/CleanShot-2026-03-04-at-15.12.40@2x.png" alt="" width="2000" height="1463"></figure><p>Click on (or near) a relevant stop point to view the stop details, and to then open the Transitland Pathways Explorer:</p><figure><img src="https://www.interline.io/content/images/2026/03/CleanShot-2026-03-04-at-16.00.08@2x.png" alt="" width="1828" height="1084"></figure><p>Transitland Pathways Explorer is available to all registered Transitland users, including those on the free plan.</p><p>As more transit operators publish GTFS-Pathways data in their feeds, we look forward to this functionality being even more widely useful.</p>]]></content:encoded></item><item><title><![CDATA[Same data, different zip: How to identify unique GTFS feed versions]]></title><description><![CDATA[Standard zip hashes are unreliable for GTFS versioning due to variable metadata. Learn to use content-aware checksums to accurately identify unique transit datasets.]]></description><link>https://www.interline.io/blog/gtfs-checksum-versioning/</link><guid isPermaLink="false">698cc3c44b4f20000160b769</guid><dc:creator><![CDATA[Interline Technologies]]></dc:creator><pubDate>Wed, 11 Feb 2026 22:45:59 GMT</pubDate><media:content url="https://www.interline.io/content/images/unsplash/photo-1531376653594-e9bcf0f0c65b.jpg" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/unsplash/photo-1531376653594-e9bcf0f0c65b.jpg" alt="photograph of overlapping zippers (as a metaphor for different zip archives)"><p>If you've ever downloaded a GTFS feed twice from the same URL and gotten two different zip files, you've encountered a surprisingly common problem in the transit data ecosystem. The data inside hasn't changed (the same stops, routes, and schedules) but the zip archive itself is different. Maybe the server regenerated it dynamically. Maybe the compression level changed. Maybe the file timestamps inside the archive shifted by a second.</p><p>For Transitland, which archives and imports <a href="https://www.transit.land/feeds">thousands of GTFS feeds</a> from around the world, this matters. We need to know: <em>has this feed actually changed, or is it just the same data in a new wrapper?</em></p><p>The answer lies in how we calculate checksums — and you can use the same tools we do.</p><h2 id="the-problem-with-hashing-zip-files">The problem with hashing zip files</h2><p>The naive approach to detecting changes is straightforward: compute a <a href="https://en.wikipedia.org/wiki/SHA-1">SHA1 hash</a> of the zip file and compare it to what you had before. If the hashes differ, something changed.</p><p>But zip archives are containers, and containers have metadata. A zip file's checksum can change for reasons that have nothing to do with the transit data inside:</p><ul><li><strong>Dynamic generation</strong>: Some transit agencies serve GTFS through vendor platforms that create the zip on the fly for each download. Same data, different zip bytes every time.</li><li><strong>Compression differences</strong>: Repackaging a zip with a different compression level produces a different file.</li><li><strong>Timestamps and ordering</strong>: The internal file modification times and the order of entries within the archive can vary between builds.</li></ul><p>If Transitland treated every new zip checksum as a new feed version, we'd be archiving and importing duplicate data constantly, wasting storage, processing time, and cluttering the archive with phantom &quot;updates.&quot;</p><h2 id="two-checksums-two-purposes">Two checksums, two purposes</h2><p>Transitland calculates two SHA1 checksums for every <a href="https://www.transit.land/documentation/concepts/static-gtfs-feed-versions">feed version</a>:</p><p><strong>Zip SHA1</strong>: a conventional hash of the entire zip archive file. This is fragile by nature (it changes whenever the packaging changes), but it's useful as a unique identifier for the exact file you downloaded. Transitland uses this as the primary key for feed versions in its <a href="https://www.transit.land/documentation/rest-api/feed_versions">REST API</a> and website URLs.</p><p><strong>Directory SHA1</strong>: a content-aware hash that looks <em>inside</em> the zip, directly at the GTFS CSV files. This is the one that solves the deduplication problem. Two zip files with completely different Zip SHA1 values will share the same Directory SHA1 if the actual transit data is identical.</p><p>When Transitland's <a href="https://www.transit.land/documentation/concepts/source-feeds">feed fetching service</a> downloads a new copy of a feed, it checks both hashes against its archive. If either one matches an existing feed version, it knows the data isn't new and skips the import.</p><h2 id="inside-the-directory-sha1">Inside the Directory SHA1</h2><p>The Directory SHA1 algorithm in <a href="https://www.transit.land/documentation/transitland-lib">transitland-lib</a> is deliberately simple, but each design choice is intentional:</p><ol><li><strong>Open the zip and enumerate its entries.</strong> We look at what's inside the archive, not the archive itself.</li><li><strong>Filter to GTFS content files.</strong> Only lowercase <code>.txt</code> files in the root directory of the archive are included — <code>stops.txt</code>, <code>routes.txt</code>, <code>trips.txt</code>, and so on. Directories, hidden files (starting with <code>.</code>), files in subdirectories, and non-<code>.txt</code> files are all excluded.</li><li><strong>Sort the files alphabetically by name.</strong> This neutralizes any variation in the order files appear within the zip.</li><li><strong>Stream each file's bytes sequentially into a single SHA1 hash.</strong> The contents of <code>agency.txt</code>, then <code>calendar.txt</code>, then <code>calendar_dates.txt</code>, and so on, all fed into one running hash computation.</li></ol><p>The result is a 40-character hex string that represents the actual transit data, independent of how it was packaged. Re-zip the same GTFS files with different compression? Same Directory SHA1. Download the feed again tomorrow from a server that regenerates the zip? Same Directory SHA1, as long as the CSV contents haven't changed.</p><p>Notably, the algorithm does <em>not</em> reorder rows within files or normalize field ordering within CSVs. It operates on the raw byte contents of each file. This is a pragmatic choice: it avoids the complexity and potential bugs of CSV parsing at the checksum stage, while still achieving the goal of being packaging-independent.</p><blockquote><p>The transitland-lib CLI does also include a <code>diff</code> command that can do deeper, row-level comparisons between two feed versions, but that's a topic for another blog post.</p></blockquote><h2 id="try-it-yourself">Try it yourself</h2><p>The <code>transitland</code> command-line interface includes a <code>checksum</code> command that computes both hashes for any GTFS feed on your local machine.</p><p>Here's a real example using <a href="https://www.transit.land/feeds/f-drt-mbta/">MBTA's static feed</a>:</p><pre><code>$ wget https://cdn.mbta.com/MBTA_GTFS.zip
$ transitland checksum MBTA_GTFS.zip

Zip SHA1 (archive file): 6d768217c1e441cc13f313595856669d4c24c013
Directory SHA1 (feed contents): 34e21ac222935b7acdd395e5f6b36dc996da0d60
Find via Transitland website: https://www.transit.land/feed-versions/6d768217c1e441cc13f313595856669d4c24c013
Find via Transitland REST API: https://transit.land/api/v2/rest/feed_versions/6d768217c1e441cc13f313595856669d4c24c013?apikey=YOUR_API_KEY
</code></pre><p>The CLI gives you everything you need to cross-reference the local file against Transitland's global archive. Click the website link and you'll land directly on <a href="https://www.transit.land/feed-versions/6d768217c1e441cc13f313595856669d4c24c013">that feed version's page</a>, where you can see when it was fetched, what date range it covers, and explore its stops, routes, and agencies. The CLI's output can also be compared against another zip file you have locally.</p><p>This is useful in a few scenarios:</p><ul><li><strong>Agency staff</strong> can verify that Transitland has successfully fetched and archived their latest feed. Publish an update, wait for the next fetch cycle, then run <code>transitland checksum</code> on the same file and check if the Zip SHA1 appears in the archive.</li><li><strong>Developers</strong> responsible for legacy systems can ensure their pipelines are only emitting new GTFS feeds when the Directory SHA1 changes from its previously cached value.</li><li><strong>Analysts</strong> who download GTFS feeds directly from agencies can confirm they're working with the same version that Transitland has processed, which is helpful when comparing your analysis against Transitland's API results.</li><li><strong>Researchers</strong> working with historical GTFS data can use the checksum to determine exactly which feed version in Transitland corresponds to a file they've been handed. They can then use Transitland's additional metadata to inform their research.</li></ul><h2 id="installing-the-cli">Installing the CLI</h2><p>The <code>transitland</code> CLI is a single binary. You can:</p><ul><li>download it prebuilt for Linux and macOS from the <a href="https://github.com/interline-io/transitland-lib/releases">transitland-lib releases page</a> on GitHub</li><li>install it <a href="https://github.com/interline-io/homebrew-transitland-lib">using the Homebrew package manager</a></li><li>build it from Golang source if you prefer</li></ul><h2 id="a-small-detail-that-makes-a-big-difference">A small detail that makes a big difference</h2><p>Checksumming transit feeds might sound like plumbing. It's the kind of plumbing that makes a global transit data platform reliable. Without content-aware hashing, Transitland would either miss real updates (by relying on unreliable zip hashes) or drown in false positives (by treating every re-downloaded zip as new data).</p><p>The <code>transitland checksum</code> command also puts this capability in your hands. Whether you're an agency verifying that your feed updates are being picked up, or an analyst reconciling local data against the Transitland archive, you can generate the same fingerprints that Transitland uses internally.</p><p>If you have questions or want to explore further, check out the <a href="https://www.transit.land/documentation/">Transitland documentation</a> or browse the <a href="https://github.com/interline-io/transitland-lib">transitland-lib source code</a> on GitHub.</p>]]></content:encoded></item><item><title><![CDATA[Easily inspect GTFS Realtime using Transitland's website or API]]></title><description><![CDATA[Web browsers and text editors can't open GTFS Realtime. But with Transitland, you now can.]]></description><link>https://www.interline.io/blog/easily-inspect-gtfs-realtime-using-transitlands-website-or-api/</link><guid isPermaLink="false">687e9eed48c47c000167b980</guid><dc:creator><![CDATA[Interline Technologies]]></dc:creator><pubDate>Tue, 22 Jul 2025 03:45:24 GMT</pubDate><media:content url="https://www.interline.io/content/images/2025/07/image-2.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2025/07/image-2.png" alt="Viewing a GTFS Realtime feed as human-readable JSON using the Transitland website"><p>GTFS Realtime feeds provide live updates about vehicle positions, trip updates, and service alerts, but they can be challenging to inspect due to their Protocol Buffer format. This binary <a href="https://protobuf.dev/">format</a> is highly efficient for data transmission but requires special tools to decode. You can't just open it up in a text editor or a web browser.</p><p>Transitland now offers two convenient ways to parse and view GTFS Realtime feeds: through our web interface and via our REST API.</p><h2 id="inspecting-gtfs-realtime-on-the-transitland-website">Inspecting GTFS Realtime on the Transitland Website</h2><p>The Transitland website now provides an intuitive interface for viewing GTFS Realtime data directly in your browser. Let's walk through how to use this feature using Portland TriMet's GTFS Realtime feed as an example.</p><h3 id="finding-a-gtfs-realtime-feed">Finding a GTFS Realtime Feed</h3><ol><li>Navigate to <a href="https://www.transit.land/">www.transit.land</a> and go to the &quot;Source Feeds&quot; section</li><li>Look for feeds with the GTFS Realtime format designation</li><li>Click on any GTFS Realtime feed to view its details</li></ol><figure><img src="https://www.interline.io/content/images/2025/07/image-1.png" alt="browsing Transitland website for GTFS Realtime feeds" width="2000" height="1361"></figure><p>For this example, we'll use the TriMet GTFS Realtime feed at <a href="https://www.transit.land/feeds/f-trimet~rt">https://www.transit.land/feeds/f-trimet~rt</a></p><h3 id="viewing-real-time-data">Viewing Real-time Data</h3><p>On the feed page, you'll see comprehensive information about the feed including:</p><ul><li><strong>Feed identification</strong>: Onestop ID and source URLs</li><li><strong>Last fetch time</strong>: When the data was most recently updated (Transitland typically refreshes GTFS Realtime feeds once per minute)</li><li><strong>Authorization details</strong>: Any required API keys or parameters (Transitland keeps its own credentials private; to access the source feed directly, you'll have to sign up for your own credentials)</li></ul><blockquote><p><a href="https://www.transit.land/documentation/concepts/source-feeds/#gtfs-realtime-feed-fetching-and-caching">Learn more</a> about how Transitland caches GTFS Realtime feeds.</p></blockquote><p>The most exciting feature is the &quot;GTFS Realtime Feed Messages&quot; section, which provides three ways to inspect each type of real-time data:</p><figure><img src="https://www.interline.io/content/images/2025/07/CleanShot-2025-07-21-at-13.25.59.png" alt="screenshot of https://www.transit.land/feeds/f-trimet~rt" width="2000" height="2743"></figure><h3 id="interactive-json-viewer">Interactive JSON Viewer</h3><p>When you click &quot;View as JSON&quot; for any real-time endpoint, a modal window opens showing:</p><ul><li><strong>Last fetched timestamp</strong>: When the data was fetched from the source feed</li><li><strong>Formatted JSON data</strong>: Human-readable data structure. Click the caret arrows to open or close nested sections.</li><li><strong>Download options</strong>: Buttons to download the entire JSON document or copy it to clipboard</li></ul><figure><img src="https://www.interline.io/content/images/2025/07/image.png" alt="Interactive JSON viewer for TriMet GTFS Realtime vehicle positions feed" width="2000" height="1799"></figure><p>The JSON viewer displays the data in a structured format that's easy to read and understand, showing all the fields defined in the <a href="https://gtfs.org/documentation/realtime/reference/">GTFS Realtime specification</a>.</p><h2 id="accessing-gtfs-realtime-via-the-transitland-api">Accessing GTFS Realtime via the Transitland API</h2><p>For programmatic access or integration into your applications, the Transitland REST API provides direct access to Transitland's cached GTFS Realtime data.</p><h3 id="api-endpoint-structure">API Endpoint Structure</h3><p>The API uses the following endpoint pattern:</p><pre><code>GET https://transit.land/api/v2/rest/feeds/{feed_key}/download_latest_rt/{rt_type}.{format}
</code></pre><p>Where:</p><ul><li><code>{feed_key}</code> is the feed's Onestop ID (e.g., <code>f-trimet~rt</code>)</li><li><code>{rt_type}</code> is one of:
<ul><li><code>vehicle_positions</code> for vehicle location data</li><li><code>trip_updates</code> for arrival/departure time changes</li><li><code>alerts</code> for service disruptions</li></ul></li><li><code>{format}</code> is either <code>json</code> or <code>pb</code> (raw Protocol Buffers)</li></ul><h3 id="example-api-call">Example API Call</h3><p>To access TriMet's cached GTFS Realtime service alerts through the Transitland REST API in JSON format:</p><pre><code><span><span>curl</span><span> https://transit.land/api/v2/rest/feeds/f-trimet~rt/download_latest_rt/alerts.json?apikey=YOUR_API_KEY
</span></span></code></pre><h3 id="api-response-format">API Response Format</h3><p>When requesting JSON format, the API returns data in the standard GTFS Realtime JSON structure. For example:</p><pre><code><span><span>{
</span></span><span><span>  &quot;header&quot;</span><span>: {
</span></span><span><span>    &quot;gtfsRealtimeVersion&quot;</span><span>: </span><span>&quot;2.0&quot;</span><span>,
</span></span><span><span>    &quot;incrementality&quot;</span><span>: </span><span>&quot;FULL_DATASET&quot;</span><span>,
</span></span><span><span>    &quot;timestamp&quot;</span><span>: </span><span>&quot;1753153829&quot;
</span></span><span><span>  },
</span></span><span><span>  &quot;entity&quot;</span><span>: [
</span></span><span><span>    {
</span></span><span><span>      &quot;alert&quot;</span><span>: {
</span></span><span><span>        &quot;activePeriod&quot;</span><span>: [
</span></span><span><span>          {
</span></span><span><span>            &quot;start&quot;</span><span>: </span><span>&quot;1675370449&quot;
</span></span><span><span>          }
</span></span><span><span>        ],
</span></span><span><span>        &quot;descriptionText&quot;</span><span>: {
</span></span><span><span>          &quot;translation&quot;</span><span>: [
</span></span><span><span>            {
</span></span><span><span>              &quot;text&quot;</span><span>: </span><span>&quot;No service to the eastbound stop at SW Tualatin-Sherwood Rd &amp; Langer Farms (Stop ID 13837) due to Tualatin-Sherwood Rd project.&quot;
</span></span><span><span>            }
</span></span><span><span>          ]
</span></span><span><span>        },
</span></span><span><span>        &quot;headerText&quot;</span><span>: {
</span></span><span><span>          &quot;translation&quot;</span><span>: [
</span></span><span><span>            {
</span></span><span><span>              &quot;text&quot;</span><span>: </span><span>&quot;&quot;
</span></span><span><span>            }
</span></span><span><span>          ]
</span></span><span><span>        },
</span></span><span><span>        &quot;informedEntity&quot;</span><span>: [
</span></span><span><span>          {
</span></span><span><span>            &quot;routeId&quot;</span><span>: </span><span>&quot;97&quot;
</span></span><span><span>          },
</span></span><span><span>          {
</span></span><span><span>            &quot;stopId&quot;</span><span>: </span><span>&quot;13837&quot;
</span></span><span><span>          }
</span></span><span><span>        ],
</span></span><span><span>        &quot;url&quot;</span><span>: {
</span></span><span><span>          &quot;translation&quot;</span><span>: [
</span></span><span><span>            {
</span></span><span><span>              &quot;text&quot;</span><span>: </span><span>&quot;https://trimet.org/alerts/&quot;
</span></span><span><span>            }
</span></span><span><span>          ]
</span></span><span><span>        }
</span></span><span><span>      },
</span></span><span><span>      &quot;id&quot;</span><span>: </span><span>&quot;164849&quot;
</span></span><span><span>    }
</span></span></code></pre><h3 id="finding-available-gtfs-realtime-feeds">Finding Available GTFS Realtime Feeds</h3><p>To discover available GTFS Realtime feeds, use the <a href="https://www.transit.land/documentation/rest-api/feeds">feeds REST endpoint</a> with the <code>spec=gtfs-rt</code> parameter:</p><pre><code><span><span>curl</span><span> https://transit.land/api/v2/rest/feeds?spec=gtfs-rt</span><span>&amp;apikey</span><span>=</span><span>YOUR_API_KEY
</span></span></code></pre><p>You can also <a href="https://www.transit.land/documentation/rest-api/feeds">filter by other criteria</a> to find feeds relevant to your needs.</p><h2 id="benefits-of-using-transitland-for-gtfs-realtime">Benefits of Using Transitland for GTFS Realtime</h2><p>Transitland's website and API provide simplified access:</p><ul><li><strong>No Protocol Buffer tools required</strong>: View data directly in JSON format</li><li><strong>Real-time updates</strong>: Access the latest data as it becomes available</li><li><strong>Standardized interface</strong>: Consistent API across all feeds</li></ul><p>Transitland also offers comprehensive coverage:</p><ul><li><strong>Multiple feed types</strong>: Vehicle positions, trip updates, and service alerts (when available)</li><li><strong>Global coverage</strong>: Access feeds from transit agencies worldwide</li></ul><blockquote><p>Don't see a GTFS Realtime feed available through Transitland? Help <a href="https://www.transit.land/documentation/atlas">add it the Transitland Atlas feed registry</a>.</p><p>Alternatively, transit agency staff are welcome to email us GTFS Realtime feed URLs at <a href="mailto:hello@transit.land">hello@transit.land</a>. You may also share auth credentials, which we will keep private.</p></blockquote><blockquote><p>Finally, please note that some agencies do not allow Transitland to redistribute their feeds in full. In these cases, GTFS Realtime feed export is not allowed through either the Transitland API or website.</p></blockquote><h2 id="getting-started">Getting Started</h2><ul><li><strong>Sign up</strong>: Inspecting GTFS Realtime feeds requires a <a href="https://app.interline.io/">free Interline account</a> (which helps us to limit concurrent requests and ensure prompt responses).</li><li><strong>For web browsing</strong>: Visit <a href="https://www.transit.land/">www.transit.land</a> and explore the Source Feeds section.</li><li><strong>For API access</strong>: Get an API key by following the <a href="https://www.transit.land/documentation/index#signing-up-for-an-api-key">sign-up instructions</a>.</li></ul><p>Whether you're an analyst learning your way around GTFS data or an agency staffer trying to quickly debug your own systems, Transitland provides the tools you need to automatically parse and read GTFS Realtime feeds. The combination of our web interface and REST API makes it simple to explore real-time transit data without the complexity of Protocol Buffer conversion tools.</p><style>html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}</style>]]></content:encoded></item><item><title><![CDATA[Announcing transitland-lib v1.0.0]]></title><description><![CDATA[We're pleased to announce the release of transitland-lib version 1.0.0. This software library, written in the Go programming language, is the foundation of the Transitland platform and integral to a growing range of purpose-built applications.]]></description><link>https://www.interline.io/blog/announcing-transitland-lib-v1-0-0/</link><guid isPermaLink="false">67d3036abc10db0001cc01b7</guid><dc:creator><![CDATA[Interline Technologies]]></dc:creator><pubDate>Fri, 28 Mar 2025 19:56:25 GMT</pubDate><media:content url="https://www.interline.io/content/images/2025/03/CleanShot-2025-03-26-at-13.52.03-1.jpg" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2025/03/CleanShot-2025-03-26-at-13.52.03-1.jpg" alt="Announcing transitland-lib v1.0.0"><p>We're pleased to announce the release of transitland-lib version 1.0.0. This software library, written in the <a href="https://go.dev/">Go programming language</a>, is the foundation of the Transitland platform and integral to a growing range of purpose-built applications.</p><h2 id="from-experiment-to-production">From experiment to production</h2><p>In 2019, the Interline team began exploring how Go's performance characteristics could serve as a new foundation for <a href="https://www.interline.io/blog/tlv2/">Transitland's v2 platform</a>. The goal was simple: process the largest and most complex GTFS feeds faster and more efficiently.</p><p>After years of iteration and real-world testing, with hundreds of thousands of GTFS feeds processed, we've finally tagged a v1.0.0. This release represents a stable production-ready library that powers critical transit data systems.</p><h2 id="real-world-uses">Real-world uses</h2><p>transitland-lib can be found within many transit data platforms operated by Interline:</p><ul><li>The canonical Transitland deployment at <a href="https://www.transit.land/">www.transit.land</a></li><li><a href="https://www.interline.io/blog/tag/511-sf-bay-regional-feed/">MTC's 511 Regional Feed for the San Francisco Bay Area</a></li><li>A new transit analysis tool for public agencies across the states of Washington, Oregon, and California</li><li>A routing engine deployment for a business client that covers an entire country</li><li>The new <a href="https://www.transit.land/documentation/routing-platform/transit-otp/">Transitland Routing API</a> beta that currently provides journey planning across the entire United States</li></ul><p>This component is also in use by other organizations. For example, Portland TriMet uses transitland-lib to efficiently clip <a href="https://www.transit.land/feeds/f-9-amtrak~amtrakcalifornia~amtrakcharteredvehicle">Amtrak's country-wide feed</a> to create a focused feed to ingest into their own <a href="https://trimet.org/home/planner/">OpenTripPlanner routing engine for the greater Portland, Oregon, region</a>.</p><h2 id="built-for-handling-many-feeds">Built for handling many feeds</h2><p>transitland-lib provides:</p><ul><li>Fast feed processing for both static GTFS and GTFS Realtime</li><li>Memory-efficient operations</li><li>Concurrent data handling</li><li>Robust error handling</li><li>Defining multiple input feeds using the <a href="https://github.com/transitland/distributed-mobility-feed-registry">Distributed Mobility Feed Registry (DMFR) format</a></li></ul><h2 id="performance-in-action">Performance in action</h2><p>Let's look at one example of transitland-lib's capabilities to process and validate massive GTFS feeds. This demonstration covers Switzerland's entire transit network, one of the largest GTFS feeds in the world (<a href="https://www.transit.land/feeds/f-u0-switzerland">view on transit.land</a>):</p><pre><code><span><span>✗ transitland validate https://data.opentransportdata.swiss/en/dataset/timetable-2025-gtfs2020/permalink
</span></span></code></pre><figure><img src="https://www.interline.io/content/images/2025/03/CleanShot-2025-03-26-at-13.52.03.jpg" alt="" width="1200" height="884"></figure><p>In about one and a half minutes on a laptop, transitland-lib downloads and processes a GTFS feed including 460 separate agencies, over 90,000 stops, over 4,000 routes, over 1.2 million trips, and a complicated set of calendar and calendar date records.</p><p>The library's ability to handle such large datasets reliably makes it an ideal choice for processing national-scale transit feeds or creating regional subsets of larger networks.</p><h2 id="extensible-architecture">Extensible architecture</h2><p>transitland-lib's extension framework enables support for both standard GTFS and <a href="https://gtfs.org/community/extensions/overview/">experimental extensions</a>. This flexibility has been crucial for:</p><ul><li>Processing <a href="https://www.transitwiki.org/TransitWiki/images/e/e7/GTFS%2B_Additional_Files_Format_Ver_1.7.pdf">MTC GTFS+ files</a> for the SF Bay Regional Feed</li><li>Supporting <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-fares-updates/">GTFS Fares-v2</a> as it has iteratively evolved from proposal to adopted specification</li><li>Supporting the addition of <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#station-pathways-and-levels">GTFS Pathways</a> using <a href="https://www.interline.io/saas-tools/station-editor/">Interline's Station Editor</a></li></ul><p>We update transitland-lib to track the latest static GTFS and GTFS Realtime specifications. To check which version of the specifications you are using, simply run the <code>transitland version</code> command:</p><pre><code><span><span>✗ transitland version
</span></span><span><span>transitland-lib version: v1.0.0
</span></span><span><span>transitland-lib commit: https://github.com/interline-io/transitland-lib/commit/bae91cd7f32c67ffe89965cab3636e22ddbce817 (time: 2025-03-11T18:47:50Z)
</span></span><span><span>GTFS specification version: https://github.com/google/transit/blob/11a49075c1f50d0130b934833b7eeb3fe518961c/gtfs/spec/en/reference.md
</span></span><span><span>GTFS Realtime specification version: https://github.com/google/transit/blob/7b9f229dfa0b539c3fcf461986638890024feb06/gtfs-realtime/proto/gtfs-realtime.proto
</span></span></code></pre><h2 id="validation-and-best-practices">Validation and best practices</h2><p>GTFS feeds can be messy. Therefore, transitland-lib:</p><ul><li>Validates required fields and data types</li><li>Ensures referential integrity between files</li><li>Checks that feeds meet many of the <a href="https://gtfs.org/documentation/schedule/schedule-best-practices/">best practices</a></li><li>Also supports validating GTFS Realtime feeds</li></ul><p>With both static GTFS and GTFS Realtime processed using the same library, we're now able to put into ongoing production-scale use <a href="https://www.interline.io/blog/trb-transit-idea-gtfs-realtime-report/">Interline's previous research with University of South Florida into GTFS Realtime validation</a>.</p><blockquote><p>While transitland-lib implements many of the same validation and best practice rules as <a href="https://github.com/MobilityData/gtfs-validator">MobilityData's static GTFS validator</a>, the purposes and advantages of each library are distinct.</p><p>The MobilityData validator lends itself well to interactive use when people and organizations creating GTFS feeds want to check to ensure their static feed is ready for submission to a third-party consumer. The MobilityData validator outputs a report that is useful for reference and sharing.</p><p>In contrast, transitland-lib is designed to efficiently process many feeds, to serve as a component within production-scale processing pipelines, and to optionally validate GTFS Realtime endpoints with the context of a static GTFS feed with associated schedules.</p></blockquote><p>transitland-lib also gives developers options for how to handle invalid data:</p><ul><li>For the canonical Transitland deployment, we've configured transitland-lib to fail <em>soft</em>. When errors are limited to certain rows in a GTFS feed, those entities are filtered out. The rest of the feed's data is imported and made available to users.</li><li>For some client deployments, we've configured transitland-lib to fail <em>hard</em>. Any error in a source feed will halt the entire workflow and instantly notify staff to review the issue.</li></ul><p>Each application may have different goals and different tradeoffs for how to handle invalid data, and transitland-lib provides options to customize and control this behavior.</p><h2 id="powerful-filtering-system">Powerful filtering system</h2><p>transitland-lib's filtering system enables modifying and transforming GTFS feeds for specific project contexts, such as:</p><ul><li>Producing date-specific GTFS feeds</li><li>Building regional routing engines with optimized data</li><li>Merging together many source feeds to create regional, state, or national scale feeds</li><li>Standardizing data across multiple agencies, with customized rules for namespacing entities</li></ul><p>Interline uses transitland-lib to <a href="https://app.interline.io/contact_forms/curated_gtfs">curate full-service data deliveries for our clients needing consistent GTFS data across states and countries</a>.</p><h2 id="using-transitland-lib">Using transitland-lib</h2><p>If you use the Transitland website or the Transitland APIs, you're already using transitland-lib.</p><p>Developers can visit <a href="https://github.com/interline-io/transitland-lib">github.com/interline-io/transitland-lib</a> to learn more about transitland-lib as a command-line interface (CLI) or a library for programmatic usage.</p><p>transitland-lib is available under a <a href="https://github.com/interline-io/transitland-lib/blob/main/LICENSE">dual license model</a> enabling open-source use under the permissive GPLv3 license and also providing flexibility to Interline's clients under a customizable business license.</p><p>We look forward to continuing to maintain transitland-lib for another six years as the GTFS and GTFS Realtime specifications continue to evolve and as the number and size of source feeds continues to grow.</p><style>html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}</style>]]></content:encoded></item><item><title><![CDATA[US National Transit Database releases data and requests more feedback]]></title><description><![CDATA[Our recommendations to the US Federal Transit Administration regarding National Transit Database: Proposed Reporting Changes and Clarifications for Report Years 2025 and 2026. ]]></description><link>https://www.interline.io/blog/us-national-transit-database-releases-data-and-requests-more-feedback-2/</link><guid isPermaLink="false">676995f6ea4ece000174601e</guid><dc:creator><![CDATA[Interline Technologies]]></dc:creator><pubDate>Tue, 24 Dec 2024 22:54:17 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/12/CleanShot-2024-12-24-at-14.58.25@2x.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/12/CleanShot-2024-12-24-at-14.58.25@2x.png" alt="US National Transit Database releases data and requests more feedback"><blockquote><p><strong>January 3, 2025</strong>: FTA has extended the comment period to January 29 regarding <a href="https://www.regulations.gov/document/FTA-2024-0013-0001">National Transit Database: Proposed Reporting Changes and Clarifications for Report Years 2025 and 2026</a>.</p></blockquote><blockquote><p>We've submitted the following public comment to the Federal Transit Administration regarding <a href="https://www.regulations.gov/document/FTA-2024-0013-0001">National Transit Database: Proposed Reporting Changes and Clarifications for Report Years 2025 and 2026</a>.</p></blockquote><blockquote><p>For background, see earlier posts on the Interline blog: &quot;<a href="https://www.interline.io/blog/us-ntd-reporting-gtfs-adopted/">US Federal Transit Administration finalizes plan to collect GTFS URLs in US National Transit Database</a>&quot; (2023) and &quot;<a href="https://www.interline.io/blog/us-ntd-reporting-gtfs/">US National Transit Database to collect GTFS URLs</a>&quot; (2022).</p></blockquote><p>Interline Technologies is the maintainer of the <a href="https://www.transit.land/">Transitland open data platform</a>. Transitland aggregates approximately 1,000 static GTFS feeds from across the United States (as well as real-time feeds, plus feeds from international sources). To power the platform's APIs, our staff and external contributors maintain the <a href="https://github.com/transitland/transitland-atlas">Transitland Atlas feed registry repository</a>, which is publicly hosted on GitHub under a permissive license. This feed registry maintains lists of public GTFS feed URLs in addition to associated metadata (including US NTD IDs). The following comments regarding the proposed NTD reporting changes are based on our experience creating and operating these open-data systems since 2015.</p><h2 id="re-section-b-additional-data-within-publicly-hosted-general-transit-feed-specification-gtfs-datasets"><strong>Re Section B. Additional Data Within Publicly Hosted General Transit Feed Specification (GTFS) Datasets</strong></h2><p>Thank you to agencies and NTD for beginning to collect GTFS feeds from reporting agencies. The <a href="https://www.transit.dot.gov/ntd/data-product/2023-annual-database-general-transit-feed-specification-gtfs-weblinks">2023 Annual Database General Transit Feed Specification (GTFS) Weblinks dataset</a> is a useful public release.</p><blockquote><p>Transitland Atlas maintainers and contributors are reviewing the NTD weblinks dataset to look for new feed sources to add to the Transitland Atlas, as well as out-of-date references to update in Transitland Atlas. To help, see the <a href="https://github.com/transitland/transitland-atlas/tree/main/scripts/ntd-gtfs-weblinks">ntd-gtfs-weblinks projects subdirectory</a>.</p></blockquote><blockquote><p><strong>For transit agencies currently unable to host their GTFS feed at a public URL</strong>: We invite you to email a copy of your latest feed to our team at <a href="mailto:hello@transit.land">hello@transit.land</a></p></blockquote><p>To further increase the use of this weblinks dataset in future years, please guide as many agencies as possible to host their GTFS feeds at URLs that are public and stable. That is:</p><ul><li>URLs - hosting their GTFS feed on an agency-controlled website (instead of submitting feed version as archives via email)</li><li>public - a URL that third-parties can download from (rather than a private submission to NTD)</li><li>stable - a URL that remains the same, while the GTFS feed archive itself is changed as the agency releases new schedules/versions (i.e., no dates in the file name)</li></ul><p>Finally, please keep in mind that it's best for agencies to be reporting to NTD the exact same GTFS feed versions that they share with third-party navigation apps and open-data aggregation platforms. This ensures that all data consumers are working from similar information, and makes it easier for data producers to focus their efforts. This is even more relevant for agencies that produce GTFS Realtime feeds as well (which need to match certain identifiers within associated static feeds). Our feedback regarding adding NTD IDs is also based on this goal of preventing agencies from having to produce separate feed files for NTD reporting purposes than they already produce for trip-planning and operational purposes.</p><h2 id="re-align-agency_id-and-ntd-id-within-gtfs-file"><strong>Re Align <code>agency_id</code> and NTD ID Within GTFS File</strong></h2><p>We support the proposed goal of adding NTD IDs to GTFS feeds. NTD IDs are a useful &quot;crosswalk&quot; between the rider-facing data in a GTFS feed and the operational/planning/financial datasets in the NTD. However, we recommend that the means of adding NTD IDs to GTFS feeds be carefully considered. Interline strongly agrees with related feedback submitted from other commenters, including <a href="https://www.regulations.gov/comment/FTA-2024-0013-0003">MBTA</a> and the <a href="https://www.regulations.gov/comment/FTA-2024-0013-0006">MobilityData</a> non-profit (of which Interline is also a member).</p><p>We recommend:</p><ol><li><strong>NTD IDs should be added to GTFS feeds in a flexible manner that can map on to an entire feed, an agency within a feed, or a subset of routes within a feed.</strong> This is necessary because rider-facing &quot;brand names&quot; of transit agencies do not always line up 1-to-1 with NTD reporting IDs, nor do NTD reporting IDs necessarily line up 1-to-1 with the internal means of organizing an agencies' data systems that produce records in agencies.txt</li><li><strong>NTD IDs be added to GTFS feeds in a way that does not conflict with existing data-producing systems.</strong> This is toward the goal of agencies reporting their existing GTFS feeds to NTD, rather than creating &quot;one off&quot; feed versions that they customize and submit once a year to NTD. Based on our experience working with a wide range of transit agencies, we recommend that data-producers have the choice to either add an entirely new file or add columns to existing files. For some agencies, it may be simpler to add a single hard-coded file to their feed, while for other agencies, it may be simpler to add an extra custom column to the relevant files in their feed. </li><li><strong>NTD IDs be added to GTFS feeds in a way that does not conflict with existing data-consuming systems.</strong> Consumers can choose to accept or ignore new columns and/or new files in GTFS feeds. This makes it simpler for them to &quot;opt in&quot; rather than to have to adapt to a change in the meaning of agency_id.</li></ol><p>Our specific recommendation for your consideration:</p><ul><li>Give reporters two approaches for adding NTD IDs to their feeds.</li><li>The first approach will be to add a file named <code>us-ntd-ids.txt</code> (or similar) to their feeds. This would be a CSV file in which each row maps an NTD ID to a chosen entity (e.g., an agency or a route). For a simple situation in which there is one NTD ID for a single agent in a feed, this file would only have one row (after the header row); for a complex situation, there may be multiple rows defining how different agencies and/or routes are mapped to different NTD IDs.</li><li>The second approach will be to add a <code>us_ntd_id</code> column (or similar) to existing <code>feed_info.txt</code>, <code>agencies.txt</code>, and/or <code>routes.txt</code> files. Data producers could add this column to just the files in which they want to list one or more NTD IDs.</li></ul><p>Our overall recommendation is to provide flexibility to data-producers in where and how they add US NTD IDs. This will lower the uncertainty of changing existing fields (such as the meaning of the <code>agency_id</code>). This will also decrease the technical trade-offs for agencies with complicated existing IT systems, which may limit how they can change or customize their GTFS feeds.</p><p>This proposal does slightly increase the burden of data-consumers to extract the relevant NTD IDs from GTFS feeds and figure out how to apply them to feed contents. We believe that is a reasonable burden for data-consumers to take on. Transitland already does similar operations across the thousands of GTFS feeds it consumes. In exchange, the benefit of this approach is that it encourages agencies to fit NTD IDs into their production systems (rather than producing one-off feed versions just for NTD reporting).</p><p>Finally, we recommend reviewing uptake of these two approaches by agencies after a year or two. Perhaps one approach will be fine and the requirement can be refined in the future. Still, we recommend beginning with reporting agencies having more initial flexibility.</p><h2 id="re-shapestxt-file-geospatial-drawing-of-routes-as-part-of-gtfs-submission"><strong>Re <code>shapes.txt</code> File (Geospatial Drawing of Routes) as Part of GTFS Submission</strong></h2><p>We agree that recommending agencies add <code>shapes.txt</code> files for all routes/trips will be beneficial to data consumers, especially trip-planners, GIS analyses, and visualizations. However, in our experience, the quality of <code>shapes.txt</code> geometries can vary wildly. Imprecise and inaccurate shapes can often be worse than no shapes. Data consumers such as Transitland and trip planning apps already have to run automated checks to see whether our systems even want to ingest <code>shapes.txt</code> records for a given trip or to instead ignore them.</p><p>The <a href="https://gtfs.org/documentation/schedule/examples/shapes/">shapes guidance webpage</a> shared by MobilityData is worth sharing with agencies. It's also worth sharing with agencies and reiterating the value of the <a href="https://gtfs.org/documentation/schedule/schedule-best-practices/">GTFS &quot;best practices.&quot;</a></p><p>Thank you for accepting this feedback.</p>]]></content:encoded></item><item><title><![CDATA[US Federal Transit Administration finalizes plan to collect GTFS URLs in US National Transit Database]]></title><description><![CDATA[To meet meet the Bipartisan Infrastructure Law's requirement to collect data on American transit agencies' "geographic service area coverage," the US Federal Transit Administration will begin to require GTFS feeds from fixed-route transit agencies that receive federal funds.]]></description><link>https://www.interline.io/blog/us-ntd-reporting-gtfs-adopted/</link><guid isPermaLink="false">67255784b62841000136edf4</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Fri, 28 Apr 2023 07:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/bil-geographic-service-coverage-area.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/bil-geographic-service-coverage-area.png" alt="US Federal Transit Administration finalizes plan to collect GTFS URLs in US National Transit Database"><p>The US Bipartisan Infrastructure Law passed by Congress and signed into law by President Biden in 2021 was big, very big. While most people have focused on the large sums of funding that are now becoming available to improve American transportation infrastructure, we’ve focused on one of the details: The “BIL” requires the US Federal Transit Administration to collect data on American transit agencies’ “geographic service area coverage.”</p><figure><img src="https://www.interline.io/content/images/2024/11/bil-geographic-service-coverage-area-1.png" alt="first page of FTA's notice and request for comments regarding additions to the NTD reporting process" width="1130" height="1462"><figcaption><p>The US Bipartian Infrastructure Law, formally known as the <a href="https://www.congress.gov/bill/117th-congress/house-bill/3684/text">Infrastructure Investment and Jobs Act</a>, added a requirement that the US National Transit Database collect data on &quot;geographic service area coverage&quot; for American transit agencies receiving federal funds.</p></figcaption></figure><h2 id="proposal">Proposal</h2><p>To meet this requirement, the FTA issued a proposal on July 7, 2022 to collect static GTFS feeds from all American transit agencies that receive federal funds and operate fixed-route transit service. The <a href="https://www.interline.io/blog/us-ntd-reporting-gtfs/">Interline team submitted feedback to FTA</a> based on our experience operating the Transitland open transit data platform, which aggregates GTFS feeds from nearly a thousand transit operators in the US (and over 3,000 when including other countries).</p><h2 id="adoption">Adoption</h2><p>On March 3, 2023, the FTA released responses to feedback and formally adopted its proposal, with no changes made to the GTFS reporting requirement. For more information see <a href="https://www.federalregister.gov/documents/2023/03/03/2023-04379/national-transit-database-reporting-changes-and-clarifications">this notice in the US Federal Register</a>.</p><h2 id="timing">Timing</h2><p>While the NTD reporting requirements have now been formally changed, agencies will have some time until they begin submitting their GTFS feeds. In response to a question from Interline about timing, the NTD team tells us that:</p><blockquote><p>NTD will begin collecting GTFS data in Report Year 2023. Agencies will submit 2023 data to the NTD beginning in October of 2023, and the associated RY23 data products containing GTFS data will be released in Fall of 2024.</p></blockquote><h2 id="data-release">Data release</h2><p>When US National Transit Database Report Year 2023 data products are released, they will be available on the <a href="https://www.transit.dot.gov/ntd/ntd-data">NTD Data</a> website.</p><h2 id="improvements-big-and-small">Improvements big and small</h2><p>The word that’s been most often said in recent years about the BIL is that it’s a “generational” amount of funding and policy change for American physical infrastructure and transportation networks.</p><p>The collection and dissemination of URLs for GTFS feeds by the US Federal Transit Administration may not be as massive of a change in dollar amounts. Yet, this change still reflects a substantive development in the way that the US federal government engages with GTFS. We’re looking forward to seeing how FTA and other parts of the US Department of Transportation continue to promote and engage with transit data.</p>]]></content:encoded></item><item><title><![CDATA[Updated GTFS-Fares v2 for the SF Bay Area]]></title><description><![CDATA[MTC and Interline have released updated fares and transfer discounts for over 30 Bay Area transit agencies using the adopted GTFS-Fares v2 "base implementation" and a series of proposed extensions. ]]></description><link>https://www.interline.io/blog/mtc-regional-gtfs-feed-fares-updates/</link><guid isPermaLink="false">672550a2b62841000136ed3b</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Tue, 04 Oct 2022 07:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/ios-directions-with-fares-2.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/ios-directions-with-fares-2.png" alt="Updated GTFS-Fares v2 for the SF Bay Area"><blockquote><p><strong>April 20, 2023</strong>: The GTFS-Fares v2 extension originally named <code>fare_containers.txt</code> has been renamed to <code>fare_media.txt</code>, as part its recent <a href="https://github.com/google/transit/pull/355">adoption in the GTFS specification</a>.</p></blockquote><p>In 2020, Interline and the Metropolitan Transportation Commission <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#fares-and-transfer-discounts">released</a> a beta version of fares and transfer discounts for eight SF Bay Area transit agencies. Since then, we’ve been working together to improve data coverage and its schema. As of this August, the Regional GTFS Feed now includes fares and transfer discounts for over 30 Bay Area transit agencies. The updated dataset uses the adopted GTFS-Fares v2 “base implementation,” as well a few proposed additions to the GTFS-Fares v2 data schema. This data is now available for use, and our team is updating it on an ongoing basis.</p><p>In this blog post we’ll cover:</p><ul><li><a href="#gtfs-fares-v2-in-action">GTFS-Fares v2 in action</a></li><li><a href="#gtfs-fares-v2-base-implementation-and-future-additions-to-the-specification">GTFS-Fares v2 base implementation and future additions to the specification</a></li><li><a href="#sf-bay-area-transit-agencies-with-gtfs-fares-v2-data">SF Bay Area transit agencies with GTFS-Fares v2 data</a></li><li><a href="#using-gtfs-fares-v2-data-for-the-bay-area">Using GTFS-Fares v2 data for the Bay Area</a></li></ul><h2 id="gtfs-fares-v2-in-action">GTFS-Fares v2 in action</h2><p>If you have an Apple iPhone running <a href="https://www.apple.com/ios/ios-16/">iOS 16</a>, you can see the estimated costs of transit journeys. This data is sourced from the the Regional Feed.</p><p>All of the above fares information is defined in the Regional GTFS Feed using the GTFS-Fares v2 specification. Here are extracts of the relevant GTFS files from the feeds for the itinerary displayed on the iPhone:</p><h3 id="fare_productstxt">fare_products.txt</h3><table><thead><tr><th>fare_product_id</th><th>fare_product_name</th><th>amount</th><th>currency</th><th>rider_category_id</th><th>fare_media_id</th></tr></thead><tbody><tr><td><code>AC:local:single</code></td><td>AC Transit local fare</td><td>2.25</td><td>USD</td><td><code>adult</code></td><td><code>clipper</code></td></tr><tr><td><code>BA:matrix:FTVL-EMBR</code></td><td>generated</td><td>2.15</td><td>USD</td><td><code>adult</code></td><td><code>clipper</code></td></tr></tbody></table><h3 id="fare_mediatxt-updated">fare_media.txt Updated</h3><table><thead><tr><th>fare_media_id</th><th>fare_media_name</th><th>fare_media_type</th></tr></thead><tbody><tr><td><code>clipper</code></td><td>Clipper</td><td>2</td></tr></tbody></table><h3 id="fare_leg_rulestxt">fare_leg_rules.txt</h3><table><thead><tr><th>leg_group_id</th><th>from_area_id</th><th>to_area_id</th><th>network_id</th><th>fare_product_id</th></tr></thead><tbody><tr><td><code>AC</code></td><td><code>AC:local</code></td><td><code>AC:local</code></td><td><code>AC</code></td><td><code>AC:local:single</code></td></tr><tr><td><code>BA</code></td><td><code>FTVL</code></td><td><code>EMBR</code></td><td><code>BA</code></td><td><code>BA:matrix:FTVL-EMBR</code></td></tr></tbody></table><h3 id="stop_areastxt">stop_areas.txt</h3><table><thead><tr><th>area_id</th><th>stop_id</th></tr></thead><tbody><tr><td><code>AC:local</code></td><td><code>55227</code></td></tr><tr><td><code>AC:local</code></td><td><code>55550</code></td></tr><tr><td><code>FTVL</code></td><td><code>FTVL</code></td></tr><tr><td><code>EMBR</code></td><td><code>EMBR</code></td></tr></tbody></table><p>Note that this data schema has changed since the examples in <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#an-example-of-fares-v2-in-practice">our first release blog post</a>. This new schema was adopted as part of the GTFS-Fares v2 “base implementation,” which we will now discuss in more detail.</p><h2 id="gtfs-fares-v2-base-implementation-and-future-additions-to-the-specification">GTFS-Fares v2 base implementation and future additions to the specification</h2><p>Since the <a href="https://bit.ly/gtfs-fares">original GTFS-Fares v2 proposal</a>, many stakeholders in GTFS have provided additional input. In May, stakeholders voted to adopt the core of the refined GTFS-Fares v2 proposal as a <a href="https://github.com/google/transit/pull/286">“base implementation”</a>.</p><p>The “base implementation” simplifies the organization of <code>fare_leg_rules.txt</code>, <code>fare_transfer_rules.txt</code>, and <code>fare_products.txt</code>. This is a breaking change, so consumers who implemented their GTFS processing software against the original proposal will need to update their tools.</p><p>To fully capture the complexity of Bay Area transit fares and transfer discounts requires additional data tables and fields that were included in the original proposal but not in the “base implementation.”</p><p>Here is an overview of all the GTFS-Fares v2 tables and columns/fields included in the Regional GTFS Feed:</p><table><thead><tr><th>GTFS file</th><th>Spec version(s)</th><th>Columns added for extensions</th><th>Documentation</th></tr></thead><tbody><tr><td><code>fare_products.txt</code></td><td>&quot;base&quot; with additional columns for extensions</td><td><ul><li><code>rider_category_id</code></li></ul></td><td><a href="https://gtfs.org/schedule/reference/#fare_productstxt">See gtfs.org reference</a></td></tr><tr><td><code>fare_leg_rules.txt</code></td><td>&quot;base&quot; with additional columns for extensions</td><td><ul><li><code>transfer_only</code>*</li></ul></td><td><a href="https://gtfs.org/schedule/reference/#fare_leg_rulestxt">See gtfs.org reference</a></td></tr><tr><td><code>fare_transfer_rules.txt</code></td><td>&quot;base&quot; with additional columns for extensions</td><td><ul><li><code>filter_fare_product_id</code>*</li></ul></td><td><a href="https://gtfs.org/schedule/reference/#fare_transfer_rulestxt">See gtfs.org reference</a></td></tr><tr><td><code>areas.txt</code></td><td>&quot;base&quot;</td><td></td><td><a href="https://gtfs.org/schedule/reference/#areastxt">See gtfs.org reference</a></td></tr><tr><td><code>stop_areas.txt</code></td><td>&quot;base&quot;</td><td></td><td><a href="https://gtfs.org/schedule/reference/#stop_areastxt">See gtfs.org reference</a></td></tr><tr></tr><tr><td><code>fare_media.txt</code></td><td>&quot;base&quot; as of March 14, 2023 Updated</td><td></td><td><a href="https://gtfs.org/schedule/reference/#fare_mediatxt">See gtfs.org reference</a></td></tr><tr><td><code>rider_categories.txt</code></td><td>extension</td><td><ul><li><code>rider_category_id</code></li><li><code>rider_category_name</code></li><li><code>min_age</code></li><li><code>max_age</code></li><li><code>eligibility_url</code></li></ul></td><td><a href="https://docs.google.com/document/d/19j-f-wZ5C_kYXmkLBye1g42U-kvfSVgYLkkG5oyBauY/edit#heading=h.9gqyl4w7f6x1">See proposal Google Doc</a></td></tr><tr><td><code>routes.txt</code></td><td>extension</td><td><ul><li><code>as_route</code></li></ul></td><td><a href="https://docs.google.com/document/d/19j-f-wZ5C_kYXmkLBye1g42U-kvfSVgYLkkG5oyBauY/edit#heading=h.v160xvt5oen5">See proposal Google Doc</a></td></tr></tbody></table><p>* <code>fare_leg_rules.txt transfer_only</code> and <code>fare_transfer_rules.txt filter_fare_product_id</code> are two fields proposed for describing partial credit on inter-agency transfers. Thanks to the <a href="https://transitapp.com/">Transit</a> app team for proposing these additions to make some of the transfer discounts in the Regional GTFS Feed more explicit for consuming software to parse.</p><p>Interline and MTC will continue to produce these “extensions” to the “base” data schema, and we’ll continue to work with data consumers, <a href="https://mobilitydata.org/">MobilityData</a>, and other stakeholders to finalize their adoption into the GTFS spec.</p><h2 id="sf-bay-area-transit-agencies-with-gtfs-fares-v2-data">SF Bay Area transit agencies with GTFS-Fares v2 data</h2><p>Interline and MTC now produce fares, intra-agency transfer discounts, and inter-agency transfer discounts for the following agencies:</p><table><thead><tr><th>511 operator code</th><th>Operator full name</th></tr></thead><tbody><tr><td><code>3D</code></td><td>Tri Delta Transit</td></tr><tr><td><code>AC</code></td><td>AC Transit (including transbay routes)</td></tr><tr><td><code>AF</code></td><td>Angel Island Tiburon Ferry</td></tr><tr><td><code>AM</code></td><td>Capitol Corridor</td></tr><tr><td><code>BA</code></td><td>Bay Area Rapid Transit (BART)</td></tr><tr><td><code>CC</code></td><td>County Connection</td></tr><tr><td><code>CE</code></td><td>Altamont Corridor Express (ACE)</td></tr><tr><td><code>CM</code></td><td>Commute.org Shuttles</td></tr><tr><td><code>CT</code></td><td>Caltrain</td></tr><tr><td><code>DE</code></td><td>Dumbarton Express Consortium</td></tr><tr><td><code>EM</code></td><td>Emery Go-Round</td></tr><tr><td><code>FS</code></td><td>Fairfield and Suisun Transit</td></tr><tr><td><code>GF</code></td><td>Golden Gate Ferry</td></tr><tr><td><code>GG</code></td><td>Golden Gate Transit</td></tr><tr><td><code>MA</code></td><td>Marin Transit</td></tr><tr><td><code>MV</code></td><td>MVgo Mountain View</td></tr><tr><td><code>PE</code></td><td>Petaluma Transit</td></tr><tr><td><code>RV</code></td><td>Rio Vista Delta Breeze</td></tr><tr><td><code>SA</code></td><td>Sonoma Marin Area Rail Transit</td></tr><tr><td><code>SB</code></td><td>San Francisco Bay Ferry</td></tr><tr><td><code>SC</code></td><td>Valley Transportation Authority (VTA)</td></tr><tr><td><code>SF</code></td><td>San Francisco Municipal Transportation Agency (SFMTA)</td></tr><tr><td><code>SI</code></td><td>San Francisco International Airport (SFO)</td></tr><tr><td><code>SM</code></td><td>SamTrans</td></tr><tr><td><code>SO</code></td><td>Sonoma County Transit</td></tr><tr><td><code>SR</code></td><td>Santa Rosa CityBus</td></tr><tr><td><code>SS</code></td><td>City of South San Francisco</td></tr><tr><td><code>ST</code></td><td>SolTrans</td></tr><tr><td><code>TD</code></td><td>Tideline Water Taxi</td></tr><tr><td><code>UC</code></td><td>Union City Transit</td></tr><tr><td><code>VC</code></td><td>Vacaville City Coach</td></tr><tr><td><code>VN</code></td><td>VINE Transit</td></tr><tr><td><code>WC</code></td><td>Western Contra Costa (WestCat)</td></tr><tr><td><code>WH</code></td><td>Livermore Amador Valley Transit Authority (LAVTA)</td></tr></tbody></table><p>Fares are included for Clipper Card, cash, and a wide variety of passes. (<a href="https://www.clippercard.com/">Clipper Card</a> is the contact-less payment card and system operated by MTC and available on buses, trains, and ferries throughout the Bay Area.) Transfer discounts between routes operated by the same agency (intra-agency transfers) are included for all methods of payment. Transfer discounts between routes operated by different agencies (inter-agency transfers) are only included for Clipper Card.</p><p>Our team is updating these fares and transfer discounts on a monthly basis.</p><h2 id="using-gtfs-fares-v2-data-for-the-bay-area">Using GTFS-Fares v2 data for the Bay Area</h2><p>Download the daily Regional Feed like so:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li></ol><p>We welcome questions sent to the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources mailing list</a>.</p><h2 id="acknowledgements">Acknowledgements</h2><p>Credit and many thanks to project team members including Ian Rees (Interline), Nome Dickerson (Garnet Consulting), Nisar Kapeel and Kapeel Daryani (MTC), and our partners at Bay Area transit agencies.</p>]]></content:encoded></item><item><title><![CDATA[US National Transit Database to collect GTFS URLs]]></title><description><![CDATA[American public transit agencies that receive federal funds are required to report information for inclusion in the National Transit Database. The Federal Transit Administration is proposing that each agency will also provide a URL for a GTFS feed in their NTD reports.]]></description><link>https://www.interline.io/blog/us-ntd-reporting-gtfs/</link><guid isPermaLink="false">672557c9b62841000136ee02</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Wed, 07 Sep 2022 07:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/us-ntd-federal-register.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/us-ntd-federal-register.png" alt="US National Transit Database to collect GTFS URLs"><p><strong>April 28, 2023</strong>: The Federal Transit Administration has now adopted these new reporting requirements. For a brief overview, see our more recent <a href="https://www.interline.io/blog/us-ntd-reporting-gtfs-adopted/">blog post describing when FTA will begin requiring US transit agencies to submit URLs for their GTFS feeds to the National Transit Database</a>.</p><p>American public transit agencies that receive federal funds are required to report information for inclusion in the <a href="https://www.transit.dot.gov/ntd">National Transit Database</a>. The Federal Transit Administration, which administers to NTD, is <a href="https://www.regulations.gov/document/FTA-2022-0018-0001">proposing</a> that each agency will also provide a URL for a GTFS feed in their NTD reports. Submitted GTFS feeds will enter the public domain. This is great news for all producers and consumers of GTFS data.</p><figure><img src="https://www.interline.io/content/images/2024/11/us-ntd-federal-register-1.png" alt="first page of FTA's notice and request for comments regarding additions to the NTD reporting process" width="1166" height="1516"><figcaption><p><a href="https://www.regulations.gov/document/FTA-2022-0018-0001">Notice and request for comments</a> on NTD reporting requirements from the Federal Transit Administration.</p></figcaption></figure><p>We’ve submitted feedback to the FTA based on our experience building and operating Transitland, and we’re sharing it here as well:</p><blockquote><p>At Interline Technologies, we specialize in software services for GTFS, GTFS Realtime, and related data specifications. Our Transitland data platform (<a href="https://www.transit.land/">www.transit.land</a>) aggregates open-data feeds from over 3,000 operators in over 50 countries, including 908 operators in the United States. We serve organizations that both produce and consume these data feeds. Our clients include public transit agencies, private mobility providers, planning firms, and academic research institutions. This is just a sampling of the many types of firms that make use of GTFS and related data feeds.</p></blockquote><blockquote><p>Based on our own experience, as well as the experience of our clients, we see much value in FTA’s proposed addition to the NTD reporting process to have agencies provide a public GTFS feed. Increasing the number and accessibility of openly licensed GTFS feeds with stable URLs from American transit agencies will have many benefits. In addition to being useful for transit trip planning (the best known use-case for GTFS), this will also assist with transportation planning, land-use planning, real-estate assessment, academic research, and local advocacy. Coverage of the largest and the smallest transit agencies is important for many of these endeavors, as is comprehensive coverage of both urban and rural operators. Using NTD reporting to promote the production and open sharing of GTFS data feeds will assist with all of these use-cases.</p></blockquote><blockquote><p>We are pleased to see that agencies will be encouraged to report a stable URL that is updated on their own servers. Transitland and similar data aggregators operate best when the “source of truth” for each feed is maintained on each agency’s servers and the aggregator fetches from that location on a regular basis.</p></blockquote><blockquote><p>We encourage FTA to consider multiple means of distributing the lists of GTFS feed URLs. While most NTD data is included in a single Excel file with multiple sheets, we would propose that FTA also disseminate a separate file that just lists GTFS feed URLs. Ideally this would be in the CSV format, be hosted at a stable URL, and include the associated NTD IDs for each GTFS feed URL. In addition to the NTD ID, we suggest also listing the agency’s ID used on the NTD website for agency profile pages. For example, the final portion of <code>https://www.transit.dot.gov/ntd/transit-agency-profiles/san-francisco-bay-area-rapid-transit-district</code></p></blockquote><blockquote><p>FTA may consider posting a CSV listing the feed URLs, associated NTD IDs, and NTD website IDs to an open-data portal such as data.gov on an ongoing basis. This will be useful for a very wide range of users.</p></blockquote><blockquote><p>At present, our staff and external contributors maintain associations between GTFS feed URLs and NTD IDs in the Transitland Atlas repository on GitHub. It is open to public use and contributions at <a href="https://github.com/transitland/transitland-atlas">https://github.com/transitland/transitland-atlas</a> While we believe this structure is likely more complex than is necessary for FTA and NTD’s purposes, we offer this as one example for how to provide information on many feeds, along with metadata about each operator and (when relevant) its US NTD ID.</p></blockquote><figure><img src="https://www.interline.io/content/images/2024/11/bart-operator-page-screenshot.png" alt="screenshot of BART operator detail page listing its US NTD ID" width="2000" height="1228"><figcaption><p>US NTD ID listed on <a href="https://www.transit.land/operators/o-9q9-bart">Transitland's operator detail page for BART</a>.</p></figcaption></figure><blockquote><p>Finally, we strongly support the proposed change that “upon publication, all GTFS data submitted to the NTD will enter the public domain.” Since 2014, our team has been working to improve the state of licensing for GTFS data. At our last company (Mapzen) we prepared a model license for GTFS feeds and discussed it in depth with USDOT OST staff. At Interline, we continue to assess the state of licenses and terms attached to public transit data. In the Transitland Atlas repository mentioned previously, our staff maintain metadata about the licenses attached to many of the feeds. This is important for our institutional clients, so they can be assured that we and they are making a good faith effort to follow the license and terms for each agency. However, all of this effort to track and review such a wide range of licenses is resource intensive. We do not think it is a good use of anyone’s time. Furthermore, we do not believe it benefits or protects transit agencies that produce GTFS data. GTFS feeds contain no privileged data, nor do they contain personally identifiable information (PII). Publishing a GTFS feed in the public domain does not entail giving up other protections such as a transit agency’s trademark of their brand name and logo. Transit agencies benefit when their GTFS feeds are useful widely by as many consumers as possible. We are pleased to see FTA proposing to simplify this overly complicated situation by instead requiring that GTFS feeds submitted to the NTD reporting process enter the public domain.</p></blockquote><blockquote><p>Thank you for taking these comments. You are welcome to reach us at <a href="mailto:info@interline.io">info@interline.io</a> if we can provide any more information.</p></blockquote>]]></content:encoded></item><item><title><![CDATA[Observed stop arrivals and departures through the SF Bay Area]]></title><description><![CDATA[Interline and the Metropolitan Transportation Commission are now releasing monthly "stop observation" datasets derived from the Regional GTFS Realtime Feed.]]></description><link>https://www.interline.io/blog/mtc-regional-stop-observations/</link><guid isPermaLink="false">672554e4b62841000136ed97</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Fri, 15 Jul 2022 22:25:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/trip-update-selection.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/trip-update-selection.png" alt="Observed stop arrivals and departures through the SF Bay Area"><p>When did the bus arrived at that stop yesterday? This isn’t a question that transit riders ask often — but it is a question that analysts and planners often do ask.</p><p>Interline and the Metropolitan Transportation Commission have been archiving and processing the <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">Regional GTFS Realtime</a> feed in order to answer these questions.</p><h2 id="what-are-stop-observations">What are stop observations?</h2><p>“Stop observations” describes transit service as was delivered to riders: the arrival time of a vehicle at a designated stop, along with any additional contextual information available from GTFS and GTFS Realtime feeds.</p><p>Stop observations are in contrast with the <code>stop_times.txt</code> file in the <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/">daily Regional Feed</a>, which uses static GTFS data to capture transit service as it is planned and scheduled. This is also in contrast with the <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#historical-feeds">monthly Historic Regional Feed</a>, which uses static GTFS data to capture transit service schedules, retrospectively, as they changed over an entire month (with a different schedule for each agency for each day).</p><p>Stop observations can be used to better understand the real-world realities of transit service as it fluctuates from day to day and location to location. Stop observations can be used to estimate on-time performance (also known as schedule adherence or punctuality) and to inform potential schedule adjustments by service planners. Stop observations can also be used by navigation app developers to provide their users with contextual information, such as an estimated range of times when a route is likely to depart from a given stop.</p><h2 id="measurement-error-and-imprecision">Measurement error and (im)precision</h2><p>Stop observations are inferred from public GTFS Realtime data feeds (trip updates and vehicle positions) and therefore, there are many potential sources for measurement error. Potential sources of error include GPS error on transit vehicles, misassignment of vehicles to “runs” of a given trip and route, imprecision or data errors in the transit agency’s computer-aided dispatch/automatic vehicle location (CAD/AVL) system, issues in the transit agency’s GTFS Realtime emitting system, issues in 511.org’s GTFS Realtime aggregation system, or issues in the <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">Regional Realtime GTFS Feed</a>.</p><p>We’ll discuss how stop observations are calculated later in this blog post, to help users better understand stop observations as measurements. When considering potential applications for stop observations, we ask users to keep the limitations of this data source in mind.</p><h2 id="what-information-is-contained-in-a-stop-observation">What information is contained in a stop observation?</h2><p>We are distributing stop observations in the Historic Regional Feed. When users request a Historic Regional Feed with stop observations for a given year and month, a <code>stop_observations.txt</code> file will be included with all stop observations for that month.</p><p>The <code>stop_observations.txt</code> file is a CSV file based on the <a href="https://bit.ly/gtfs-performance">GTFS-Performance draft proposal</a>. GTFS-Performance is an extension to the GTFS specification prepared by Remix, Swiftly, and their partners.</p><p>The following table is based on the GTFS-Performance draft proposal and covers columns included in our implementation of <code>stops_observations.txt</code>, as well as columns that are in the proposal but not in our implementation:</p><table><thead><tr><th>Field Name</th><th>Details</th><th>Regional GTFS stop observations support</th></tr></thead><tbody><tr><td><code>trip_id</code></td><td><p>Identifies a trip.</p><p> When <code>schedule_relationship=DUPLICATED</code>, then this references the trip that is copied, like GTFS Realtime <code>TripUpdate.trip.trip_id</code>. </p></td><td><p>✅</p></td></tr><tr><td><code>trip_start_time</code></td><td><p> See GTFS Realtime <code>TripDescriptor.start_time</code>. </p><p>Required to specify the start time of a trip that’s not defined in Static GTFS which occurs when using frequency-based service defined by <code>frequencies.txt</code> or when an ad-hoc added service based on an existing trip via <code>TripUpdate.tripschedule_relationship=DUPLICATED</code>.</p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>stop_id</code></td><td><p>Identifies a stop</p></td><td>✅ (added by Interline for developer convenience)</td></tr><tr><td><code>stop_sequence</code></td><td><p>Identifies the order of stops for a particular trip.</p></td><td><p>✅</p></td></tr><tr><td><code>schedule_relationship</code></td><td><p>Valid options are:</p><ul><li> 0 - SCHEDULED </li><li> 1 - SKIPPED </li><li> 2 - NO_DATA </li><li> 3 - UNSCHEDULED </li><li> 4 - CANCELED </li><li> 5 - DUPLICATED </li><li>6 - MODIFIED</li></ul></td><td><p>✅</p></td></tr><tr><td><code>schedule_adherence_stop</code></td><td><p>Indicates if this observation is used for reporting purposes when a subset of stops are utilized, instead of all stops.</p></td><td><p>No</p></td></tr><tr><td><code>trip_start_date</code></td><td><p>The start date of this trip instance in <code>YYYYMMDD</code> format to specify the service date of the corresponding trip. This should match the value in <code>TripDescriptor.start_date</code>.</p></td><td><p>✅</p></td></tr><tr><td><code>vehicle_id</code></td><td><p> Unique identifier for the vehicle that served this stop. This should match the value provided in real-time via <code>VehicleDescriptor.id</code>. </p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>scheduled_arrival_time</code></td><td><p> GTFS static scheduled arrival time </p></td><td><p>✅ (added by Interline for developer convenience)</p></td></tr><tr><td><code>observed_arrival_time</code></td><td><p>Observed arrival time at a specific stop for a specific trip on a route. For times occurring after midnight on the service day, enter the time as a value greater than <code>24:00:00</code> in <code>HH:MM:SS</code> local time for the day on which the trip schedule begins.</p></td><td><p>✅</p></td></tr><tr><td><code>observed_departure_time</code></td><td><p>Observed departure time at a specific stop for a specific trip on a route. For times occurring after midnight on the service day, enter the time as a value greater than <code>24:00:00</code> in <code>HH:MM:SS</code> local time for the day on which the trip schedule begins.</p></td><td><p>✅ (when available)</p></td></tr><tr><td><code>new_departure_time</code></td><td><p>Required when <code>schedule_relationship=DEPARTURE_TIME_MODIFIED</code>.</p></td><td><p>No</p></td></tr><tr><td><code>dwell_time_secs</code></td><td><p>Seconds spend boarding/alighting riders at the stop.</p></td><td><p>No</p></td></tr><tr><td><code>scheduled_dwell_time_secs</code></td><td><p>Seconds scheduled for boarding/alighting riders at the stop (when provided in agency source RT feed).</p></td><td><p>✅</p></td></tr><tr><td><code>boardings</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s board_alight.boardings</p></td><td><p>No</p></td></tr><tr><td><code>alightings</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s board_alight.alightings</p></td><td><p>No</p></td></tr><tr><td><code>source</code></td><td><p>Valid options are:</p><ul><li>0 - Manual</li><li>1 - Door Sensor</li><li>2 - Estimated using GPS/AVL data</li></ul></td><td><p>No</p></td></tr><tr><td><code>uncertainty</code></td><td><p>See <code>StopTimeEvent.uncertainty</code>.</p></td><td><p>✅</p></td></tr><tr><td><code>occupancy_status</code></td><td><p>See <code>VehiclePosition.occupancy_status</code>.</p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>occupancy_percentage</code></td><td><p>See <code>VehiclePosition.occupancy_percentage</code>.</p></td><td><p>✅ (when provided in agency source RT feed)</p></td></tr><tr><td><code>occupancy_type</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s <code>board_alight.load_type</code>.</p></td><td><p>No</p></td></tr><tr><td><code>occupancy_source</code></td><td><p>See <a href="https://gtfsride.org/">GTFS-ride</a>'s <code>board_alight.source</code>.</p></td><td><p>No</p></td></tr></tbody></table><h2 id="how-are-stop-observations-calculated">How are stop observations calculated?</h2><p>Stop observations are derived retrospectively using an archive of the Regional GTFS Realtime Feed (updated every 15 seconds). Once a day, our workflow processes GTFS Realtime trip updates and vehicle positions from the archive, combined together with the day’s static GTFS daily regional feed.</p><p><strong>Deriving stop observations from trip updates:</strong> When an agency supplies GTFS Realtime trip updates, our workflow identifies the final <code>TripUpdate</code> sent before a vehicle reaches a given stop. From that record, our workflow extracts the most recent <code>StopTimeUpdate</code>. Here is an example showing three trip updates fetched from BART. The first two trip updates are before the train reaches Montgomery Station; the third trip update is after the train departs Montgomery Station. The portion of all of the records that is use to create a stop observation is marked with the orange box:</p><figure><img src="https://www.interline.io/content/images/2024/11/trip-update-selection-1.png" alt="illustration of three GTFS Realtime feed trip updates, showing the portion of stop time updates retained to create a stop observation record" width="1167" height="943"></figure><p><strong>Deriving stop observations from vehicle positions</strong>: When an agency supplies vehicle positions for a given route, our workflow also identifies the last update sent before the stop location. To do this, we need to consider the geometry of the route alignment from <code>shapes.txt</code> in the static RG feed. The workflow snaps the latitude/longitude point from each vehicle position to the route alignment, enabling it to determine from which update to take the <code>StopTimeUpdate</code> portion. Like many geometric algorithms, this approach can be stymied by complicated paths, such as looping routes that visit the same station multiple times. Our workflow can take advantage of stop sequence, when specified. Still, the algorithm can only do its best to estimate when the route alignment is complicated and/or the update frequency is infrequent.</p><p>By being able to derive stop observations from two different parts of GTFS Realtime feeds, we have also been able to test and tune our workflow, to ensure that the two approaches agree when source feeds are of high quality.</p><h2 id="related-data-specifications">Related data specifications</h2><p>Our stop observations data model is based on the <a href="http://bit.ly/gtfs-performance">GTFS-Performance</a> proposal. The GTFS-Performance proposal is, in turn, inspired by the larger <a href="https://apps.trb.org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=4687">TCRP G-18</a> project. The GTFS-Performance <code>stop_visits.txt</code> is supposed to represent a subset of the TCRP G-18 <code>stop_visits.txt</code> file. Thank you to the creators of GTFS-Performance and to the G-18 research team.</p><p>GTFS-Performance and TCRP G-18 go well beyond the functionality that we have implemented in our system for producing stop observations, to also address more internal operating concerns of transit agencies. We have intentionally implemented our system to create stop observations using only publicly available GTFS and GTFS Realtime inputs.</p><h2 id="accessing-stop-observations">Accessing stop observations</h2><p>Stop observations are now included in historical monthly RGs. To download a historical monthly RG including stop observations, make an HTTP request like so:</p><pre><code>wget https://api.511.org/transit/datafeeds?operator_id=RG&amp;historic=2022-03-so&amp;api_key=xxx
</code></pre><p>Replace <code>2022-03</code> with a given year and month. Note the addition of <code>-so</code> to request the inclusion of stop observations. Note that <code>wget</code> is a Unix command that can be replaced by any other utility to download the URL; also note that <code>xxx</code> should be replaced by your own <a href="https://511.org/open-data/token">511.org Open Data API key</a>.</p><p>Monthly RGs without stop observations are roughly 50Mb in size; with stop observations, the monthly RG will be roughly 500Mb to download as a compressed ZIP archive and then roughly 2Gb of uncompressed CSV files.</p><p>Stop observations are available on a more frequent basis to staff at MTC and Bay Area transit agencies. Transit agencies may contact the 511 SF Bay Data Portal at <a href="mailto:transitdata@511.org">transitdata@511.org</a> for more information.</p><h2 id="using-stop-observations-to-understand-transit">Using stop observations to understand transit</h2><p>Public-transit agencies can be hesitant to release this type of data. In part, it’s a challenge to assemble operations data in an ongoing and consistent manner. In part, it’s a risk that the general public will take operations data out of context.</p><p>While we began this blog post by asking <em>When did the bus arrived at that stop yesterday?</em>, we hope that users of stop observations will ask an even wider range of questions of this dataset, especially questions that can explore the complexity of daily operations for over 20 agencies operating buses, trains, subways, and ferries across the Bay Area and questions that can help to improve the experience for riders using navigation apps and similar software.</p>]]></content:encoded></item><item><title><![CDATA[Our GTFS Realtime validation report published by the Transportation Research Board]]></title><description><![CDATA[In 2019 and 2020, Interline and the Center for Urban Transportation Research at University of South Florida collaborated on a Transit IDEA Program grant from the US Transportation Research Board. Our goal has been to help transit agencies to validate and improve the quality of their GTFS Realtime feeds. This first grant is now complete and TRB has published the full report with our findings and proposed next steps.]]></description><link>https://www.interline.io/blog/trb-transit-idea-gtfs-realtime-report/</link><guid isPermaLink="false">67255750b62841000136ede8</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Tue, 19 Jan 2021 08:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/trb-t-93-report-cover.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/trb-t-93-report-cover.png" alt="Our GTFS Realtime validation report published by the Transportation Research Board"><p>In 2019 and 2020, Interline and the <a href="https://www.cutr.usf.edu/">Center for Urban Transportation Research</a> at University of South Florida collaborated on a <a href="http://apps.trb.org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=4695">Transit IDEA Program grant</a> from the US Transportation Research Board. Our goal has been to help transit agencies to validate and improve the quality of their GTFS Realtime feeds. This first grant is now complete and TRB has published <a href="http://www.trb.org/Main/Blurbs/181415.aspx">the full report with our findings and proposed next steps</a>.</p><figure><img src="https://www.interline.io/content/images/2024/11/trb-t-93-report-cover-1.png" alt="TRB Transit IDEA T-93 report cover" width="1224" height="1584"><figcaption><p>The full report can be downloaded from the <a href="http://www.trb.org/Main/Blurbs/181415.aspx">TRB website</a>.</p></figcaption></figure><h2 id="executive-summary">Executive Summary</h2><p>Real-time transit information has been shown to have many benefits for transit riders and agencies, including shorter perceived and actual wait times, a more welcoming experience for new riders, and an increased feeling of safety, and increased ridership. Real-time transit data is, in comparison with many other potential operational or capital improvements to bus or rail service, an affordable means of increasing ridership. In the last few years, a real-time complement to the General Transit Feed Specification (GTFS) format, GTFS Realtime, has emerged, which enables transit agencies to share real-time predictions, vehicle positions, and service alert data in a standardized format. Despite its promise, adoption of GTFS Realtime by transit agencies has been hampered by readily available validation tools.</p><figure><img src="https://www.interline.io/content/images/2024/11/gtfs-realtime-architecture-diagram.png" alt="technical architecture diagram showing GTFS Realtime feeds connecting transit agency data systems to consuming applications" width="836" height="842"><figcaption><p>GTFS Realtime feeds (the blueish box on the right) are a critical connection between the complexity of transit agencies' data systems and consuming applications, like smartphone navigation apps.</p></figcaption></figure><p>In this project, Interline Technologies LLC and the Center for Urban Transportation Research (CUTR) at the University of South Florida (USF) have created a prototype platform that makes GTFS Realtime validation tools readily available to, potentially, all transit agencies in North America. Our team is building upon two open-source projects: the <a href="https://github.com/CUTR-at-USF/gtfs-realtime-validator/">GTFS Realtime validator prototype</a> and <a href="https://www.transit.land/">Transitland</a>, an open transit data platform. This project applies the open-source and open-data community models to the challenges of creating and improving GTFS Realtime data.</p><p><strong>Stage 1: Build and Test GTFS Realtime Data Platform</strong>: The research team combined the Transitland open data platform, the GTFS Realtime Validator, and a list of 162 GTFS Realtime feed endpoints (provided by a partner organization). The combined platform collects GTFS Realtime data from each feed, runs the validator process, and produces a report on any detected errors. Each report shows the counts of data entities, the percentage with errors, and a brief text description of any errors. Links take users to additional documentation about each error type. Some errors also provide further contextual information in maps and tables to assist users as they try to determine root causes.</p><figure><img src="https://www.interline.io/content/images/2024/11/transitland-validation-report-screenshots.png" alt="screenshots of the Transitland GTFS Realtime validation reports" width="2000" height="902"><figcaption><p>Screenshots of example GTFS Realtime validation reports. <em>Left</em>: Listing errors (with red severity labels) and a warning (with a yellow severity label) detected in 24 hours of vehicle-position updates. <em>Right</em>: Viewing a specific issue to understand why a vehicle (the little red dot) is far from its assigned route (the blue line).</p></figcaption></figure><p><strong>Stage 2: Testing and Expanding Catalogue</strong>: The project team has tested the platform by preparing validation reports for seven public-transit agencies and reviewing the results in the platform user interface with agency staff members over video calls. In these user-testing sessions, the project team collected information from agency staff about how GTFS and GTFS Realtime data are currently created at each agency, known issues, and any open goals. After being given a tour through the platform and its interface, agency staff reviewed the reports for their own GTFS Realtime feeds. Agency staff were asked to provide input on both the specific quality checks and the overall presentation and approach used by the platform with a standardized question list. The research team has summarized notes from the user-testing sessions and used these results to inform preliminary plans for expanding the platform for a wider range of users in the future.</p><p><strong>Product Pay-Off Potential</strong>: By combining the open-source components of a GTFS Realtime validator with a catalog of GTFS Realtime feeds, hosted on Transitland’s cloud servers, this project will make the process of validating real-time data simple and accessible to agency staff from any computer with a web browser. As a result, GTFS Realtime data will improve in quality and availability. Transit riders will have a better experience (which has been linked to higher ridership), agency staff will provide better service with less effort and cost, and system vendors will provide a higher quality product.</p><p><strong>Product Transfer</strong>: To succeed, this platform will need to be usable by agency staff and to provide them with results that they can act upon, both within the context of their agencies and with their vendors. Therefore, the project team has involved agency stakeholders early and often in the project. The team recruited nine transit agencies to write letters of support as part of the Transit IDEA proposal, and seven agencies have participated in our user-testing process. These agencies represent a diverse range of rider population sizes, staff skill level, and location (urban and rural). The final report includes information on immediate plans to continue and expand Transitland’s catalog GTFS Realtime feeds and longer term plans to develop a GTFS Realtime certification process supported by the platform.</p><h2 id="next-steps">Next Steps</h2><p>We invite everyone to browse Transitland’s <a href="https://www.transit.land/feeds/">list of GTFS Realtime feeds</a>. Please help us to maintain this list by contributing additions and edits to the <a href="https://github.com/transitland/transitland-atlas">open Transitland Atlas feed registry</a>.</p><p>Do you work for a transit agency that publishes GTFS Realtime data? Please check to make sure it’s on Transitland. If not, please add it to the <a href="https://github.com/transitland/transitland-atlas">open Transitland Atlas feed registry</a> or <a href="mailto:info@interline.io">contact us for assistance</a>.</p><p>Do you work for a transit agency that has issues with GTFS Realtime data, or do you represent a group with a stake in the future of GTFS Realtime data? We’re looking to collaborate on opening this GTFS Realtime validation platform for self-serve use by agencies. <a href="mailto:info@interline.io">We welcome your interest</a>.</p><h2 id="acknowledgements">Acknowledgements</h2><p>Research team members include:</p><ul><li><a href="https://www.linkedin.com/in/irees">Ian Rees</a>, Interline, Principal</li><li><a href="https://www.linkedin.com/in/seanbarbeau/">Sean Barbeau</a>, Center for Urban Transportation Research (CUTR), University of South Florida (USF), Principal Mobile Software Architect for R&amp;D</li></ul><p>Thanks to members of the T-93 expert review panel (ERP):</p><ul><li>Patricia Collette</li><li>Stephen M. Stark</li><li>Aaron Antrim</li><li>Carol Schweiger</li><li>Murat Omay</li><li>Santosh Mishra</li></ul><p>Many thanks to Velvet Basemera-Fitzpatrick, Ph.D., Transit IDEA Program Manager.</p><p>Thanks to the following transit agencies and staff members for participating in user testing:</p><ul><li>Metropolitan Transportation Commission (San Francisco): Nisar Ahmed</li><li>Metro Transit (Minneapolis): Laura Matson, Mark DiPasquale, Gary Nyberg, Joey Reid, Juan Villanueva</li><li>Massachusetts Bay Transportation Authority (Boston): Paul Swartz, Jessie, Richards, Logan Nash</li><li>TriMet (Portland, Oregon): Mike Gilligan, Guy Tinat</li><li>Tompkins Consolidated Area Transit (Ithaca, New York): Tom Clavel, Matthew Yarrow</li><li>Washington Metropolitan Area Transit Authority (Washington, D.C.): Stephanie Jones, Richard Carman</li><li>Hillsborough Area Regional Transit (Tampa, Florida): Dexter Corbin, William Mozel, Tim Wictor</li><li>Rogue Valley Transportation District (Medford, Oregon): Melissa Lowry</li></ul>]]></content:encoded></item><item><title><![CDATA[SF Bay Area's Regional GTFS Feed Expanded]]></title><link>https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/</link><guid isPermaLink="false">672547ebb62841000136ed0a</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Mon, 10 Aug 2020 07:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/headway-july-only.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/headway-july-only.png" alt="SF Bay Area's Regional GTFS Feed Expanded"><blockquote><p><strong>October 4, 2022</strong>: MTC and Interline have released updated fares and transfer discounts for all Bay Area transit agencies using the adopted GTFS-Fares v2 &quot;base implementation.&quot; See <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-fares-updates/">this blog post for more information</a>.</p></blockquote><blockquote><p><strong>September 10, 2021</strong>: MTC and Interline have now released the Regional GTFS <em>Realtime</em> Feed, which provides up-to-the-minute service alerts, trip updates, and vehicle positions using the same ID scheme as the static Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">this blog post for an overview</a>.</p></blockquote><p>Earlier this year, <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/">Interline and the Metropolitan Transportation Commission released the Regional GTFS Feed for the San Francisco Bay Area</a>. The Regional GTFS Feed is produced on a daily basis and made available through <a href="https://511.org/open-data">511 SF Bay’s Open Data Portal</a> and its Open Transit Data API.</p><p>We’re pleased to now share a series of additions to the Regional GTFS Feed that we’ve released together over recent months:</p><ul><li><a href="#historical-feeds"><strong>Historical Feeds</strong></a> which provide a retrospective look at an entire month of service</li><li><a href="#station-pathways-and-levels"><strong>Station Pathways and Levels</strong></a> to provide richer wayfinding for all types of riders</li><li><a href="#fares-and-transfer-discounts"><strong>Fares and Transfer Discounts</strong></a> [beta] to calculate the cost of transit journeys on and across the eight largest agencies</li></ul><h2 id="historical-feeds">Historical Feeds</h2><p>Since we released the Regional GTFS Feed in January, much has changed across the Bay Area. Transit agencies and their dedicated staff have rapidly reduced and re-targeted transit service for those riders who perform essential work or depend upon transit for their necessary journeys.</p><p>Using the newly released Historical Feeds component of the Regional GTFS Feed, we can visualize how transit service has changed throughout the Bay Area from January to July of 2020:</p><figure><img src="https://www.interline.io/content/images/2024/11/headway-animation.jpg" alt="" width="2000" height="1582"><figcaption><p>Buses, trains, and ferries throughout the San Francisco Bay Area color-coded by frequency: the reddest route lines are routes that operate at least every 10 minutes; medium hues of red are 10 - 20 minute headways; the lightest hue of red are routes with service less than once every 20 minutes. This animation is created by using the Historical Feeds for the months of January through July of 2020.</p></figcaption></figure><p>Historical Regional Feed products are fully valid GTFS feeds, but they differ somewhat in their contents from the daily Regional Feed products. Read on to understand the process used to produce the Historical Regional Feed products and their key differences, or skip to the end of this section to <a href="#download-historical-feed-products">download Historical Feed products</a>.</p><h3 id="slicing-regional-feeds">Slicing regional feeds</h3><p>Each day, the Regional Feed is produced from the versions of agency feeds on 511.org that provide the best view of service on that day. Each month, the Historical Regional Feed creation process takes these Regional Feeds and combines them together, taking one day of service from each feed, which we are calling a “slice.”</p><p>For example:</p><table><thead><tr><th>Feed filename</th><th>Published</th><th>Contributes service slice for</th></tr></thead><tbody><tr><td>mtc-regional-feed-2020-04-24.zip</td><td>2020-04-24</td><td>2020-04-24</td></tr><tr><td>mtc-regional-feed-2020-04-23.zip</td><td>2020-04-23</td><td>2020-04-23</td></tr><tr><td>mtc-regional-feed-2020-04-22.zip</td><td>2020-04-22</td><td>2020-04-22</td></tr><tr><td>mtc-regional-feed-2020-04-21.zip</td><td>2020-04-21</td><td>2020-04-21</td></tr><tr><td>mtc-regional-feed-2020-04-20.zip</td><td>2020-04-20</td><td>2020-04-20</td></tr></tbody></table><p>If the Regional Feed for a given day is missing, the closest previous day provides service. For instance, if 2020-04-22 was missing, the 2020-04-21 feed slice would cover both 2020-04-21 and 2020-04-22.</p><h3 id="global-entity-copying">Global entity copying</h3><p>Agencies, stops, and routes are considered “global”, and are handled using a simple ID-based merge with the most recent version winning. For example, if BART has a route with ID “OR-S” that is called “Richmond - Warm Springs”, but then later renames it to “Richmond to Warm Springs”, then the latter version will be used.</p><h3 id="trip-hashing-comparison-and-copying">Trip hashing, comparison, and copying</h3><p>Trips are more complicated and handled separately. A simple combining of all the trips and stop_times in all of the input files can easily create a GTFS feed that is too large for practical use, especially given that programs like <a href="https://www.interline.io/routing-platform/">OpenTripPlanner</a> need to hold the entire schedule in memory. Therefore, duplicate copies of trips are detected using a hash based approach and only copied to the output once. This reduces the output size by approximately 90%.</p><p>For example, here are three hypothetical versions of Trip ID “BA:2210503” from three consecutive days of input regional feeds.</p><table><thead><tr><th>Feed filename</th><th>Trip ID</th><th>Route ID</th><th>Service ID</th><th>Headsign</th><th>1st stop</th><th>2nd stop</th><th>n stops</th><th>Hash</th></tr></thead><tbody><tr><td><code>2020-04-24.zip</code></td><td><code>BA:2210503</code></td><td><code>BA:OR-S</code></td><td><code>BA:Wkd_BASE-Weekday-07</code></td><td>Warm Springs/South Fremont</td><td><code>RICH 5:03</code></td><td><code>DELN 5:07</code></td><td><code>PLZA 5:10...</code></td><td><code>8c4ecb</code></td></tr><tr><td><code>2020-04-23.zip</code></td><td><code>BA:2210503</code></td><td><code>BA:OR-S</code></td><td><code>BA:Wkd_BASE-Weekday-07</code></td><td>Warm Springs/South Fremont</td><td><code>RICH 5:03</code></td><td><code>DELN 5:07</code></td><td><code>PLZA 5:10...</code></td><td><code>8c4ecb</code></td></tr><tr><td><code>2020-04-22.zip</code></td><td><code>BA:2210503</code></td><td><code>BA:OR-S</code></td><td><code>BA:Wkd_BASE-Weekday-07</code></td><td>Warm Springs to South Fremont</td><td><code>RICH 5:04</code></td><td><code>DELN 5:06</code></td><td><code>PLZA 5:12...</code></td><td><code>a4bf1a</code></td></tr></tbody></table><p>For each of these, the hashing function takes into account all trip attributes, all the calendar attributes for that trip, and the full details of each entry in <code>stop_times.txt</code>. Any change in any field will result in a different hash. This allows us to directly compare trips between versions of the input feed. Above, all details and schedule for 2020-04-23 and 2020-04-24 match exactly, so these trips two will be considered identical. The trip for 2020-04-22 has some minor differences in name and schedule, so will generate a different hash, and be considered a different trip.</p><p>As the historical feed merging program processes each input feed, it calculates the hash of each trip in the feed. If it has not seen a trip before, it copies it to the output and notes the hash for future use. If it has been seen before, it is not copied again. To prevent clashes, the original Trip IDs are appended with the trip hash (e.g. <code>BA:2210503</code> -&gt; <code>BA:2210503:8c4ecb</code>). The merging program then takes all trips in the input feed (both seen and unseen) and examines the calendars to see which are active for each day in this slice, and then creates <code>calendar_dates.txt</code> entries for each trip on each day where that trip is scheduled to run. The original service IDs are changed to be the same as the hash appended Trip ID, and the calendar is unrolled into a day-by-day format, but it works reliably. This hashing approach is resource efficient and allows us to create historical feeds of arbitrary duration while minimizing the output size.</p><p>Example output <code>calendar_dates.txt</code>:</p><table><thead><tr><th>Service ID</th><th>Date</th><th>Exception Type</th></tr></thead><tbody><tr><td><code>BA:2210503:8c4ecb</code></td><td>2020-04-24</td><td>1 (Added)</td></tr><tr><td><code>BA:2210503:8c4ecb</code></td><td>2020-04-23</td><td>1 (Added)</td></tr><tr><td><code>BA:2210503:a4bf1a</code></td><td>2020-04-22</td><td>1 (Added)</td></tr></tbody></table><p>In this way, the <code>8c4ecb</code> version of the trip is scheduled to run on the two days of input data where it was seen, and the <code>a4bf1a</code> version is scheduled to run on the other day.</p><h3 id="differences-between-regional-and-historic-feeds">Differences between Regional and Historic feeds</h3><p>Historic Regional Feeds are equivalent to the original daily Regional Feeds in the stops, routes, and scheduled services they contain. Using a Historic will produce the same output in a routing engine or another type of analysis.</p><p>Historic Feeds are different from Regional Feeds in their specific GTFS structure:</p><ul><li><code>calendars.txt</code> records are removed and rewritten in <code>calendar_dates.txt</code></li><li><code>trips.txt</code> records are hashed and compared (as described above)</li><li>IDs for global records are namespaced (as described above)</li></ul><p>These differences should not affect routing engine or similar types of analysis. However, keep these differences in mind if you are trying to use historical feeds to understand changes in GTFS data and its practices over time at Bay Area agencies.</p><h3 id="download-historical-feed-products">Download Historical Feed products</h3><p>To use the Historical Feed products:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG&amp;historic=YYYY-MM</code> (for example, to request May 2020: <code>historic=2020-05</code>)</li><li>At the start of each month, the Historic Feed for the last month is created and posted. You may download as many months as you wish and combine them to analyze as many months/quarters/years as you wish at once.</li></ol><p>As of this blog post, the following months are available for download (that is, you can you any of these values for the <code>historic</code> query parameter):</p><ul><li><code>2020-01</code></li><li><code>2020-02</code></li><li><code>2020-03</code></li><li><code>2020-04</code></li><li><code>2020-05</code></li><li><code>2020-06</code></li><li><code>2020-07</code></li></ul><h2 id="station-pathways-and-levels">Station Pathways and Levels</h2><p>The Regional Feed exists to both merge together individual agency GTFS feeds and to serve as a home for new GTFS data that describes cross-agency conditions. The daily Regional Feed now comes with layouts for 35 key transit stations across the Bay Area, each of which serves multiple agencies. Using the newly added <code>pathways.txt</code> and <code>levels.txt</code> files in GTFS, Interline and MTC are now able to provide more detailed information about how to transfer from one agency to another. This includes information on elevators, escalators, and routes that may not be accessible to those in wheelchairs. Pathways and levels information will equip trip planning apps to provide more helpful wayfinding information to their users, particular those who are new to the Bay Area or who have limited vision.</p><figure><img src="https://www.interline.io/content/images/2024/11/university-ave-pathways.jpg" alt="" width="2000" height="948"><figcaption><p>Pathways and levels throughout the University Ave/Downtown Palo Alto station served by Caltrain regional rail on two platforms and VTA and SamTrans buses at bus bays.</p></figcaption></figure><h3 id="_35-regional-transit-hubs">35 Regional Transit Hubs</h3><p>Here is a list of the 35 stations currently in the Regional Feed:</p><table><thead><tr><th>Transit station/hub</th><th>County</th></tr></thead><tbody><tr><td>Embarcadero BART</td><td>San Francisco</td></tr><tr><td>Montgomery BART</td><td>San Francisco</td></tr><tr><td>Caltrain Station 4th &amp; King</td><td>San Francisco</td></tr><tr><td>Salesforce Transit Center</td><td>San Francisco</td></tr><tr><td>12th St Oakland City Center BART</td><td>Alameda</td></tr><tr><td>El Cerrito Del Norte BART</td><td>Contra Costa</td></tr><tr><td>19TH St Oakland BART</td><td>Alameda</td></tr><tr><td>Powell ST BART</td><td>San Francisco</td></tr><tr><td>Civic Center BART</td><td>San Francisco</td></tr><tr><td>Walnut Creek BART</td><td>Contra Costa</td></tr><tr><td>Richmond BART/Amtrak</td><td>Contra Costa</td></tr><tr><td>San Jose Diridon Station</td><td>Santa Clara</td></tr><tr><td>Palo Alto Station</td><td>Santa Clara</td></tr><tr><td>San Rafael Transit Center</td><td>Marin</td></tr><tr><td>Millbrae BART</td><td>San Mateo</td></tr><tr><td>Pleasant Hill BART</td><td>Contra Costa</td></tr><tr><td>San Francisco Ferry Terminal</td><td>San Francisco</td></tr><tr><td>Daly City BART</td><td>San Francisco</td></tr><tr><td>Santa Rosa Transit Mall</td><td>Sonoma</td></tr><tr><td>Union City BART</td><td>Alameda</td></tr><tr><td>MacArthur BART</td><td>Alameda</td></tr><tr><td>Dublin/ Pleasanton BART</td><td>Alameda</td></tr><tr><td>Warm Springs/ South Fremont BART</td><td>Alameda</td></tr><tr><td>Santa Clara Caltrain</td><td>Santa Clara</td></tr><tr><td>Oakland Coliseum BART</td><td>Alameda</td></tr><tr><td>SFO</td><td>San Francisco</td></tr><tr><td>Fairfield Transportation Center</td><td>Solano</td></tr><tr><td>OAK</td><td>Alameda</td></tr><tr><td>Petaluma Transit Mall</td><td>Marin</td></tr><tr><td>Vallejo Ferry Terminal</td><td>Solano</td></tr><tr><td>Mountain View Station</td><td>Santa Clara</td></tr><tr><td>Great America</td><td>Santa Clara</td></tr><tr><td>Napa Intermodal</td><td>Napa</td></tr><tr><td>SJC</td><td>Santa Clara</td></tr><tr><td>Great Mall/Milpitas BART</td><td>Santa Clara</td></tr></tbody></table><p>Note that just as the ongoing pandemic has made our work on the Historic Feed all the more relevant, our work on Station Pathways and Levels has had to adapt to current circumstances. Our staff have been working from home using aerial imagery, architectural drawings, station maps, and other materials. Interline’s Station Editor tool also works on tablets, and we look forward to again going out into the field to all of these stations to correct and improve details.</p><p>We welcome questions and corrections sent to the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources mailing list</a>.</p><h3 id="using-station-pathways-and-levels">Using Station Pathways and Levels</h3><p>Download the daily Regional Feed like so:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li><li>Look for the <code>pathways.txt</code> and <code>levels.txt</code> files inside the zip archive.</li><li>For more information, see the <a href="https://gtfs.org/reference/static#pathwaystxt">static GTFS documentation</a> and the <a href="https://bit.ly/gtfs-pathways">GTFS-Pathways extension proposal document</a>.</li></ol><h2 id="fares-and-transfer-discounts">Fares and Transfer Discounts</h2><p>To date, the Regional GTFS Feed has focused on how to plan a journey by transit. Now we’re curating and adding additional data to help riders understand how to pay for their journeys. The Bay Area’s Regional GTFS Feed is now the first in the world to include data using the GTFS Fares-v2 specification. This <a href="https://bit.ly/gtfs-fares">newly proposed specification</a> allows us to model the many different fare products and discounts that are available to the Bay Area’s transit riders. For example, we can now account for how some agencies provide riders a discount when they pay by Clipper Card rather than by cash. (<a href="https://www.clippercard.com/">Clipper Card</a> is the contact-less payment card and system operated by MTC and available on buses, trains, and ferries throughout the Bay Area.) We can also capture how riders can receive a discount when transferring from certain agencies to other agencies. Read on for a detailed example of how Fares-v2 work in practice, or skip ahead to <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/#using-fares-and-transfer-discounts">use the Fares-v2 beta data</a>.</p><h3 id="an-example-of-fares-v2-in-practice">An Example of Fares-v2 in Practice</h3><figure><img src="https://www.interline.io/content/images/2024/11/bart-to-ac-transit-route.png" alt="map of a journey by BART and AC Transit originating in San Francisco, transferring in Oakland, and ending in Alameda" width="1606" height="1042"><figcaption><p>A journey by BART and AC Transit, from San Francisco to Oakland to Alameda, which we will use in the following example of how to calculate the total cost. (Credit: BART Trip Planner)</p></figcaption></figure><p>Fares-v2 expands the traditional route- and zone-based GTFS fares model with several additional files, each focused on modeling a different part of a complex fare scheme. Imagine a rider making a transfer from BART to AC Transit; both BART and AC Transit have different prices for adult fares, eligible discount fares, and different prices when paying with cash or when using a Clipper Card; additionally, the discount applied when transferring from BART to AC Transit has different values and rules when using Clipper. Calculating the individual fares for each leg of this trip requires a description of any zones that apply to routes and stops (<code>fare_networks.txt</code> and <code>fare_areas.txt</code>), the costs and requirements of the base fare for each leg (<code>fare_leg_rules.txt</code>), any discount categories that might include the rider (<code>fare_profiles.txt</code>), and the type of payment used by the rider (<code>fare_containers.txt</code>). All of these factors are considered and then used to search for any transfer discounts that may apply for the trip (<code>fare_transfer_rules.txt</code>).</p><p>Let’s work through the hypothetical trip above, first taking BART from Embarcadero Station to 12th St. in Oakland, and then riding an AC Local bus, 51A, to Alameda.</p><p>The following is an except from <code>fare_leg_rules.txt</code> with some columns removed for brevity. This file contains sets of rules that are matched against a single leg of a trip, without regard to transfers. The columns <code>from_area_id</code>, <code>to_area_id</code>, <code>fare_container_id</code> and <code>fare_category_id</code> references values defined in the other files mentioned above.</p><table><thead><tr><th>order</th><th>leg_group_id</th><th>from_area_id</th><th>to_area_id</th><th>amount</th><th>fare_container_id</th><th>fare_category_id</th><th>notes</th></tr></thead><tbody><tr><td>90</td><td>BA:</td><td>EMBR</td><td>12TH</td><td>1.35</td><td>clipper</td><td>BA:Senior/Disabled Clipper</td><td>Embarcadero to 12th with Clipper and discount</td></tr><tr><td>100</td><td>BA:</td><td>EMBR</td><td>12TH</td><td>3.70</td><td>clipper</td><td></td><td>Embarcadero to 12th with Clipper</td></tr><tr><td>110</td><td>BA:</td><td>EMBR</td><td>12TH</td><td>4.20</td><td></td><td></td><td>Embarcadero to 12th with cash surcharge</td></tr></tbody></table><p>The BART leg would match the first the rules above based on <code>from_area_id</code> and <code>to_area_id</code>, with fares of $1.35, $3.70, and $4.20. However, the first two rules are only available when using a Clipper Card (<code>fare_container_id</code>), and the first of these is only available to riders eligible for a discounted fare (<code>fare_category_id</code>). The <code>order</code> field is used to select the correct fare when multiple rules match the trip leg: any applicable discounted Clipper fare would match first, followed by adult Clipper fare, then finally the fare including additional charge for cash riders. In the event two (or more) rules match with the same order value, both are considered valid options to present to the rider.</p><table><thead><tr><th>order</th><th>leg_group_id</th><th>from_area_id</th><th>to_area_id</th><th>amount</th><th>fare_container_id</th><th>fare_product_id</th><th>fare_category_id</th><th>notes</th></tr></thead><tbody><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>2.25</td><td>clipper</td><td></td><td></td><td>Adult local fare with clipper $2.25</td></tr><tr><td>110</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>2.5</td><td></td><td></td><td></td><td>Adult local fare $2.5</td></tr><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>1.12</td><td></td><td></td><td>AC:senior</td><td>Discounted local fare $1.12 with Clipper</td></tr><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>1.25</td><td></td><td></td><td>AC:senior</td><td>Discounted local fare $1.25</td></tr><tr><td>100</td><td>AC:local</td><td>AC:local</td><td>AC:local</td><td>0</td><td></td><td>AC:local:monthly</td><td></td><td>Free local fare with monthly pass</td></tr></tbody></table><p>The AC Transit fare rules are similar, with the addition of a few extra columns. The 51A is an East Bay local only route (<code>to_area_id</code> is assumed), and matches each rule, and the rules for discounts and cash are applied as above. The <code>fare_product_id</code> field describes additional rules that are available to users of certain prepaid products, such as day passes and monthly passes. In this case, a rider with an Adult Local 31 Day Pass ($84, described in <code>fare_products.txt</code>) would enjoy a free ride. More complex AC Transit fares use additional columns not pictured here (<code>fare_network_id</code>) and handle cases such as riding a Transbay bus and the different prices applied when riding within the East Bay vs. taking the trip all the way to the Salesforce Transit Center in downtown San Francisco.</p><p>Any applicable transfers are calculated by reading the rules in <code>fare_transfer_rules.txt</code> and matching each leg of the trip against each subsequent leg of the trip. This can create quite complicated models, but let’s start by looking at our BART and AC Transit legs. The clever bit is that many <code>fare_leg_rules.txt</code> rules can share the same <code>leg_group_id</code>, which simplifies the number and types of transfer rules that must be defined. For example, all of the BART rules above use <code>BA:</code> and all of the AC Transit rules use <code>AC:local</code>. The table below is an except of <code>fare_transfer_rules.txt</code>.</p><table><thead><tr><th>order</th><th>from_leg_group_id</th><th>to_leg_group_id</th><th>fare_container_id</th><th>amount</th><th>duration_limit</th><th>duration_limit_type</th><th>fare_transfer_type</th><th>spanning_limit</th><th>notes</th></tr></thead><tbody><tr><td>100</td><td>BA:</td><td>AC:local</td><td>clipper</td><td>-0.5</td><td>90</td><td>2</td><td>1</td><td>2</td><td>1 credit of $-0.50 to AC w/in 90 minutes</td></tr><tr><td>110</td><td>BA:</td><td>AC:local</td><td></td><td>-0.25</td><td></td><td></td><td>1</td><td>3</td><td>Cash: 2 fare credits of $0.25 each</td></tr></tbody></table><p>Two different discounts apply when taking an AC Transit leg after a BART leg. When using Clipper, a $0.50 (<code>amount=-0.5</code>, <code>fare_transfer_type=1</code>) discount is applied to the first AC Transit leg (<code>spanning_limit=2</code>, which means it can only match a subjourney of two legs). Alternatively, a paper transfer slip given to cash users contains two tabs, each of which applies a $0.25 discount to the AC Transit cash fare (<code>spanning_limit=3</code>, or a subjourney of up to three legs). Additionally, the Clipper discount expires within 90 minutes after tagging off BART (<code>duration_limit=90</code>, <code>duration_limit_type=2</code>). Because the <code>order</code> value is different, these two transfer options are mutually exclusive to Clipper and cash riders respectively.</p><p>The combination of matching fare rules and fare transfer rules allows describing even complicated transfers, such as discounts to SamTrans local bus riders who hold a Caltrain 2 zone or higher monthly pass, the several categories of upgrade charges when transfering from an AC Transit Local to an AC Transit Transbay bus, and the many varied products and transfers that apply full or partial fare to trips on the Dumbarton Express.</p><h3 id="agencies-in-the-beta-release">Agencies in the Beta Release</h3><p>Note that this is a “beta” release. The GTFS Fares-v2 specification is still being finalized. It does not fully capture some functionality, like how AC Transit and VTA use “fare capping” so that riders never have to pay more in one day than the cost of a day pass. We also expect that as trip planners and other apps begin to consume the Regional Feed’s fare data that we may need to revise some of the data or our schema.</p><p>To our knowledge, Interline is the only organization to have a “rules engine” that can calculate journey costs using GTFS Fares-v2 data. We expect that as other organizations adopt the Interline fares rules engine or build their own equivalents, we may need to revisit some of the assumptions built into the engine.</p><p>The beta release of data includes fares and transfer discounts within and between eight agencies:</p><ul><li>BART</li><li>SFMTA</li><li>AC Transit</li><li>SamTrans</li><li>VTA</li><li>Caltrain</li><li>Golden Gate Transit (bus only; not ferries)</li><li>Dumbarton Express</li></ul><p>This list includes the seven largest agencies and a key connector across San Francisco Bay (Dumbarton Express). It should provide a broad enough sample of fares and transfer discounts to power a wide range of potential applications. This sample is also deep enough to inform analyses of fares, the cost of riding transit, and ideally even ways to reform fares and transfer discounts to produce more equitable outcomes for Bay Area transit riders.</p><h3 id="using-fares-and-transfer-discounts">Using Fares and Transfer Discounts</h3><p>Download the daily Regional Feed like so:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li><li>For more information about the fares and transfer discount files in the GTFS feed, see the <a href="https://bit.ly/gtfs-fares">GTFS-Fares v2 proposal document</a>.</li></ol><p>Interline and MTC expect to improve both the Fares-v2 beta data and schemas as we learn more. We welcome questions and corrections sent to the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources mailing list</a>.</p><h2 id="public-transit-in-2020">Public Transit in 2020</h2><p>A final note that behind every piece of transit data and every multi-modal trip plan are real people who drive vehicles, serve riders, and maintain equipment and facilities. Thank you to all the front-line workers keeping the Bay Area’s public transit systems and riders moving safely!</p><h2 id="acknowledgements">Acknowledgements</h2><p>Credit and many thanks to project team members including Ian Rees and Ruth Miller (Interline), Nisar Kapeel and Kapeel Daryani (MTC), and our partners at Bay Area transit agencies.</p>]]></content:encoded></item><item><title><![CDATA[A Regional GTFS Feed for the San Francisco Bay Area]]></title><description><![CDATA[Last year, Interline began a "startup in residence" at the Metropolitan Transportation Commission. Now we're pleased to publicly release our first creation: the San Francisco Bay Area's Regional GTFS Feed.]]></description><link>https://www.interline.io/blog/mtc-regional-gtfs-feed-release/</link><guid isPermaLink="false">6725531fb62841000136ed64</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Wed, 08 Jan 2020 08:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/mtc-regional-feed-all-routes.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/mtc-regional-feed-all-routes.png" alt="A Regional GTFS Feed for the San Francisco Bay Area"><blockquote><p><strong>September 10, 2021</strong>: MTC and Interline have now released the Regional GTFS <em>Realtime</em> Feed, which provides up-to-the-minute service alerts, trip updates, and vehicle positions using the same ID scheme as the static Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-realtime-feed/">this blog post for an overview</a>.</p></blockquote><blockquote><p><strong>August 10, 2020</strong>: MTC and Interline have released additional functionality in the Regional GTFS Feed. See <a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-additions/">this more recent blog post</a>.</p></blockquote><p>Last year, <a href="https://www.interline.io/blog/metropolitan-transportation-commission-selects-interline/">Interline began a “startup in residence” at the Metropolitan Transportation Commission</a>. Now we’re pleased to publicly release our first creation together: the San Francisco Bay Area’s Regional GTFS Feed.</p><p>Every day, Interline’s workflow processes 31 agency feeds and creates a single unified static GTFS feed.</p><figure><img src="https://www.interline.io/content/images/2024/11/mtc-regional-feed-animation.jpg" alt="" width="1530" height="1990"><figcaption><p>Transit routes around the Bay Area, first by agency feed, then from the aggregated Regional GTFS Feed.</p></figcaption></figure><h2 id="why-a-regional-feed">Why a Regional Feed?</h2><p>People don’t stop their commutes or their travels at transit agency boundaries. Similarly, many applications require combining multiple agency feeds: trip planners, visualizations, data analyses, travel demand models, etc. <strong><em>The Regional GTFS Feed will provide data consumers an easier option.</em></strong> Less effort to ingest multiple feeds, more time toward your actual goals.</p><p><strong><em>The Regional GTFS Feed enables the addition of cross-agency data</em></strong>: fares, transfer discounts, other transfer information, and transit hub layouts and accessibility information. These types of data reference IDs and entities from multiple agencies and are a natural fit within a consolidated feed.</p><p>The workflow that creates the Regional GTFS Feed also identifies conflicts between agency feeds, data entities, and IDs. <strong><em>By validating agency feeds together, the workflow identifies data quality issues that are only apparent once individual feeds are combined.</em></strong> We’re sharing this information with MTC and agency staff to help further improve all feeds.</p><p>Data consumers are still able to access individual feeds from agency websites and <a href="https://511.org/developers/list/apis/">511.org developer APIs</a>. The Regional GTFS Feed is produced using a “conservative” approach: data transformation rules are targeted and constrained. Our motto is <em>Merge. Don’t lose.</em></p><h2 id="future-plans">Future Plans</h2><p>In 2020, we’re continuing to work with MTC to further enrich the Regional GTFS Feed:</p><ul><li>We’re <strong><em>mapping key transit hubs</em></strong> around the Bay Area, creating GTFS-Pathways data using Interline’s new Gotransit editing tools.<a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/#footnote-1">*</a> This will help trip planning websites and apps to provide better directions to those who use wheelchairs or have other mobility limitations.</li><li>We’re <strong><em>cataloging fares, tickets, and transfer discounts around the Bay Area</em></strong>, creating GTFS-Fares v2 data, also using Interline’s Gotransit editing tools.<a href="https://www.interline.io/blog/mtc-regional-gtfs-feed-release/#footnote-2">**</a> Transit fares and discounts across agencies is a problem of combinatorial complexity. The Regional GTFS Feed will not be able to precisely capture every possible fare, pass, and transfer discount. The goal is to capture sufficient detail to create realistic fare estimates for a variety of rider types riding and transferring between the agencies of highest ridership.</li><li>To start, the Regional GTFS Feed is updated once per day. In the future, we’ll also <strong>create historical versions of the Regional GTFS Feed</strong> that combine “slices in time” from different agency feed versions. While trip planning is best powered by the latest GTFS feed, many types of analyses and travel models instead require archival feed versions from the past.</li></ul><p>Please watch 511.org’s <a href="https://511.org/open-data/transit">Open Transit Data</a> section for updates as these additions are released.</p><h2 id="powered-by-transitland">Powered by Transitland</h2><p>The Regional GTFS Feed has much in common with <a href="https://www.transit.land/">Transitland</a>, our open data platform that aggregates GTFS data for over 2,500 agencies around the world. We’ve used many of the lessons we’ve learned from Transitland to create the Regional GTFS Feed workflow. In fact, the workflow is build using the core, open-source ingredients of <a href="https://www.transit.land/news/2019/10/17/tlv2.html">Transitland version 2</a> (also known as “Tlv2”) The 30+ agency feeds are represented as a DMFR (a Distributed Mobility Feed Registry) and processed using the Gotransit library. The Bay Area’s Regional GTFS Feed workflow is, in effect, a Transitland of its own, which is exposed using a static GTFS export.</p><h2 id="to-use-the-feed-and-ask-questions">To Use the Feed and Ask Questions</h2><p>To use the new Regional GTFS Feed:</p><ol><li><a href="https://511.org/open-data/token">Sign up for a 511 Open Data API token</a></li><li>Download from <code>http://api.511.org/transit/datafeeds?api_key=[your_key]&amp;operator_id=RG</code></li><li>The feed is updated daily; download it as frequently as your application needs updates.</li></ol><p>To provide feedback or ask questions, please visit the <a href="https://groups.google.com/forum/#!forum/511sfbaydeveloperresources">511SFBayDeveloperResources</a> Google group.</p><h2 id="acknowledgements">Acknowledgements</h2><figure><img src="https://www.interline.io/content/images/2024/11/mtc-logo.png" alt="Metropolitan Transportation Commission logo" width="516" height="128"></figure><p>Thanks to our partners in MTC’s Technology Services division. They’ve been serving transit data through 511.org since before GTFS and GTFS Realtime specifications. It’s a pleasure learning from their experience and working together to take advantage of the latest advances in transit data.</p><figure><img src="https://www.interline.io/content/images/2024/11/stir-banner-1.png" alt="Startup in Residence logo" width="231" height="128"></figure><p>Thanks to also to <a href="https://www.cityinnovate.com/">City Innovate</a> for managing the Startup in Residence (STIR) program. We encourage procurement officers and civic-oriented startups to learn more about the program.</p><h2 id="notes">Notes</h2><p><a id="footnote-1"></a></p><p><sup>*</sup> For more on the GTFS-Pathways additions, see <a href="http://bit.ly/gtfs-pathways">this Google doc</a>.</p><p><a id="footnote-2"></a></p><p><sup>**</sup> For more on the GTFS-Fares-v2 proposal, see <a href="http://bit.ly/gtfs-fares">this Google doc</a>.</p>]]></content:encoded></item><item><title><![CDATA[Transportation Research Board funds an open platform for transit agencies to improve the quality of their real-time data]]></title><description><![CDATA[We’re excited to share that the Transportation Research Board has awarded a Transit IDEA Program grant to Interline. We’re using this unique opportunity to expand Transitland to support GTFS Realtime feeds.]]></description><link>https://www.interline.io/blog/transportation-research-board-funds-gtfs-realtime/</link><guid isPermaLink="false">67255705b62841000136eddc</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Wed, 23 Jan 2019 08:00:00 GMT</pubDate><content:encoded><![CDATA[<blockquote><p><strong>January 19, 2021</strong>: Interline and CUTR@USF have now published a Transportation Research Board report on this project. See <a href="https://www.interline.io/blog/trb-transit-idea-gtfs-realtime-report/">this more recent blog post</a>.</p></blockquote><p>We’re excited to share that the <a href="http://www.trb.org/">Transportation Research Board</a> has awarded a <a href="http://apps.trb.org/cmsfeed/TRBNetProjectDisplay.asp?ProjectID=4695">Transit IDEA Program grant</a> to Interline. We’re using this unique opportunity to expand <a href="https://www.transit.land/">Transitland</a> to support <a href="https://developers.google.com/transit/gtfs-realtime/">GTFS Realtime</a> feeds. Our goal is for Transitland to provide a suite of validation tools for public-transit agency staff to validate the contents and quality of their real-time data feeds.</p><h2 id="high-quality-real-time-data-is-hard-but-worthwhile">High-quality real-time data is hard but worthwhile</h2><figure><img src="https://www.interline.io/content/images/2024/11/trb-idea-the-problem.png" alt="the problem statement" width="960" height="720"></figure><p>Real-time transit information has many benefits to transit riders and agencies, including shorter perceived and actual wait times, a more welcoming experience for new riders, and an increased feeling of safety, and increased ridership. Real-time transit data is, in comparison with many other potential operational or capital improvements to bus or rail service, an affordable means of increasing ridership. In the last few years, a real-time complement to the General Transit Feed Specification (GTFS) format, GTFS Realtime, has emerged.</p><p>Despite its promise, adoption of GTFS Realtime by transit agencies has been hampered by a lack of clear documentation and readily available validation tools. Hardware and software vendors for automatic vehicle location (AVL) systems also vary in the depth and quality of their support for the GTFS Realtime specification. As a result, transit agencies must invest significant time and effort to create and maintain high quality GTFS Realtime feeds. Furthermore, bad data has been shown to have a negative effect on ridership, the rider’s opinion of the agency, and the rider’s satisfaction with multimodal trip-planning apps. Thus, transit agencies must put even more effort toward ensuring that the GTFS Realtime data produced by their systems is of sufficient quality.</p><h2 id="our-solution-gtfs-validation-in-transitland">Our Solution: GTFS Validation in Transitland</h2><p>In this project, Interline and the <a href="https://www.cutr.usf.edu/">Center for Urban Transportation Research</a> are creating a prototype platform that makes GTFS Realtime validation tools readily available to, potentially, all transit agencies in North America. We will build upon two open-source projects: the <a href="https://github.com/CUTR-at-USF/gtfs-realtime-validator">GTFS Realtime validator prototype</a> and Transitland, an open transit data platform. This project will apply the open-source and open-data community models to the challenges of creating and improving GTFS Realtime data.</p><p>By combining the open-source components of a GTFS Realtime validator with a catalog of GTFS Realtime feeds, hosted on Transitland’s cloud servers, this project will make the process of validating real-time data simple and accessible to agency staff from any computer with a web browser. As a result, GTFS Realtime data will improve in quality and availability. Transit riders will have a better experience (which has been linked to higher ridership), agency staff will provide better service with less effort and cost, and system vendors will provide higher quality product.</p><h2 id="our-partner-center-for-urban-transportation-research">Our Partner: Center for Urban Transportation Research</h2><p>We’re glad to have on our project team an expert on GTFS Realtime: Sean Barbeau. Sean and his colleagues at the University of South Florida’s Center for Urban Transportation Research have been involved in GTFS, GTFS Realtime, and related transit data specifications and systems for many years. Most recently, they have created a GTFS Realtime validator prototype. We’ll be working together to test and deploy this validator tooling within the Transitland platform.</p><p>For more about their preliminary findings about the quality of GTFS Realtime feeds, see <a href="https://medium.com/@sjbarbeau/introducing-the-gtfs-realtime-validator-e1aae3185439">this recent summary of their TRB paper</a>. Here are the most common errors and warnings their prototype validator found in a sample of GTFS Realtime feeds:</p><figure><img src="https://www.interline.io/content/images/2024/11/validator-test-errors.png" alt="most common errors found in a sample of GTFS Realtime feeds" width="624" height="431"><figcaption><p>Image Credit: Sean Barbeau, Transportation Research Board 2018 paper <a href="https://trid.trb.org/view/1496848">18–05585 &quot;Quality Control — Lessons Learned from the Deployment and Evaluation of GTFS-realtime Feeds&quot;</a>.</p></figcaption></figure><h2 id="about-the-transit-idea-program">About the Transit IDEA program</h2><p>Here’s an overview of the <a href="http://www.trb.org/IDEAProgram/IDEATransit.aspx">Transit IDEA Program</a>:</p><blockquote><p>The Transit IDEA Program is part of the Transit Cooperative Research Program, a cooperative effort of the Federal Transit Administration (FTA), the Transportation Research Board (TRB), and the Transit Development Corporation (a nonprofit educational and research organization of the American Public Transportation Association). The program is funded by the FTA and is managed by TRB.</p><p>The Transit IDEA Panel has established four high-priority focus areas to encourage proposals in the following areas:Increasing transit ridershipImproving transit safety, security, and emergency preparednessImproving transit capital and operating efficienciesProtecting the environment and promoting energy independence.</p></blockquote><p>IDEA stands for “Innovations Deserving Exploratory Analysis” — which we think is an apt description for the Transitland platform providing GTFS Realtime feed validation to agency staff!</p><h2 id="an-open-call-to-transit-agency-staff">An open call to transit agency staff</h2><p>If your agency provides a GTFS Realtime feed, we welcome you to participate in the user-testing phase of this project. Please write us at <a href="mailto:info@interline.io">info@interline.io</a> for more information.</p>]]></content:encoded></item><item><title><![CDATA[Transitland Version 2]]></title><description><![CDATA[What first started as two simple, separate, little experiments has grown into the seeds of a major revision of the entire Transitland platform: Transitland Version 2.]]></description><link>https://www.interline.io/blog/tlv2/</link><guid isPermaLink="false">67255876b62841000136ee13</guid><dc:creator><![CDATA[Drew Dara-Abrams]]></dc:creator><pubDate>Tue, 08 Jan 2019 08:00:00 GMT</pubDate><media:content url="https://www.interline.io/content/images/2024/11/tlv2-architecture-diagram.png" medium="image"/><content:encoded><![CDATA[<img src="https://www.interline.io/content/images/2024/11/tlv2-architecture-diagram.png" alt="Transitland Version 2"><blockquote><p><strong>March 4, 2021</strong>: Here's a more recent blog post <a href="https://www.interline.io/blog/transitland-v2-search/">demonstrating Transitland v2's new ability to search across transit operators, feeds, and routes</a>.</p></blockquote><blockquote><p><strong>February 25, 2021</strong>: Here's a more recent blog post <a href="https://www.interline.io/blog/transitland-v2-vector-map-tiles/">demonstrating Transitland v2's new global transit map and vector tiles</a>.</p></blockquote><p>It's been a quiet year on this blog, and we're overdue to share all the work that's been going on behind the scenes since the <a href="https://www.interline.io/blog/transitland-continues/">last post</a>. What first started as two simple, separate, little experiments has grown into the seeds of a major revision of the entire Transitland platform: Transitland Version 2.</p><h2 id="tlv2-functionality-and-goals">Tlv2 Functionality and Goals</h2><p><em>Tlv2</em> (as we've come to call this new version) provides continuity on the most used parts of the Transitland platform. Tlv2 will still provide a canonical service at transit.land that includes a Feed Registry of open GTFS data feeds from around the world, a queryable API, and a range of user interfaces. Tlv2, like Tlv1, will be powered by open-source components. Tlv2's API will continue to be accessible without charge, and most of the Tlv1 API will continue to be available during the transition. (As before, we will enforce rate limits to ensure no one user hogs too many server resources; more on how rate limits will change below.)</p><p>What we are changing in Tlv2 is in support of three goals:</p><ul><li>Higher Performance</li><li>More Modular Architecture</li><li>Distributed Deployments</li></ul><h2 id="goal-1-higher-performance">Goal #1: Higher Performance</h2><p>Tlv1 successfully parses and imports GTFS feeds from thousands of transit agencies. However its &quot;FeedEater&quot; component slows down or halts on a handful of the very larger GTFS feeds — typically single feeds that cover entire countries. This has always been a constraint on Tlv1.</p><p>Interline's new <code>transitland-lib</code> (tlib) library solves this problem. Transitland-lib is a new transit data library written in the <a href="https://golang.org/">Go programming language</a>, which provides finer-grained control of memory usage, as well as much easier patterns for concurrency and parallelization of complex tasks. Transitland-lib replaces replaces FeedEater, and will be used for feed validation, pre-processing, and importing data into the Tlv2 database. Transitland-lib is efficient enough to handle the largest feeds in a small memory footprint, and is fast enough to encourage interactive use; imports that took hours with FeedEater are completed in minutes.</p><p>Transitland-lib began as an experiment by Ian Rees over the last winter holidays, and it now complete enough to power multiple Interline projects, from the San Francisco Bay Area Regional GTFS Feed (which merges ~30 feeds into one) to a transit-to-routing-engine pipeline that Interline built for another client (which covers transit service for an entire country!). We look forward to releasing transitland-lib under an open-source license shortly.</p><p>Transitland's most demanding API users have also hit performance limitations over the years. As part of Tlv2, we'll be providing bulk data dumps that give users another option. Use the Tlv2 API to explore data or power a small web app; use Tlv2 data extracts to power a large analysis, a giant web app, or a country-wide routing engine.</p><h2 id="goal-2-more-modular-architecture">Goal #2: More Modular Architecture</h2><p>Tlv1 has succeeded in part due to its &quot;monolith&quot; architecture. FeedEater, the API, and administrative operations all live within the same <a href="https://www.transit.land/documentation/datastore/">Transitland Datastore</a> codebase. This has allowed for efficient reuse of data models, helpers, and test code. The single Datastore codebase is also straightforward to deploy and configure for the canonical transit.land site. The Datastore codebase can be reused — some have in fact redeployed it for their own regional or internal GTFS management needs—but customizing and redeploying the Datastore is not straightforward.</p><p>For Tlv2, we're breaking the Transitland Datastore into modular components. Some components will be in Go (data processing), some will remain in Ruby (general back-end) and JavaScript (user interfaces). The canonical transit.land deployment will continue to use all. Other deployments by Interline, our partners, or open-source users may only use a subset. This more modular architecture will also allow deployments that connect Tlv2 components to internal data pipelines, routing engines, and other tools that may not be relevant to the canonical transit.land deployment.</p><p>While the software components of Tlv2 will be more modular, they will still be glued together using the data schemes developed for Transitland, including <a href="https://www.transit.land/documentation/concepts/onestop-id-scheme">Onestop IDs</a> and a common feed format spec (described below). Additionally, we will be sharing our database schema in versions appropriate for PostgreSQL and SQLite to help simplify additional application development, including outside of the canonical transit.land.</p><h2 id="goal-3-distributed-deployments">Goal #3: Distributed Deployments</h2><p>The Feed Registry at transit.land will continue to be our canonical list of GTFS feeds. However, some deployments of Tlv2 components may want to use just a subset of the Feed Registry or a private list of feeds. We also want to open up the canonical Feed Registry for more collaborative tending, beyond just staff from Interline and Trillium Transit.</p><p>The second of the two experiments that has led to Tlv2 is called the <a href="https://github.com/transitland/distributed-mobility-feed-registry">Distributed Mobility Feed Registry</a>. DMFR (for short) is a standard way of representing a list of GTFS feeds as one or more JSON files. It also currently supports GTFS-RT, and we expect it to extend to GBFS (bike rental) and MDS (e-scooter rental) in the future.</p><p>(Some history: This experiment actually started way back in the early tests of Transitland, before the fully built out Tlv1 monolith. It's still <a href="https://github.com/transitland/transitland-datastore">archived</a> on GitHub. Now we've come full circle to revisit and build out this approach.)</p><p>Soon the <code>transitland/distributed-mobility-feed-registry</code> repository on GitHub will become the source of truth for the Transitland Feed Registry of public open data feeds. We'll accept pull requests against the JSON files to add or edit feeds. This should substantially lower the burden for outside contributors to contribute new feeds and help manage existing records. However, this will be just be one repository of DMFR files. In effect we're blowing up the single Transitland Feed Registry and inviting the creation of many.</p><p>Transitland-lib can manage a datastore using one or more DMFR files, meaning that the canonical transit.land, distributed instances of Tlv2, and temporary interactive sessions with transitland-lib on the command line can all share the same representation for mobility data feeds. At Interline, we've gone one step further and also use DMFR files to configure some of our routing engines and any other workflow that takes two or more mobility data feeds as input. We welcome other consumers or transformers of GTFS, GTFS-RT, and related data to try a DMFR file for their own needs.</p><h2 id="tlv2-architecture-diagram">Tlv2 Architecture Diagram</h2><p>Now that we've introduced and discussed the new components of Tlv2, here's a diagram of how they connect:</p><figure><img src="https://www.interline.io/content/transit-land/tlv2/tlv2-architecture-diagram.png" alt="Tlv2 architecture diagram" width="768" height="942"></figure><h2 id="gradual-deprecation-of-tlv1">Gradual Deprecation of Tlv1</h2><p>Tlv1 requires a fair amount of attention and resources to maintain. While developing Tlv2, we've already had to split our attention. As we shift our focus to deploying and supporting Tlv2, we'll be gradually deprecating Tlv1. Don't worry — it won't be too quickly.</p><p><strong>Step 1: Tlv2 API Preview</strong> The initial release of Tlv2 will include a preview of a new <a href="https://graphql.org/">GraphQL</a> API that provides flexible, structured access to transit data. All new data will be available through this API from the start. API access will require a key. Sign-up will be free. We will institute tight rate limits at first and increase them over time, as we better understand the type and complexity of queries that users run.</p><p><strong>Step 2: Deprecate Tlv1 Schedule API</strong> Immediately upon Tlv2 release, we will stop importing new <a href="https://www.transit.land/documentation/datastore/schedules.html">schedule data</a> into Tlv1. The <code>ScheduleStopPair</code> data model is the most expensive part of Tlv1 to run and maintain. We'll first stop imports, keeping the API available against the last imported version. We'll share tutorials on how to switch to the Tlv2 GraphQL API to access schedules. Then, after sufficient notice, we'll empty old Tlv1 schedule data and remove that API endpoint.</p><p><strong>Step 3: Other Tlv1 Entity API Endpoints</strong> While the Tlv2 GraphQL API will make a number of complex queries much simpler than with the current Tlv1 API, it may take a while for us to make all Tlv1 query options available in the Tlv2 API. During this time, we'll try to continue to power the Tlv1 APIs (with the exception of the schedules endpoint). We'll recommend that you build any new applications on the Tlv2 API, in case we do have to announce any specific incompatibilities during the Tlv1-to-Tlv2 API transition.</p><p>We'll post updates to this site. Or watch <a href="https://twitter.com/transitland">@transitland</a> and <a href="https://twitter.com/interline_io">@interline_io</a> for updates on Twitter.</p><h2 id="how-you-can-help">How You Can Help</h2><p>Here are ways you can help with the rollout of Tlv2:</p><ul><li>Provide your input on Tlv2 by taking our <a href="https://docs.google.com/forms/d/e/1FAIpQLSdmhW28zcP1kBMBkmelAt7OQGqKjRcoILjsbZvGXLsGfmFNGw/viewform">survey and signing up for the beta tester list</a>.</li><li>Comment on the <a href="https://funding.communitybridge.org/projects/transitland">DMFR GitHub repository</a>. Contribute edits once we migrate over the Feed Registry to DMFR files.</li><li>If you work for a GTFS producing transit operator or mobility provider, <a href="mailto:info@interline.io">contact Interline</a> to learn more about &quot;Tlv2 on premise&quot; and regional GTFS feeds. It's consulting projects like this that help us serve both mobility providers and further the development of transitland-lib.</li><li>Contribute financially to the upkeep and growth of Tlv2 through the Linux Foundation's <a href="https://funding.communitybridge.org/projects/transitland">CommunityBridge donation page for Transitland</a>. These contributions go directly toward upkeep of the canonical transit.land.</li></ul>]]></content:encoded></item></channel></rss>