Bug when service exceptions are outside service dates?

[josiekre]

I might be missing something, but I'm almost certain I've found a discrepancy between the original GTFS and the resulting schedules from schedule_stop_pairs.

For example, from SHA1 ff834518ff5728fc178966b3465c5057c3ade019, route r-dqcjp-nm for trips 47 and 48 have a service end date of 2018-12-25 but they should be 2018-03-31.

There's a service exception on 2018-12-25 for this service ID (in GTFS Red_Winter_Monday_Friday), which is invalid because it is past the service end date. This might be what has caused the service end date in the API to span past its actual service date.

I caught this because there are repetitive services for this route between 2018-03-31 until 2018-12-25 now when pulling data from the API.

[drewda]

Hi @josiekre, thanks for your question. You are right that there are occasionally differences between original GTFS feeds and ScheduleStopPair contents. When the contents of a feed are about to expire, Transitland automatically extends the appropriate SSPs out into the future. This is so that routing engines that use Transitland data will still return trips to their users -- the schedules may be out of date, but many routing providers would rather provide something rather than nothing to their users.

If a feed version has had its schedule extended, you should see tags called extend_to_date and extend_from_date. For example, see this feed version: http://transit.land/api/v1/feed_versions/55b23bcff5ccf13a08e1b1fbe49308e34f086ebd

If you have any follow ups, feel free to comment and we'll reopen the issue.

[josiekre]

That does not seem to be the case for this feed. See: http://transit.land/api/v1/feed_versions/ff834518ff5728fc178966b3465c5057c3ade019.

In short, this feed had two service IDs for one route (Red bus line) Monday - Friday, one which went Jan 1 - March 31 2018 and the other which goes Apr 1 - Oct 6 2018. Right now, both are returning in the feed even though the first should be gone.

Thanks for looking into this with me.

[drewda]

@josiekre you're right, this is likely a different issue than feed extension.

FWIW, we do have a test to catch situations of service exceptions that are provided outside of actual service windows: https://github.com/transitland/transitland-datastore/blob/master/spec/models/schedule_stop_pair_spec.rb#L306-L322

We'll see if there are any other explanations for this behavior.

[irees]

@josiekre
Hi, your initial assumption is correct. When importing GTFS, Transitland expands the service start_date/end_date to include all service exceptions. See: https://github.com/transitland/gtfs/blob/master/lib/gtfs/service_period.rb#L78-L84 .

This was to simplify some of the "service on date" queries, but in retrospect may have been a poor decision. I agree it should probably be handled differently, e.g. only expanding the service range when reading only from calendar_dates.txt without a value from calendars.txt. I created an issue for making this change. #1261

[josiekre]

I'm solving it on my end right now by looking for unique service per route somehow (I think). It's a while ago so the details escape me. But I'm going to close this since you've moved it to a new issue.