Bring RPDE in line with JSON-LD and remove duplication

[nickevansuk]

Proposer

ODI

Requirements

• Bring RPDE in line with JSON-LD
• Remove duplication of properties
• Allow feeds to be combined with Opportunity API (Safely combine updates from the Opportunity API and RPDE Feeds openactive-archive/opportunity-api#10)

Proposal

• Replace "kind" with JSON-LD "@type"
• Use JSON-LD "@id"
• Augment JSON-LD with "@modified" to allow any entity to have a modified date
• Augment JSON-LD with "@deleted" to allow any entity to have a "deleted" state (using a boolean instead of the current updated|deleted field as there are only two possible values for that field).
• Use @ for type, id, modified and deleted, to distinguish between control properties and data properties.
• Items with "@deleted": true MUST only include "@id", "@modified" and "@deleted" properties (note that the Opportunity API could also return such small response for objects that have been deleted, for consistency).
• "@id" and "@modified" are also used in the Opportunity API to allow items retrieved to be comparable to the those in the feed.
• Make "items" singular in line with schema.org naming contentions
• Add "type": "RealtimePage" and "@context": "https://openactive.io/" to the base of the feed
• "@modified" MUST use ISO 8601 format with sub-millisecond precision

Example

{
  "@context": "https://openactive.io/",
  "type": "RealtimePage"
  "next": "http://api.letsride.co.uk/public/v1/rides?afterModified=2018-09-30T18%3A45%3A10.263Z&afterId=http%3A%2F%2Fwww.example.com%2Fevents%2F2710",
  "item": [
    {
      "@id": "http://www.example.com/events/2710",
      "@type": "SessionSeries",
      "@modified": "2018-09-30T18:45:10.242Z",
      "name": "Speedball",
      "offers": [
        {
          "@type": "Offer",
          "@id": "https://example.com/events/2710#/offers/878",
          "validThrough": "2018-10-29T11:00:00Z",
          "validFrom": "2018-10-01T11:00:00Z",
          "description": "Winger space for Speedball.",
          "name": "Speedball winger position",
          "price": 10.00,
          "priceCurrency": "GBP",
          "isCancellable": true,
          "cancellationValidUntil": "2018-10-28T11:00:00Z"
        }
      ],
      "duration": "PT1H",
      "organizer": {
        "@type": "Organization",
        "name": "Central Speedball Association",
        "url": "http://www.speedball-world.com",
        "termsOfService": { "type": "TermsOfService", "url": "url"}
      },
    },
    {
      "@id": "http://www.example.com/events/2710",
      "@type": "SessionSeries",
      "@modified": "2018-09-30T18:45:10.263Z",
      "@deleted": true
    }
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/"
}

Open questions

• "modified" as ISO 8601 format with sub-millisecond precision - is this necessary? As long as the format is consistent between the Opportunity API and RPDE feed could we still use native integer changeNumber as the timestamp instead?

[nickevansuk]

Now depends on openactive-archive/opportunity-api#10

[nickevansuk]

Just to note that schema.org has since introduced https://schema.org/DataFeed and https://schema.org/DataFeedItem, which are both applicable here.

A suggested mapping of the current RPDE terms to schema.org (also picking up on the points in #54), is as follows:

RPDE 1.0	Class	Property	Type
next	oa:RealtimePagedDataFeed	oa:nextUrl	schema:URL
items	oa:RealtimePagedDataFeed	schema:dataFeedElement	Array of schema:DataFeedItem
license	oa:RealtimePagedDataFeed	schema:license	schema:URL
	oa:RealtimePagedDataFeed	schema:schemaVersion	schema:URL
state	schema:DataFeedItem	oa:itemDeleted	schema:Boolean
kind	schema:DataFeedItem	schema:item.@type	schema:Property
id	schema:DataFeedItem	schema:identifier	schema:Text OR schema:Integer
modified	schema:DataFeedItem	schema:version	schema:Integer
data	schema:DataFeedItem	schema:item	schema:Thing

A few notes on this mapping:

• The purpose of this mapping is to bring RPDE in line with schema.org and JSON-LD, without making any functional changes to the now thoroughly tried-and-tested RPDE mechanics.
• oa:RealtimePagedDataFeed subclasses schema:DataFeed to indicate the specific type of data feed this represents, which aids validation and tooling.
• state is replaced by a boolean oa:itemDeleted, as RPDE only specifies two states - so this is a more compact representation. Note that oa:itemDeleted defaults to false, so is not required to be present in the scenario that was previously described as state of updated.
• identifier would still be used within RealtimePagedDataFeedItem rather than using item.@id in order to maintain the ?afterId= parameter behaviour without require full URL encoding (and also in line with Overloading of "id" across JSON-LD and RPDE #87). ?afterId= should also be renamed to ?afterIdentifier=.
• schemaVersion could additionally be added to reference the model in use by the feed (building on suggestions in Additional properties in RPDE response #86).
• The use of RPDE should be specified as a mime type via the Content-Type header, in line with the tooling - although this feed would be fully compatible with application/ld+json and the root @type of RealtimePagedDataFeed infers RPDE, the underlying transport mechanism of RPDE should still be specified via Content-Type ala application/atom+xml.
• As per Enforce integer modified date and afterTimestamp as MUSTs. #89 ordering remains an integer to allow it to be compatible with both afterId and afterChangeNumber ordering strategies.
• For items, version is preferred to modified as it does not imply any relationship to time, and respects the invariant (Remove overloaded use of modified property #80 (comment)) - this maintains the simplicity of implementation as data consumers can treat both ordering strategies in the exactly the same way. ?afterChangeNumber= and ?afterTimestamp= parameters should not be renamed to ?afterVersion=, as their separation is useful for validation and debugging.
• kind is replaced by item.@type, as there is no need to duplicate the property within DataFeedItem when it is mandatory within Thing contained in the item. This means that deleted items require an empty Thing with only @type specified, to preserve the ability to combine multiple kinds within one feed, and for the data consumer to easily store these in separate places.
• Suggest we remove the dependency on the Opportunity API for this next version, partly as the Opportunity API is not on the near-term roadmap, and partly as the idea of combining real-time feed data with the latest search results from an Opportunity API is an architectural nightmare. Better for implementers to focus on improving the performance of their RPDE implementation.
• We should include cache headers in this next version of the RPDE spec, including the use of the expires and Cache-Control headers to hint at data consumer polling frequency and inform the CDN, as per FEATURE: Using a CDN to scale RPDE #79. Such practices are already in use in production, so this is just formalising this into the spec.

{
  "@context": "https://openactive.io/",
  "@type": "RealtimePagedDataFeed",
  "nextUrl": "http://www.example.com/api/rpde/sessions?afterTimestamp=1453931925&afterIdentifier=%7Bd97f73fb-4718-48ee-a6a9-9c7d717ebd85%7D",
  "dataFeedElement": [
    {
      "@type": "DataFeedItem",
      "identifier": "{c15814e5-8931-470c-8a16-ef45afedaece}",
      "version": 1453931101,
      "item": {
        "@type": "Event",
        "@id": "https://bookingsystem.com/hulahoop/e/ev-ssyp-20160509191500",
        "identifier": "session-01jz93i3k1p3",
        "startDate": "2016-05-09T18:15:00Z",
        "endDate": "2016-05-09T19:15:00Z",
        "remainingAttendeeCapacity": 0,
        "url": "https://bookingsystem.com/hulahoop/e/ev-ssyp-20160509191500?r=oa",
        "attendanceCount": 29,
        "location": {
          "@type": "Place",
          "identifier": "location-jbnentuhxsnp",
          "url": "https://leisurecentre.org.uk/leisure-centre/camden/kentish-town-sports-centre",
          "name": "Kentish Town Sports Centre",
          "address": {
            "@type": "PostalAddress",
            "addressLocality": "Camden",
            "addressRegion": "London West",
            "postalCode": "NW5 3DU",
            "streetAddress": "Kentish Town Sports Centre"
          }
        }
      }
    },
    {
      "@type": "DataFeedItem",
      "identifier": "{c15814e5-8931-470c-8a16-ef45afedaece}",
      "version": 1453931925,
      "itemDeleted": true,
      "item": {
        "@type": "Event"
      }
    }
  ],
  "schemaVersion": "https://www.openactive.io/modelling-opportunity-data/2.0",
  "license": "https://creativecommons.org/licenses/by/4.0/"
}