inline documentation for all defined simple and derived types in json schema/XML/etc

[DanSmith]

Originally posted by @borsna in #80 (comment)

One thing that could improve the json schema is adding a short documentation and example e.g:

{
  "type": "object",
  "description": "Represents a localized text value associated with a specific language.",
  "properties": {
    "LanguageTag": {
      "type": "string",
      "description": "IETF BCP 47 language tag that specifies the language and optional regional variant of the text.",
      "examples": [
        "en",
        "en-US",
        "sv",
        "de"
      ]
    },
    "Value": {
      "type": "string",
      "description": "The localized text value in the specified language."
    }
  },
  "required": [
    "LanguageTag",
    "Value"
  ],
  "additionalProperties": false,
  "examples": [
    {
      "LanguageTag": "en",
      "Value": "Hello world!"
    },
    {
      "LanguageTag": "sv",
      "Value": "Hej världen!"
    }
  ]
}

[DanSmith]

Add documentation from https://cogsdata.org/docs/modeler-guide/primitive-types/ to the simple and derived type defs.

[duffgui]

Following discussions in #60 and #78, here is a first feedback on the regexp error (already mentioned) based on the last version of the JSON Schema after some tests, research and help:
There seems to be still an issue with the regular expression used in different definitions such as gMonthDay spotted by all validators:

On schema['$defs']['gMonthDay']['properties']['Timezone']['pattern']:
'^(Z)|((\+|\-)(00|0[0-9]|1[0-9]|2[0-3]):([0-9]|[0-5][0-9]))$'

A correct version seems to be:
^(Z|((\\+|-)(00|0[0-9]|1[0-9]|2[0-3]):([0-9]|[0-5][0-9])))$
In the original expression (^Z) | (...$), the alternation operator | has lower precedence than the anchors. So the regex is interpreted as: (^Z) | (...$).
This means that the ^ applies only to the Z branch and the $ applies only to the timezone-offset branch. As a consequence, the expression does not enforce that the entire string matches one of the two alternatives. For example, it could incorrectly match:
"abc+01:30" or "Zzzz" because one branch is only anchored at the beginning and the other only at the end.
The updated version ^(Z|(...))$ places the alternation inside a single grouped expression, so the anchors apply to the entire union: ^( A | B )$ meaning: the whole string must be either Z or a valid timezone offset.
There was a more critical issue around the escaping of -.
The original version with (\\+|\\-) becomes in JSON the following regexp (\+|\-).
The issue is that in ECMAScript regular expressions (used by JSON Schema), \- outside a character class is considered an invalid or non-standard escape sequence. The minus sign does not need escaping outside []. So \- should simply be -. That is why the updated version uses (\\+|-) which becomes (\+|-) and is valid ECMAScript regex syntax.

I am validating the rest of a test file. Maybe more remarks to come in a next comment.

[DanSmith]

Thanks @duffgui, this looks like an XML vs JSON regex issue where XML is implicitly anchored and json pattern in unanchored.

This is easy enough to fix in the json schema, and is the only internal regex pattern:
https://github.com/Colectica/cogs/blob/723d375e46b37602f24ee286f32f29531655d2e8/Cogs.Publishers/FluentJsonSchemaPublisher.cs#L245
Could update it to ^(Z|[+-](?:[01]\\d|2[0-3]):[0-5]\\d)$ or your suggestion.

We may need a strategy for model defined regex patterns that are user generated, and output to both the XML schema and JSON Schema. Right now any Lifecycle regex defined in the model are XML schema derived. Perhaps a conversion with warnings for regex parts that are not convertible (character class subtraction, unicode escapes, etc).

[duffgui]

Based on the very small test file I have built from the last version of the JSON Schema, here are two remarks on naming, only for documentation purposes.

1. The TopLevelReference element is a named property and becomes "topLevelReferences". I guess pluralising array properties is inevitable when turning UML/XMI classes into structural JSON property names. But the naming still feels awkward since the JSON layer partially renames concepts according to container semantics.
2. The DDI model distinguishes IdentifiableType (identifiable only), VersionableType (identifiable and versionable), MaintainableType (identifiable + versionable + maintainable/reusable).
   Examples:

• CodeType → only IdentifiableType
• LogicalRecordType → only IdentifiableType
• Category → maintainable/versionable
• Variable → maintainable/versionable

In XML, this distinction is not strongly exposed structurally. A Code often still contains ID, Agency, Version and URN. The difference is mostly semantic and governed by lifecycle rules. So XML remains relatively homogeneous.
But the JSON schema introduces a structural distinction and distinguishes:

• embedded identifiable components, an example where Code is treated as a property containing embedded objects.:

"Code": {
  "type": "array",
  "items": {
    "$ref": "#/$defs/CodeType"
  }
}

• and top-level maintainable/versionable entities, an example of a Category defined as as first-class document resources.

"Category": {
  "type": "object",
  ...
}

What feels a bit confusing is the fact that:

• one becomes a named property (Code with"$type": "CodeType")
• the other becomes a top-level discriminated object (Category with $type: "Category").

Again, these artefacts seem to me inevitable when trying to encode maintainability and reuse semantics directly into the JSON structure, but I guess it is worth mentioning them somewhere in the documentation.