You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by GitBox <gi...@apache.org> on 2022/07/08 14:44:30 UTC
[GitHub] [unomi] jsinovassin commented on a diff in pull request #457: UNOMI-625 : add documentation on json schema
jsinovassin commented on code in PR #457:
URL: https://github.com/apache/unomi/pull/457#discussion_r916887216
##########
manual/src/main/asciidoc/jsonSchema/introduction.adoc:
##########
@@ -0,0 +1,280 @@
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+=== Introduction
+
+==== What is a JSON Schema
+
+https://json-schema.org/specification.html[JSON Schema] is a powerful standard for validating the structure of JSON data.
+Described as a JSON object, a JSON schema file contains format, types, patterns, and more.
+Used against JSON data, a JSON schema validates that the data is compatible with the specified schema.
+
+Example of a basic JSON schema that validate that the path property is a string property:
+
+[source]
+----
+{
+ "$id":"https://unomi.apache.org/schemas/json/example/1-0-0",
+ "$schema":"https://json-schema.org/draft/2019-09/schema",
+ "title":"Example of a basic schema",
+ "type":"object",
+ "properties":{
+ "path":{
+ "type":"string",
+ "$comment":"Example of a property."
+ }
+ }
+}
+----
+
+[source]
+----
+{
+ "path": "example/of/path" //Is valid
+}
+----
+
+[source]
+----
+{
+ "path": 100 // Is not valid
+}
+----
+
+Apache Unomi is using json-schema-validator to integrate JSON schema.
+The library and its source code is available at: https://github.com/networknt/json-schema-validator[https://github.com/networknt/json-schema-validator], you can refer to the feature’s pom.xml available at https://github.com/apache/unomi/blob/master/extensions/json-schema/services/pom.xml#L35[json-schema/service/pom.xml] to identify which version of the library is currently integrated.
+
+You can discover and play with JSON schema using online tools such as https://www.jsonschemavalidator.net/[JSON Schema Validator].
+Such tools allow you to validate a schema against JSON data (such as the example above), and can point to particular errors.
+More details about JSON schema are available on the official specification’s website: https://json-schema.org/specification.html[https://json-schema.org/specification.html]
+
+==== Key concepts
+
+This section details concepts that are important to understand in order to use JSON schema validation with Apache Unomi.
+
+===== $id keyword
+
+The *$id* keyword:
+
+Each schema in Apache Unomi should have a *$id*, the *$id* value is an URI which will be used to retrieve the schema and must be unique.
+
+Example:
+
+[source]
+----
+{
+ "$id":"https://unomi.apache.org/schemas/json/example/1-0-0"
+}
+----
+
+===== $ref keyword
+
+The *$ref* keyword allows you to reference another JSON schema by its *$id* keyword.
+It’s possible to separate complex structures or repetitive parts of schema into other small files and use *$ref* to include them into several json schemas.
+
+Example with a person and an address:
+
+[source]
+----
+{
+ "$id": "https://example.com/schemas/address",
+ "type": "object",
+ "properties": {
+ "street_address": { "type": "string" },
+ "city": { "type": "string" },
+ "state": { "type": "string" }
+ }
+}
+----
+
+[source]
+----
+{
+ "type": "object",
+ "properties": {
+ "first_name":{ "type": "string" },
+ "last_name": { "type": "string" },
+ "shipping_address": {
+ "$ref": "https://example.com/schemas/address"
+ },
+ "billing_address": {
+ "$ref": "https://example.com/schemas/address"
+ }
+ }
+}
+----
+
+More details about *$ref* can be found in the specifications: https://json-schema.org/understanding-json-schema/structuring.html#ref[https://json-schema.org/understanding-json-schema/structuring.html#ref]
+
+===== allOf keyword
+
+The allOf keyword is an array of fields which allows schema composition.
+The data will be valid against a schema if the data are valid against all of the given subschemas in the allOf part and are valid against the properties defined in the schema.
+
+[source]
+----
+{
+ "$id": "https://unomi.apache.org/schemas/json/example/1-0-0",
+ "$schema": "https://json-schema.org/draft/2019-09/schema",
+ "type": "object",
+ "allOf": [
+ {
+ "type": "object",
+ "properties": {
+ "fromAllOf": {
+ "type": "integer",
+ "$comment": "Example of allOf."
+ }
+ }
+ }
+ ],
+ "properties": {
+ "myProperty": {
+ "type": "string",
+ "$comment": "Example of a property."
+ }
+ }
+}
+----
+
+Valid JSON:
+
+[source]
+----
+{
+ "myProperty": "My property",
+ "fromAllOf" : 10
+}
+----
+
+Invalid JSON:
+
+[source]
+----
+{
+ "myProperty": "My property",
+ "fromAllOf" : "My value"
+}
+----
+
+It’s also possible to use a reference *$ref* in the *allOf* keyword to reference another schema.
+
+In UNOMI, there is an example of using *$ref* in the *allOf* keyword to validate the properties which are defined in the event schema.
+This schema contains properties common to all events.
+It’s done in the the view event schema.
+The file can be found on github: https://github.com/apache/unomi/blob/master/extensions/json-schema/services/src/main/resources/META-INF/cxs/schemas/events/view/view.json#L13[view.json]
+More details about allOf can be found in the specifications: https://json-schema.org/understanding-json-schema/reference/combining.html#allof[https://json-schema.org/understanding-json-schema/reference/combining.html#allof]
+
+===== unevaluatedProperties keyword
+
+The *unevaluatedProperties* keyword is useful for schema composition.
+This keyword is similar to *additionalProperties* except that it can recognize properties declared in sub schemas.
+When setting the *unevaluatedProperties* value to *false*, the properties which are not present in the properties part and are not present in the sub schemas will be considered as invalid.
+
+Example with the following schema:
+
+[source]
+----
+{
+ "$id": "https://unomi.apache.org/schemas/json/example/1-0-0",
+ "$schema": "https://json-schema.org/draft/2019-09/schema",
+ "type": "object",
+ "allOf": [
+ {
+ "$ref": "https://unomi.apache.org/schemas/json/subschema/1-0-0"
+ }
+ ],
+ "properties": {
+ "myProperty": {
+ "type": "string",
+ "$comment": "Example of a property."
+ }
+ },
+ "unevaluatedProperties": false
+}
+----
+
+Sub schema:
+
+[source]
+----
+{
+ "$id": "https://unomi.apache.org/schemas/json/subschema/1-0-0",
+ "$schema": "https://json-schema.org/draft/2019-09/schema",
+ "type": "object",
+ "properties": {
+ "fromAllOf": {
+ "type": "string",
+ "$comment": "Example of allOf."
+ }
+ }
+}
+----
+
+With the following data, the validation will fail because the property *myNewProperty* is not defined neither the *properties* part nor the *allOf* part.
+
+[source]
+----
+{
+ "myProperty": "My property",
+ "fromAllOf" : 10,
+ "myNewProperty": "another one" //Not valid
+}
+----
+
+==== How are JSON Schema used in Unomi
+
+JSON Schema is used in UNOMI to validate the data coming from the two public endpoints */contextRequest* and */eventCollector*.
+Both endpoints have a custom deserializer which will begin by validating the payload of the request, then will filter invalid events present in this payload.
+If an event is not valid it will not be processed by the system.
+The internal events are not validated by JSON schema as they are not sent through the public endpoints.
+
+In UNOMI, each event type must have an associated JSON schema.
+To validate an event, UNOMI will search for a schema in which the target of the schema is *events*, and with the name of the schema matching the event type.
+
+A custom keyword named *self* has to be present in the JSON schemas to store the information related to each schema.
+The following example is the *self* part of the view event JSON schema.
+Having the target set to *events* and the name set to *view*, this schema will be used to validate the events of type *view*.
+
+[source]
+----
+…
+"self":{
+ "vendor":"org.apache.unomi",
+ "target" : "events",
+ "name": "view",
+ "format":"jsonschema",
+ "version":"1-0-0"
+},
+…
+----
+
+Link to the schema on github: https://github.com/apache/unomi/blob/master/extensions/json-schema/services/src/main/resources/META-INF/cxs/schemas/events/view/view.json[view.json].
+
+A set of predefined schema are present in UNOMI, these schemas can be found under the folder : https://github.com/apache/unomi/tree/master/extensions/json-schema/services/src/main/resources/META-INF/cxs/schemas[extensions/json-schema/services/src/main/resources/META-INF/cxs/schemas].
+
+These schemas will be loaded in memory at startup.
+Each schema where the *target* value is set to *events*, will be used to validate events.
+The others are simply used as part of JSON schema or can be used in additional JSON schemas.
+
+It’s possible to add JSON schema to validate your own event by using the API, the explanations to manage JSON schema through the API are
+in the <<Create / update a JSON schema to validate an event, Create / update a JSON schema to validate an event>> section.
+
+Contrary to the predefined schemas, the schemas added through the API will be persisted in elasticsearch in the jsonSchema index.
+Schemas persisted in Elasticsearch do not need a restart of the platform to reflect changes.
Review Comment:
I think you wanted to say ... "do not require a restart..."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@unomi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org