]> Loco-labs

476 chemin du gaf de Famian BOLLENE 84 500 FR philippe@loco-labs.io https://github.com/loco-philippe/NTV/blob/main/README.md

General Internet Engineering Task Force JSON SEMANTIC This document describes a set of simple rules for unambiguously and concisely encoding semantic data into JSON Data Interchange Format. These rules are based on an NTV (Named and Typed Values) data structure applicable to any simple or complex data. The JSON-NTV format is its JSON translation.  

Introduction

Presentation The semantic level of JSON or CSV shared data remains low. It is often limited to the type of data defined in those exchange formats (strings for CSV formats; numbers, strings, arrays and objects for JSON formats). JSON-NTV proposes to increase the semantic level of the JSON entities by adding two additional pieces of information to a JSON entity :

• name: interpretation of the JSON value in human language or detailed information or link to external information,
• type: interpretation of the JSON value in a data standard (eg. GeoJSON, datetime), in a data catalog or in a software language.

The NTV entity is thus a triplet with a mandatory element (value) and two additional elements (name, type).

• For example, Paris location can be represented by :
  • a name : "paris",
  • a type : geoJSON Point coordinates ,
  • a value : [2.3522, 48.8566]

The easiest way to add that information into a JSON value is to use a JSON object with a single member. The first term is the additional elements using the syntax JSON-ND . The second term is the JSON value.

• The JSON value of the previous example is:
  • { "paris:point" : [2.3522, 48.8566] }

With this approach, two NTV entities are defined :

• a primitive entity which is not composed of any other entity,
• a structured entity which is an ordered sequence of NTV entities.

as well as two JSON formats depending on the presence of the additional elements :

• simple format when name and type are not present
• named format when name or type is present

• Example (entity composed of two other primitive entities):
  • { "cities::point": [[2.3522, 48.8566], [4.8357, 45.7640]] } simple format for primitive entities
  • {"cities::point": {"paris":[2.3522, 48.8566], "lyon":[4.8357, 45.7640]}} named format for primitive entities

A JSON-NTV generator produces a JSON value from a NTV entity and vice versa a JSON-NTV parser transforms a JSON value into a NTV entity. The conversion between NTV entity and native entity is outside the scope of this note.

Key design features The format is focused on simplicity, lightness and web usage. The key features of this format are the following:

• JSON as the base format
  • JSON is simple and readable as simple text
  • JSON supports rich structure including nesting and basic types
  • JSON is web-native and very widely used and supported
  • JSON format has binary representation (eg. CBOR format)
• high semantic level of data
  • wide variety of data typing
  • tree-like and customisable data typing
• compatibility with existing formats
  • All JSON data is a JsonNTV data
  • All NTVtypes are standard types
• compatibility with any type of platform
  • types and structures are independent of software and hardware platforms
  • the NTV concept is applicable to all types of "jsonable" data
• reversibility
  • the NTV entity coming from a JSON value is identical to the NTV entity from which the JSON value comes
  • the JSON representation of an NTV entity is the same as the JSON value used to construct the NTV entity
• tree structure
  • all NTV entities are trees
  • tree properties are applicable to NTV entities

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. This document also uses the following terms:

JsonType, JsonText, JsonName, JsonValue, JsonObject, JsonMember, JsonElement, JsonArray, JsonNumber, JsonString, JsonFalse, JsonNull, JsonTrue:

These terms are defined as type, text, name, value, object, member, element, array, number, string, false, null, true in the JSON grammar.

JsonPrimitive:

A JsonNumber, JsonString, JsonFalse, JsonTrue or JsonNull.

JsonUnnamed:

A JsonObject without a single member.

JsonNamed:

A JsonObject with a single member.

NTVsingle, NTVlist:

NTVlist and NTVsingle entities can be abbreviated as:

• NVsingle : entity with default NTVtype "json",
• NVlist : entity without NTVtype,
• TVsingle, TVlist : entity without NTVname,
• Vsingle, Vlist : NVsingle or NVlist without NTVname.

 

NTV structure

NTV layers NTV and JsonNTV structures are defined as shown in :

• NTV is a data triplet (NTVname, NTVtype, NTVvalue) that contains all the information needed to build the native entity.
• JsonNTV is the JsonValue of the NTV entity. JsonNTV is used to build an interchangeable JsonText

NTV layers | |--------->| NTV |--------->|native| |JsonText| |JsonNTV| |triplet| |entity| | |<----------| |<---------| |<---------| | +--------+ JSON dump +-------+ NTV dump +-------+ from_obj +------+ ]]>

Example:

• Native layer
  • new_year_2022 = datetime.date(2022, 01, 01) is a Python variable "new_year_2022" corresponding to a datetime object
• NTV layer
  • ("new_year_2022", "date", "2022-01-01") is a triplet
• JsonNTV layer
  • { "new_year_2022:date" : "2022-01-01" } is a JsonValue
• JsonText layer
  • '{"new_year_2022:date":"2022-01-01"}' is a JsonString

NTV entities Two categories of entities (one primitive and one structured) are defined:

• NTVsingle for the primitive entity,
• NTVlist for an ordered sequence of NTV entities

An NTV entity is therefore a tree where the leaves nodes are the NTVsingle entities and where the inner nodes are the NTVlist entities

NTVsingle The data triplet of NTVsingle entities is composed by:

• NTVsingleValue: This NTVvalue is the JSON representation (JsonValue) of the main content of the NTV entity,
• NTVsingleType: This NTVtype defines the type of entity and the conversion rules between the native entity and the NTV entity. The default value is the "json" NTVtype.
• NTVname: The NTVname is an additional textual information (JsonString)

In other words, any entity that has on the one hand a function of encoding it into a JsonValue and on the other hand a function of creating from a JsonValue can be taken into account. This approach is very general because the majority of computer objects are defined by a list of parameters (e.g. *args in python) and/or a list of key/values (e.g. **kwargs in python) which simply translate into a JsonArray or a JsonObject. The consistency between NTVsingleValue and NTVsingleType is outside the scope of this note.

NTVlist The data triplet of NTVlist entities is composed by:

• NTVlistValue: This NTVvalue is an ordered sequence of included NTV entities.
• NTVlistType: This NTVtype is a Namespace or a default NTVtype to apply to the NTV entities included. This NTVlistType avoids including a NTVtype (if default NTVtype) or reduces the length (if Namespace) in the JSON representation (JsonNTV) of the included NTV entities. The default value is "None".
• NTVname: The NTVname is an additional textual information (JsonString)

Example of equivalent JSON representations:

• where NTVlistType is None for the global NTVlist
  • [ { ":point" : [2.3522, 48.8566]}, {":point" : [4.8357, 45.7640]} ]
• where NTVlistType is "point" for the global NTVlist
  • { "::point" : [ [2.3522, 48.8566], [4.8357, 45.7640] ] }

If JSON-value is { "::dat" : ["2022-01-28T18-23-54", {":point": [1.1, 2.2] ] } }, the parsers deduce that the first NTVvalue has a "dat" NTVtype and the second a "point" NTVtype.

NTVtype and Namespace NTVtype is defined in a nested structure called Namespace:

• A NTVtype is defined in a parent Namespace,
• A Namespace is defined in a parent Namespace,

This structuring of type makes it possible to reference any type of data that has a JSON representation and to consolidate all the shared data structures within the same tree of types.

Namespace A Namespace is included in a Namespace parent (NamespaceParent) and may contain Namespace childs. A Namespace is represented by a string followed by a point (NamespaceName). The global Namespace (root node in the Namespace tree) is represented by an empty string and its NamespaceParent is None.

NTVtype A NTVtype is defined by a name (NTVtypeName) and a Namespace parent (NamespaceParent). The NTVtype is unique in the Namespace parent. NTVtype and the rules to encode or decode NTVvalues MUST be understood by data producers and data consumers. So NTVtype and rules associated have to be defined in a specification shared by a large community. On the other hand, it must be possible for everyone to share data according to their own data structure. There are therefore two categories of NTVtype:

• custom NTVtype (and Namespace) which can be created by anyone without control,
• shared NTVtype (and Namespace) that are defined in a single, shared repository.

The corresponding rules are as follows:

• NTVtype (or Namespace) whose name begins with '$' is of type 'custom'
• NTVtype (or Namespace) included in a Namespace of type 'custom' is also of type 'custom'
• Each 'shared' Namespace or NTVtype is uniquely managed.

Example:

• If "fr." is a Namespace attached to the global Namespace and containing the Namespace 'BAN.' and the NTVtype 'dep', then:
  • "fr.dep" is a shared NTVtype,
  • "fr.$test" is a custom NTVtype,
  • "fr.$example.one" is a custom NTVtype
  • "fr.BAN.$test" is a custom NTVtype

Three categories of NTVtype are defined (None, Simple, Generic).

• The "None" NTVtype is used with NTVlist to indicate the absence of a default Type.
• Simple NTVtype is associated to conversion rules between a native entity and a NTV entity.
• Generic NTVtype is equivalent to a set of Simple NTVtype. This indicates that parsers use an associated simple NTVtype to decode the JsonNTVvalue.

Example:

• "dat" is the generic NTVtype for "datetime" and "timeposix"
• If a JSONvalue is { "::dat" : ["2022-01-28T18-23-54", 123456.78] }, the parser deduces that the first entity has a "datetime" NTVtype and the second a "timeposix" NTVtype.

 

JsonNTV

JsonNTV format The JsonNTV format is the JsonValue representation of an NTV entity as defined in , which uses ABNF from .

JsonNTV - ABNF

The JsonNTV format is built with the NTVname, NTVvalue and the JsonNTVtype. Two JsonNTV formats are defined:

• named format (if NTVname or JsonType is not an empty string):
  • { JsonNTVname : JsonNTVvalue }
• simple format (if NTVname and JsonType are empty string):
  • JsonNTVvalue

This format allows full compatibility with existing JSON structures:

• a JsonNumber, JsonString, JsonNull or JsonBoolean is the representation of a Vsingle entity,
• a JsonObject with a single member is the representation of an NVsingle or TVsingle entity,
• a JsonArray is the representation of a Vlist entity,
• a JsonObject without a single member is the representation of a Vlist entity composed with unnamed entities.

Note :

• JsonNTV : 21, { ":" : 21 } and { ":json" : 21 } represent the same Vsingle entity,

JsonNTVname JsonNTVname is the concatenation of NTVname and JsonType. If the JsonNTVname contains one colon, the entity is a NTV-single. If the JsonNTVname contains two adjacent colons, the entity is an NTV-list. JsonNTVname is defined in , which uses ABNF from .

JsonNTVname - ABNF

The JsonSepType is composed with the separator singleSep or listSep and the JsonNTVtype.

• For NTVsingle entities:
  • The JsonSepType is identical to the singleSep joined with JsonNTVtype if JsonNTVtype is not "json".
  • If JsonNTVtype is "json" and JsonNTVvalue is not a JsonArray and not a JsonObject with not a single member, the JsonSepType is empty.
  • If JsonNTVtype is "json" and JsonNTVvalue is a JsonArray or a JsonObject with not a single member, the JsonSepType is ":".
• For NTVlist entities:
  • The JsonSepType is identical to the listSep joined with JsonNTVtype if JsonNTVtype is not "json".
  • If JsonNTVtype is "json" and JsonNTVvalue is not a JsonArray and not a JsonObject with not a single member, the JsonSepType is "::".
  • If JsonNTVtype is "json" and JsonNTVvalue is a JsonArray or a JsonObject with not a single member, the JsonSepType is empty.
• For NTV entities included in another entity, the JsonSepType MAY be set to :
  • The empty string if the NTVtype of the structured NTV entity is identical to the NTVtype of the NTV entity,
  • relative NTVtype if the NTVtype of the structured NTV entity is a Namespace shared with the NTVtype of the NTV entity.

JsonNTVtype The JSON representation of a NTVtype (JsonNTVtype) is composed by all the nested NamespaceName and the NTVtypeName. The JsonNTVtype is defined in , which uses ABNF from .

JsonNTVtype - ABNF

Example for an absolute representation of an NTVtype defined in two nested Namespace in the global Namespace:

• "ns1.ns2.type"
• where:
  • ns1. is a Namespace defined in the global Namespace,
  • ns2. is a Namespace defined in the ns1. Namespace,
  • type is a NTVtype defined in the ns2. Namespace

JsonNTVvalue The JsonNTVvalue is the JsonValue representation of NTVvalue as defined in , which uses ABNF from .

JsonNTVvalue - ABNF

For a NTVsingle, JsonNTVvalue is the NTVvalue. For a NTVlist, JsonNTValue has two representations:

• a JsonArray where JsonElements are the JsonNTV of included NTV entities,
• a JsonObject where the JsonMembers are the JsonMembers of the JsonNTV of included NTV entities.

Note:

• The JsonObject option is available only if all the NTV entities included have a JsonNTVname and all the JsonNTVname are different (e.g. {":point" : [2.3522, 48.8566], ":point" : [4.8357, 45.7640]} is not a valid JSON-value)

 

Examples Examples of JsonNTV representation of NTV entities: Vsingle :

• "lyon"
• 52.5
• { ":" : [4.8357, 45.7640] }

NVsingle :

• { "city" : "paris" }
• { "paris:" : [4.8357, 45.7640] }

TVsingle:

• { ":point" : [4.8357, 45.7640] }

NTVsingle:

• { "paris:point" : [2.3522, 48.8566] }
• { "paris:ntv" : { "coordinate:point" : [4.8357, 45.7640] } }

Vlist (composed with JsonArray):

• [4, 45]
• ["paris"]
• [ [2.3522, 48.8566], {"lyon" : [4.8357, 45.7640]} ]
• [ { ":point" : [2.3522, 48.8566]}, {":point" : [4.8357, 45.7640]} ]
• []

Vlist (composed with JsonObject) :

• { "name": "white", "firstname":"walter", "surname":"heisenberg" }
• { "paris:point" : [2.3522, 48.8566] , "lyon" : "france"}
• { "paris" : [2.3522, 48.8566], "" : [4.8357, 45.7640]}
• { }

NVlist :

• { "simple list": [4, 45.7]}
• { "cities": [{":point": [2.3522, 48.8566]}, {":point": [4.8357, 45.7640]}]}

TVlist :

• { "::point": [[2.3522, 48.8566], {"lyon" : [4.8357, 45.7640]} ] }
• { "::dat": [ "2022-01-28T18-23-54Z", "2022-01-28", 1234.78 ] }

NTVlist :

• { "cities::point": [[2.3522, 48.8566], {"lyon": [4.8357, 45.7640]}] }
• { "generic date::dat": [ "2022-01-28T18-23-54Z", "2022-01-28", 1234.78]}
• { "various::point": [ [2.3, 48.8], { ":date": "2022-01-28"}, {"json": 25 }, { "another NTVlist": [1,2,3]} ]}

NTVlist and NVlist (composed with JsonObject) :

• { "cities::point": { "paris": [2.352, 48.856], "lyon": [4.835, 45.764]}}
• { "cities": {"paris:point": [2.3522, 48.8566] , "lyon": "france"}}

 

Parsing a JsonValue A first decoding step identifies the following objects:

• JsonNTVvalue, NTVname (if present), separator (if present), JsonNTVtype (if present).

The tables below present the rules for identifying an NTV entity. Decoding JsonValue

JsonValue	NTV entity
JsonPrimitive	Vsingle
JsonUnnamed	Vlist
JsonArray	Vlist
JsonNamed	see

Decoding JsonNTVvalue

Separator	JsonNTVvalue	NTV entity
None	JsonPrimitive	NVsingle
None	JsonNamed	NVsingle
None	JsonUnnamed	NVlist
None	JsonArray	NVlist
":"	JsonValue	TVsingle or NTVSingle
"::"	JsonUnnamed	NVlist or TVlist or NTVlist
"::"	JsonArray	NVlist or TVlist or NTVlist

The NTVvalue of NTVsingle is the JsonNTVvalue. The NTVvalue of NTVlist is build by decoding data included in JsonNTVvalue. For NTV entities not included in a NTVlist, the NTVtype is built with:

• "json" JsonNTVtype if JsonNTVtype is None and if the NTV entity is a NTVsingle,
• the NTVtype build with the JsonNTVType.

For NTV entities included in a NTVlist, the NTVtype is built with :

• the JsonNTVtype if it is an absolute NTVtype and if the entity is a NTVsingle,
• the concatenation of the JsonNTVtype of the NTVlist and the JsonNTVtype (if it is a relative NTVtype),
• the JsonNTVtype of the NTVlist if the JsonNTVtype is None.

If the resulting NTVtype is inconsistent, the NTVtype is set to None. The repository gives some examples of NTV usage.

 

NTV extended structure The NTVvalue of NTV entities are JsonValue. In the NTV extended structure, NTVvalue can be every kind of data. With this structure, the NTV representation is a "json like" data where:

• JsonNTVvalue is always NTVvalue,
• JsonNTVtype is optional (if NTVtype can be deduced from the NTVvalue),
• JsonNTVname is optional (if NTVname can be deduced from the NTVvalue).

Examples of NTV extended representation:

• • { "date object" : [ datetime.date(2022, 1, 18), datetime.date(2023, 1, 12) ] }
• this "json like" structure can be converted in JsonNTV format:
• • { "date object::date" : [ "2022-01-18", "2023-01-12" ] }

 

IANA Considerations Any JsonValue is a JsonNTVValue and conversely, any JsonNTVvalue is a JsonValue. Thus, any JSON data may or may not be treated as JsonNTV data, so there is no need to create a specific MIME media type for JsonNTV. All properties of the MIME media type "application/json" are applicable.

Security Considerations The format used for NTV data exchanges is the JSON format. So, all the security considerations of apply. The NTV structure provides no cryptographic integrity protection of any kind.

  References Normative References Informative References "Google" "W3C" "ISO"  

Appendix 1 - Global NTVtype and Namespace The structure of NTVtype by Namespace makes it possible to have NTVtype or Namespace corresponding to recognized standards at the global level. A standard NTVtype / Namespace is a NTVtype / Namespace defined in the Global Namespace. The standard NTVtype and Namespace are listed below.

Json Json NTVtype have a generic NTVtype : "json" Json NTVtype

NTVtypeName (generic)	NTVvalue	example NTVvalue
json	generic NTVtype
number (json)	JsonNumber	10
boolean (json)	JsonBoolean	"true"
null (json)	JsonNull	"null"
string (json)	JsonString	"value"
array (json)	JsonArray	[1.1, 2.2]
object (json)	JsonObject	{"value1": 1, "value2": 2}

Numbers Numbers NTVtype

NTVtypeName	NTVvalue	comment
int	JsonNumber	integer
int8, int16, int32, int64	JsonNumber	signed integer
uint8, uint16, uint32, uint64	JsonNumber	unsigned integer
float	JsonNumber	floating point real
float16, float32, float64	JsonNumber	floating point real

Datation Datation NTVtype have a generic NTVtype : "dat" Datation NTVtype

NTVtypeName (generic)	NTVvalue	example NTVvalue
year	fullyear	1998
month	month	10
day	mday day of month	21
wday	wday day of week	7
yday	yday day of year	360
week	week	38
hour	hour	20
minute	minute	18
second	second	54
dat	generic NTVtype
date (dat)	date	"2022-01-28""
time (dat)	timespec-base [time-fraction]	"T18:23:54", "18:23", "T18"
timetz (dat)	timespec-base [time-fraction] time-zone	"T18:23:54+0400"
datetime (dat)	iso-date-time (without time-zone)	"2022-01-28T18-23-54"
datetimetz (dat)	iso-date-time (with time-zone)	"2022-01-28T18-23-54+0400"

Period and Duration Period and Duration NTVtype

NTVtypeName	NTVvalue	example NTVvalue
duration	duration	"P3Y6M4DT12H30M5S"
timearray	JsonArray	[dat1, dat2]dat1, dat2 have "dat" NTVtypeName
period	period	"2022-01-01 / 2022-01-30""2022-01-01 / P3Y6M4DT12H30M5S"

Location Location NTVtype have a generic NTVtype : "loc". The CRS (Coordinate Reference Systems) is geographic, using the World Geodetic System 1984 (WGS 84) datum, with longitude and latitude units of decimal degrees (EPSG:4326). Location NTVtype

NTVtypeName (generic)	NTVvalue	example NTVvalue
loc	generic NTVtype
point (loc)	Point coordinates	[ 5.12, 45.256 ] (lon, lat)
line (loc)	LineString coordinates	[pt1, pt2, pt3]ptx has "point" NTVtypeName
polygon (loc)	Polygon coordinates	[rg1, rg2, rg3]rgx has "line" NTVtypeName
multipolygon (loc)	MultiPolygon coordinates	[pl1, pl2, pl3]plx has "polygon" NTVtypeName
bbox (loc)	Bbox coordinates	[ -10.0, -10.0, 10.0, 10.0 ]
geojson	geoJSON object	{"type": "point", "coordinates": [40.0, 0.0]}
codeolc	Open Location Code	"8FW4V75V+8F6"

Structured data Structured NTVtype

NTVtypeName	NTVvalue
row	row
field	column
tab	table
ntv	JsonNTV

The data structure associated to this NTVtypeName are defined in specific document.

Normalized String Structured NTVtype

NTVtypeName	NTVvalue	example NTVvalue
uri	URI (RFC3986)	"https://www.ietf.org/rfc/rfc3986.txt" "urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6" "geo:13.4125,103.86673" (RFC5870)""info:eu-repo/dai/nl/12345""mailto:John.Doe@example.com" "news:comp.infosystems.www.servers.unix"
email	adress (RFC5322)	"John Doe <jdoe@machine.example>"
file	file-hier-part (RFC8089)	"///path/to/file""//host.example.com/path/to/file"

Keywords are not defined as Normalized String (eg. "id", "mandatory", "units"), they can be used as custom NTVtype (eg. "$id", "$mandatory", "$units")

Namespace The global Namespace includes Namespaces for countries, dependent territories and special areas as defined in The JsonNamespace for those Namespace is composed by the two digits of the country following by a dot. Example :

• "fr." is the France JsonNamespace
• "un." is The United Nations JsonNamespace

Each Namespace defines a list of included NTVtype and Namespace.

Custom NTVtype and Namespace Custom NTVtype and Namespace can be created in any Namespace. below presents some examples of custom NTVtype. Examples of custom NTVtype

JsonNTVtype	comment or JsonNTV example
"$id"	defined in the global Namespaceeg. { ":$id": 5426849" }
"$iata"	IATA airport codeeg. {"Paris Nord:$iata": "CDG"}
"$uic.station"	UIC station codeeg. {"Nantes station:$uic.station" : "8748100"}
"fr.$city"	NTVtype "city" in "fr." Namespaceeg. {":fr.$city" : "Paris"}
"$schemaorg."	"schemaorg" catalogeg. { ":$schemaorg.propertyID": "NO2" } { ":$schemaorg.unitText": "mg/m3"}
"$darwincore."	"darwincore" catalogeg. { ":$darwincore.acceptedNameUsage": "Tamias minimus" }

Appendix 2 - Complete ABNF notation

Collected ABNF grammar

Acknowledgements TBD

Contributors Thanks to all of the contributors.