Awesome
JSON Schema Profile
The goal of JSON Schema Profile is to augment the vocabulary of JSON Schema to represent properties of the data as opposed to focusing only on the structure.
Definitions
Bloom filter
This is a string which represents a serialized Bloom filter. Currently this is a Base64 encoded serialized value of the specific Bloom filter class used by JSONoid, but we plan to make this a more reusable format.
Bloom filters are useful to check if specific values were observed for a particular property without the need to store all the values.
Histogram
| property | description |
|---|
bins | An array of two-element arrays where the first element is the mean of the bin and the second is the number of elements in the bin |
hasExtremeValues | A Boolean indicating whether the histogram contains values which cannot be represented in the given bounds. This usually only occurs for extremely large absolute values and is rarely observed in practice |
Statistics
| property | description |
|---|
variance | The variance of all values of this property |
stdev | The standard deviation of all values of this property |
skewness | The skewness of all values of this property |
kurtosis | The kurtosis of all values of this property |
Arrays
| property | description |
|---|
lengthHistogram | A histogram of array lengths |
Booleans
| property | description |
|---|
pctTrue | Percentage of the Boolean values which are true |
Integers
| property | description |
|---|
bloomFilter | A Bloom filter of integer values |
distinctValues | An estimate of the number of distinct values (cardinality) of this property |
histogram | A histogram of integer values |
statistics | A set of statistics of integer values |
Numbers
| property | description |
|---|
bloomFilter | A Bloom filter of number values |
distinctValues | An estimate of the number of distinct values (cardinality) of this property |
histogram | A histogram of number values |
statistics | A set of statistics of number values |
Objects
| property | description |
|---|
fieldPresence | An object where the value represents the percentage of the time the corresponding key appears |
Strings
| property | description |
|---|
bloomFilter | A Bloom filter of string values |
distinctValues | An estimate of the number of distinct values (cardinality) of this property |
lengthHistogram | A histogram of string lengths |