# Schema Organization

The data in Articonf is organized in Use-Cases, Tables, Layers and Nodes. That way, a clear seperation of schema information and actual application data is ensured.

# Trace

A trace is an incoming piece of data. It is a JSON encoded object that can be nested arbitrarily deep. This raw datapoint is the input in Articonf. There are no rules to the format of the document except that it has to be an object ```{}``` and must not be a list ```[]```.

__Example:__ A running example will be a pizza shop, that provides its data to Articonf. The shop serves different pizzas and offers a loyalty program where he needs to protocol customers at every visit.

A trace for the shop could look like this: 

    {
		"type": "blockchain-transaction",
		"ApplicationType": "pizzashop",
		"docType": "pizza",
		"id": 1,
		"name": "Diavolo",
		"dough": {
			"type": "wheat",
			"spinach": false
		},
		"toppings": [
			{
				"name": "Tomato Sauce",
				"price": 1.00
			},
			{
				"name": "Cheese",
				"price": 0.50
			},
			{
				"name": "Chilli Oil",
				"price": 0.50
			},
			{
				"name": "Peppers",
				"price": 1.50
			}
		]
	}

# Use-Cases

Each trace is assigned to exactly one Use-Case, a label which assigns the piece of data to a Third-Party data source. This Use-Case has to be unique in the system, it is not allowed that there exist two different Use-Cases with the same label.

__Example:__ The field ```ApplicationType``` serves as the Use-Case of trace. In the example above, that would be ```pizzashop```.

# Tables

Tables describe how incoming data gets translated into flattened objects for further processing. One table is used for one kind of incoming datum and one Use-Case can have multiple tables. The incoming JSON data can be nested arbitrarily deep, the job of the table is to *flatten* it in a way, that the data is no longer nested. In the trace, the field ```docType``` serves as the identifier for the table used.

The flattening of a trace with a table results in an object that is no longer nested and that does not contain any lists. Each table must also contain a mapping with the key ```UniqueID```. This property is later used to uniquely identify a trace. The value obtained in this field is [hashed](https://en.wikipedia.org/wiki/Hash_function) afterwards, resulting in an seemingly random string that identifies a trace.

## Flattening Language

To describe the flattening of nested JSON data, a simple syntax was implemented comprising of the following operators:

- __//__ ... With the // operator, a path through a nested object can be described. ```"doughType": "dough//type"``` means that the flattened object will contain a property called "doughType" which gets filled with the value ```type``` of the ```dough``` object in the trace.

- __+__ ... The + Operator concatenates values from the trace. The datatype of a property including the ```+``` operator will automatically become ```String```. ```"fullName": "name+description"``` means that the flattened object will contain a property called ```fullName``` which is the concatenation of the values ```name``` and ```description``` in the trace.

- __[0]__ ... The index operator is used to access an element in a list. ```"firstTopping": "toppings[0]//name"``` means that the flattened object will contain a property called ```firstTopping``` which is the ```name``` property of the first entry of the list ```toppings``` in the trace. The ```0``` here can be replaced with any valid numeric python list index.

__Example:__ The pizza shop has two data kinds: The pizza orders and customer visits. So the shop will add 2 tables, which are called "pizza" and "customer".

The table could look like this:

    {
        "use_case": "pizzashop",
        "name": "pizza",
        "mappings": {
            "UniqueID": "name",
            "name": "name",
            "doughType": "dough//type",
            "fullName": "name+description",
            "firstTopping": "toppings[0]//name"
        }
    }

The properties ```use_case``` and ```name``` indicate the Use-Case the table belongs to and the tables name. ```mappings``` is the map that indicates how the flattened object is constructed out of the trace, in this example the resulting object will have 4 properties.

# Layers

Once the data has been added to Articonf, the layers take *slices* out of the flattened traces. This is a preproccessing step needed in order to perform the clustering later. Each layers holds two lists: ```properties``` and ```cluster_properties```. The first one selects a set of properties from the underlying table which are the properties stored in the layer while the latter one  marks a subset of these properties as *properties to be clustered after* later on. 

__Example:__ For the ```pizza``` table, a layer was defined which looks like this:

    {
        "use_case": "pizzashop",
        "table": "pizza",
        "name": "name_layer",
        "properties": [
            "UniqueID",
            "name",
        ],
        "cluster_properties": [
            "name"
        ],
    }

Here, ```UniqueID``` and ```name``` were selected as attributes for the layer, but only ```name``` is a cluster attribute.

# Nodes

Finally, a node is a data point which belongs to a layer. It contains the data from the trace that is selected by the overlaying layer. This is the most fine-grained result of the initial trace.


__Example:__ The node for the initial trace of the pizzashop and the layer ```name_layer``` above would look like this:

    {
        "use_case": "pizzashop",
        "layer": "layer_name",
        "name": "pizza",
        "mappings": {
            "UniqueID": "name",
            "name": "name",
            "doughType": "dough//type",
            "fullName": "name+description",
            "firstTopping": "toppings[0]//name"
        }
    }