Implement support for complex field types in the PLC4X API (PLC4X-113)

[Łukasz Dywicki <lu...@code-house.org>]

I think that going with a 'DataType' which defines is primitive/map/list
methods is not a valid start. What's gonna happen if we will get a
java.util.Date and all surrounding variant types?
My point is that this part of library should leave a lot of freedom for
type definitions as different PLCs will definitely have their own edge
cases to be covered.
If you will start with collections at beginning you will quickly run
into troubles when you will only think of generics. Top level type
representing a "data type" needs to stay unaware of specific types for
clarity of code and stability of APIs.

When I think of type system I see following variants (don't mind it -
its just sample pulled out from hat):
Type<T>
 ` ContainerType<T> (map, list, set, array)
 ` GenericType<T> (map, list, set, array?, json?)
 ` KeyValueType (map, struct)
 ` CollectionType<T>
   ` SetType<T> extends CollectionType<Set<T>>
   ` ListType<T> extends CollectionType<List<T>>
 ` PrimitiveType<T>
  ` ArrayType // built into JVM, primitive container
  ` Int32Type
 ` JavaType<T> // user supplied types for mapping purposes
  ` EnumType<T extends Enum<T>>
  ` LongType
  ` UTF8Type<String>
    ` JsonType<T> // mapped value type, not a field type!

Primary role of fairly free "type" definition of the top is
composability at bottom levels. A numeric types can be reflected by a
NumberType and so on. I can also think of very special kinds of
structures which PLCs can support such structs with names or not or
various mutable or not counter types. Above structure leaves space to
support these.

My conclusion in this area come from earlier Cassandra background.
Cassandra type mappings at infrastructure level have similar concept to
what you linked. Core (marshal) types and their CQL driver
representations follow similar concept and it works. However at our
mapper layer which we did back in 2013 we started from generics.

```
public interface Type<T> {
    Serializer<T> getSerializer(); // handle to/from byte sequence
    String getTypeName(); // will we need that at all for plcs?
}
```

After several months we got more elements into schema which come from
needs we had and growing Cassandra abilities.

Over time we ended up with below type definitions:
```
// Type which can be mapped 1:1 to Java class and object fields
public interface JavaType<T> extends Type<T> {
    Class<T> getJavaType();
}

// Parametrized type ie. List<String> but also Identifiable<Long>
interface GenericJavaType<T> extends JavaType<T> {
    List<Class<?>> getTypeArguments();
}

// Edge case type which requires special handling
public interface ComplexType<T> extends Type<T> {
}
```

Because we worked with Java all type definitions required generic type
bound. This was something which we introduced on our own as Cassandra
type system didn't provide that and we needed that to build object
mapping layer. If you think of evolution of OPM module then generics
will turn to be a quite nice addon which calcite seems to skip.

Collection types which you mentioned in example link can be reflected by
separate Map/List/Set types as shown in beginning and verified via
instanceof operator. They could be also embedded in interface with
"default" implementations returning false, however my small advice would
be to not do that.

This are my ½€ :-)

Cheers,
Łukasz

On 18.04.2019 09:27, Julian Feinauer (JIRA) wrote:
> Julian Feinauer created PLC4X-113:
> -------------------------------------
> 
>              Summary: Implement support for complex field types in the PLC4X API
>                  Key: PLC4X-113
>                  URL: https://issues.apache.org/jira/browse/PLC4X-113
>              Project: Apache PLC4X
>           Issue Type: New Feature
>           Components: API
>             Reporter: Julian Feinauer
> 
> 
> *This is a (possibly) incompatible change.*
> 
> Currently we have support for primitives as well as lists (arrays) as first class "ciitzens" of the PLC4X model.
> Especially, when thinking about OPC UA, but also other protocols like ADS (and even S7) support complex objects in their programming model.
> Thus, it is necessary to refactor / extend the API to support these complex objects.
> 
> I guess we need at least
> * primitives
> * lists
> * maps (should be equal to "structs")
> 
> Basically, this should be roughly equivalent to what one can represent with JSON.
> Thus, the API could be structured i two ways.. first, we could use an internal Node Tree to represent the objects, but we could also provide to use someone elses NodeTree or structure to represent the Objects.
> For example Jackson.. this would even allow us to directly map these objects to POJOs via Jackson.
> 
> As example for such a model, one can look at that of Apache Calcite: https://github.com/apache/calcite/blob/3fa29455664bec0056c436491b369e0cd72242ea/core/src/main/java/org/apache/calcite/rel/type/RelDataType.java
> 
> They have an interface with methods
> 
> 
> {code:java}
> public interface DataType
> {
>     // Checker
>     boolean isMap;
>     boolean isList;
>     boolean isPrimitive;
> 
>     // Getter for complex types
>     DataType get(String key);
>     DataType get(Long i);
>     
>     // Primitive getters
>     long getLong();
>     String getString();
> 
>     //...
> }
> {code}
> 
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>