You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Dominik Riemer <ri...@apache.org> on 2020/08/11 20:34:30 UTC
Re: Core-UI serialization
Hi,
I've now completed the Core/UI serialization changes and wrote a short documentation that explains how to create Typescript models from Java models:
https://cwiki.apache.org/confluence/display/STREAMPIPES/Core-UI+Serialization
Feel free to ask if you have any questions or something is unclear!
Dominik
On 2020/06/18 06:34:11, Dominik Riemer <ri...@apache.org> wrote:
> Hi Philipp,
>
> yes, either one or two, currently it's two (one for streampipes-model and one for streampipes-model-client), see [1].
>
> Dominik
>
> [1] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-145/ui/src/app/core-model/gen
>
> On 2020/06/17 09:05:36, Philipp Zehnder <ze...@fzi.de> wrote:
> > Hi Dominik,
> >
> > the code generation looks good and I look forward to use it.
> >
> > Do I understand it correctly, there will be one file containing the whole generated model, right?
> >
> >
> > Philipp
> >
> > > On 10. Jun 2020, at 18:00, Dominik Riemer <ri...@apache.org> wrote:
> > >
> > > Hi Philipp,
> > >
> > > coming back to your questions after doing more experiments and code refactoring:
> > >
> > > * I integrated a Maven Plugin [1] to streampipes-model and streampipes-model-client. It creates typescript files by executing mvn typescript-generator:generate and outputs a single file to target/typescript-generator.
> > > * I added an annotation @TsModel, the plugin picks up every class with this annotation and generates typescript classes [2].
> > > * There is a folder core-model/gen in the UI where model files need to be copied to [3]
> > > * The serialization layer is now based on JSON/Jackson, it's still WIP and there are quite a few things that need to be modified in the UI. I'm gradually replacing @Gson annotations in the REST interfaces with @JacksonSerialized so that all models can be easily deserialized using some utility methods provided by the typescript-generator
> > > * To support serialization of polymorphic classes (e.g., lists of static properties), type hints need to be added to the abstract class, see an example here [4]. This is similar to the GSON adapters we had before.
> > > * So coming back to your questions, in case any data models served by our REST interfaces are changed, the typescript-generator needs to be run (copying can probably be automated). In case classes need to be changed in the UI you can either cast types to <any> or create wrapper objects. In general, the new approach should make UI development much less error-prone.
> > >
> > > It would be great if you all have a look - feedback is welcome!
> > >
> > > Dominik
> > >
> > >
> > > [1] https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/pom.xml <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/pom.xml>
> > > [2] https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model-shared/src/main/java/org/apache/streampipes/model/shared/annotation/TsModel.java <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model-shared/src/main/java/org/apache/streampipes/model/shared/annotation/TsModel.java>
> > > [3] https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/ui/src/app/core-model/gen/streampipes-model.ts <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/ui/src/app/core-model/gen/streampipes-model.ts>
> > > [4] https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/src/main/java/org/apache/streampipes/model/schema/EventProperty.java <https://github.com/apache/incubator-streampipes/blob/STREAMPIPES-145/streampipes-model/src/main/java/org/apache/streampipes/model/schema/EventProperty.java>
> > >
> > > On 2020/06/05 08:19:47, Philipp Zehnder <zehnder@fzi.de <ma...@fzi.de>> wrote:
> > >> Hi Dominik,
> > >>
> > >> +1
> > >> I totally agree, we need a unified solution for the communication of the backend and front-end.
> > >>
> > >> I have question regarding the TypeScript code generation. You mentioned we could generate the classes in a Maven build, right?
> > >> How would the development of the UI then look like? How can a UI developer generate those classes and can they be changed or are they immutable?
> > >>
> > >> Philipp
> > >>
> > >>> On 2. Jun 2020, at 23:33, Dominik Riemer <ri...@apache.org> wrote:
> > >>>
> > >>> Hi all,
> > >>>
> > >>>
> > >>>
> > >>> during the last days, while starting to migrate the editor UI module to
> > >>> Angular, I did some experiments with different serialization approaches.
> > >>>
> > >>>
> > >>>
> > >>> Currently, we use two different serialization approaches in the UI:
> > >>>
> > >>> 1. JSON for most things and the editor module also uses JSON to
> > >>> represent static properties (which are used to render the UI components in
> > >>> the pipeline element configuration)
> > >>> 2. The (newer) Connect module uses JSON-LD (which is also used in the
> > >>> core to represent pipeline elements) to represent static properties.
> > >>>
> > >>>
> > >>>
> > >>> Both approaches have advantages and disadvantages:
> > >>>
> > >>> For (1), we use GSON for serializing JSON, which is easy and fast for most
> > >>> cases, but there is no direct mapping to TypeScript classes for complex
> > >>> objects (like static property definitions with polymorphism), so that we
> > >>> rely on the <any> type here in most cases (especially in the editor module)
> > >>> -> all type information is lost and UI development is more error-prone.
> > >>>
> > >>> For (2), we use a self-made JSON-LD serializer named Tson-LD. This works
> > >>> great, but TypeScript-classes need to be manually modified every time
> > >>> classes in streampipes-core change, which increases maintenance effort and
> > >>> also is rather error-prone. On the other hand, Tson-LD deserializes JSON-LD
> > >>> into TypeScript classes, which is quite convenient from a developer
> > >>> perspective.
> > >>>
> > >>>
> > >>>
> > >>> As we will soon be able to completely remove AngularJS from the UI, it also
> > >>> makes sense to rethink and streamline our serialization approach (mainly to
> > >>> have a single component that cares about generating UI components from
> > >>> static properties, both in Connect and the Pipeline Editor).
> > >>>
> > >>>
> > >>>
> > >>> So after doing some research and testing I'd propose the following approach:
> > >>>
> > >>> * Use only JSON as a serialization format for UI-core communication
> > >>> and remove JSON-LD from the UI
> > >>> * Use Jackson over GSON as a serialization library, as this better
> > >>> works with the next point:
> > >>> * Use typescript-generator [1] to auto-generate TypeScript classes
> > >>> from Java models. There is not much configuration needed and there is a
> > >>> Maven plugin which could automatically run the code generation at every
> > >>> build.
> > >>>
> > >>>
> > >>>
> > >>> This approach would have less overhead, guarantees that all models are
> > >>> always up-to-date and reduces maintenance effort. Drawback is that quite a
> > >>> few UI modules would need to be changed.
> > >>>
> > >>>
> > >>>
> > >>> I have already a prototype running which shows that the approach works quite
> > >>> well. But before I continue, I'm interested whether you think this is a good
> > >>> or bad idea and if you have any other best practice approaches?
> > >>>
> > >>>
> > >>>
> > >>> Dominik
> > >>>
> > >>>
> > >>>
> > >>> [1] <https://github.com/vojtechhabarta/typescript-generator>
> > >>> https://github.com/vojtechhabarta/typescript-generator
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>
> > >> .........................................................
> > >> M. Sc. Philipp Zehnder
> > >> Wissenschaftlicher Mitarbeiter | Research Scientist
> > >> Information Process Engineering (IPE)
> > >>
> > >> FZI Forschungszentrum Informatik
> > >> Haid-und-Neu-Str. 10–14
> > >> 76131 Karlsruhe, Germany
> > >> Tel.: +49 721 9654-805
> > >> Fax: +49 721 9654-806
> > >>
> > >> zehnder@fzi.de <ma...@fzi.de> <mailto:zehnder@fzi.de <ma...@fzi.de>>
> > >> https://www.fzi.de/mitarbeiter/philipp-zehnder
> > >>
> > >> .........................................................
> > >> FZI Forschungszentrum Informatik
> > >> Stiftung des bürgerlichen Rechts
> > >> Stiftung Az: 14-0563.1 Regierungspräsidium Karlsruhe
> > >> Vorstand: Prof. Dr. Andreas Oberweis, Jan Wiesenberger, Prof. Dr.-Ing. J. Marius Zöllner
> > >> Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus
> > >> .........................................................
> >
> > .........................................................
> > M. Sc. Philipp Zehnder
> > Wissenschaftlicher Mitarbeiter | Research Scientist
> > Information Process Engineering (IPE)
> >
> > FZI Forschungszentrum Informatik
> > Haid-und-Neu-Str. 10–14
> > 76131 Karlsruhe, Germany
> > Tel.: +49 721 9654-805
> > Fax: +49 721 9654-806
> >
> > zehnder@fzi.de <ma...@fzi.de>
> > https://www.fzi.de/mitarbeiter/philipp-zehnder
> >
> > .........................................................
> > FZI Forschungszentrum Informatik
> > Stiftung des bürgerlichen Rechts
> > Stiftung Az: 14-0563.1 Regierungspräsidium Karlsruhe
> > Vorstand: Prof. Dr. Andreas Oberweis, Jan Wiesenberger, Prof. Dr.-Ing. J. Marius Zöllner
> > Vorsitzender des Kuratoriums: Ministerialdirigent Günther Leßnerkraus
> > .........................................................
> >
> >
>