You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ode.apache.org by Aaron Anderson <aa...@acm.org> on 2018/11/02 04:57:36 UTC

Apache ODE 2.0 Proposal

Hi All,
Here are my general thoughts on a possible architecture for a potential new version of ODE. It is by no means complete and any feedback is welcomed.
Thanks,
Aaron

Apache ODE 2.0 Development Proposal



Motivation
There is a great demand in the marketplace for DigitalTransformation [1] – marrying best of breed Cloud based system toincrease automation and empower intelligent business decision making.An underlying facet of this drive towards automation is businessprocess management. While there are many available Open Source BPMplatforms many use legacy architectures and technologies that do notscale in a Cloud environment. Most are also often designed for aspecific process format such as BPMN and support for extension andalternative specifications are an afterthought. Commercial BPMsolutions are extremely costly. Apache ODE 2.0 can satisfy the marketdemand for a scalable and flexible Open Source process orchestrationplatform that many can benefit from.


Guiding Principles
   
   - Cloud Architecture – ODE 2.0 should adhere to the Cloud Architecture Goals listed here [2]. This goal can be realized using Apache Ignite as the foundation of the system. ODE developers should focus on BPM support and work with the Apache Ignite project identify and remedy any deficiencies or defects with performance and scalability if they occur.  
   - General Process Management – ODE 2.0’s main focus should be to provide generic framework of common business process features that can be used or not by extension modules. ODE 2.0 should not be an academic exercise in mapping processes to a universal process object model; instead it should be considered an extension of Apache Ignite for business process orchestration requirements.  
   - Extensibility – Apache ODE should not only be capable of providing support for existing BPM specifications but it also should be future proof and be capable of supporting emerging standards or proprietary orchestration requirements. If the question is “I would like to implement process management specification XYZ” the answer should be Apache ODE 2.0. 

Key Terminology
   
   - Persistent Process – A long running process that has a beginning and end that can span multiple systems be activated and passivated. 
   - Dialect – A representation of process orchestration, such as BPEL, BPMN, SCXML, etc. Dialects typically need to be interpreted for execution. 
   - Identifier – Everything in ODE 2.0 is addressed using a Qname (namespace and local part) or generated GUID which can be used interchangeably. 
   - Module – An Apache ODE 2.0 extension that is bundled as a single JAR achive in the server classpath. A module may contain a dialect compiler, processor, numerous instructions, exchanges, CLI commands, etc. The ODE runtime will discover implementations and register them accordingly. A module may have dependencies on other modules and can include it’s own CDI extension to support it’s own extensions. 
   - Tenant – Support multitenancy [3] through isolated Ignite clusters [4].  
   - Endpoint Registry – Every process interacts with other processes or external systems. Managing which endpoints are bound to each other is a complex task and central registry should be maintained. The registry can map a logical endpoint address (Qname) to a physical URL. 
   - Exchange – ODE 2.0 processes will interact with external entities. Exchanges are mediation points were external requests and responses, like HTTP requests, are converted into Ignite computing requests that in turn are processed by ODE or module implementations. 
   - Assembly – A unit of deployment in a JAR format. It may contain Java code and libraries, static resources such as single page application HTML files, or dialect representations like executable BPMN xml files. Assemblies have a YAML descriptor file or necessary metadata is extracted through annotations and CDI inspection. Assemblies may have dependencies on other assemblies. 
   - Composite - A configuration of an assembly. Concrete bindings are provided for all of the registered assembly endpoints. All required assembly configuration is also provided. Composites are environment specific so there may be a dev/qa/prod composite variants. 
   - Compiler – A dialect specific lexical analyzer that converts a dialect file such as BPMN into a sequence of instructions. As the compiler parses the dialect file extensions can indicate or override how the instructions are ordered and configured. 
   - Instruction – A CDI annotated Java class that interacts through Ignite with ODE or assembly services to create, mutate, or destroy persisted state in Ignite. Instructions should be single purpose so that they can be rearanged in different execution order. Instructions will be provided contextual references through CDI or Ignite lookups to perform discrete processing tasks. Instructions should be annotated with source line and column numbers to easily correlate them back dialect source files for troubleshooting or diagnostic purposes. 
   - Executable – A namespace aware XML representation of a sequence of instructions arranged in blocks. EXI or FI may be used to compress the representation for performance purposes.  
   - Processor – A dialect specific interpreter that is activated on demand that executes instructions in sequence. The processor sets up the executational context the instructions are invoked in and acts on directives issued by the instructions, like create a process, destroy it, activate, passivate, etc.

Major Features
   
   - Virtual Processor – Implements a primitive metaphor everyone in computing is familiar with. A dialect implementer is free to fracture [5] source lines through a dialect specific compiler into a sequence of instruction blocks. Upon activation through an enpoint the dialect specific processor is activated and executes each instruction in sequence. There is no ODE 2.0 standard for process storage in ignite, common instructions, or executable XML file format. Instead ODE 2.0 provides optional utilities for assisting with these tasks that can be extended or replaced. For instance a StAX xml parser that supports registered extensions for Qnames, Ignite storage functions for creating/updating/destroying processes, etc. 
   - Cloud Scale – Apache Ignite provides all the necessary capabilities and more. 
   - Modules – ODE or other module extensions that are added to the server classpath and are discovered and registered through CDI.  
   - Assembly – A bundle of executable code or content that is intended to be deployed as a service to ignite. Assemblies are jar files akin to Tomcat web applications. They are stored in Ignite as binary objects and are lazy loaded and extracted to a temp file on Ignite nodes after they are deployed for easy and efficient access. As dynamic deployable unit’s assemblies have their own classloader and CDI BeanManager. The assembly’s bean manager may delegate to the parent server classloader and bean manager to inject select API instances but in general the server classpath is shielded from the assembly [6]. Assemblies with dependencies will be loaded in the same classpath. This can help with deploying a single shared dependency like a jar file or WSDL file and have it shared across multiple assemblies. Assemblies are deployed, undeployed, released, and recalled. If the assembly contains a dialect source file it also supports build, clean, and verify.  
   - Composite – Composites are the configuration for one or more assemblies. They contain environment specific settings such as bindings for physical addresses and compiler preferences. Composites are created, updated, deleted, activated, and deactivated. 
   - Management – All management is performed through CLI like other Cloud service providers. The CLI bootstrap entry point creates a custom classloader and bean manager so only CLI specific instances are loaded. Any necessary configuration files are in the YAML format. Activated composites can be instrumented with diagnostic configuration such as state logging or break points. 





Technologies
   
   - Apache Ignite - While Apache Ignite offers a variety of distributed computing functionality significant capabilities include persistence [7], indexing (ACID SQL 99 support) [8], and eventing [9]. 
   - OpenWebBeans - CDI scopes, extensions 
   - Apache Tomcat - HTTP server 
   - Apache CXF – REST, OIDC support 
   - STaX and FastInfoset or Apache Avro for process executable format 
   - YAML - composite application descriptor
   - Apache CLI

References 
   
   -  https://www.forbes.com/sites/jimmiller/2018/08/13/what-is-a-digital-transformation-and-why-should-you-care/#65ed41c57d35 
   - http://blog.grapeup.com/read/yet-another-look-at-cloud-native-apps-architecture-33  
   -  https://en.wikipedia.org/wiki/Multitenancy 
   -  https://apacheignite.readme.io/v2.6/docs/tcpip-discovery 
   -  http://ode.apache.org/developerguide/jacob.html 
   -  http://mail-archives.apache.org/mod_mbox/openwebbeans-user/201810.mbox/browser  
   - https://apacheignite.readme.io/v2.6/docs/distributed-persistent-store  
   - https://apacheignite.readme.io/v2.6/docs/distributed-sql  
   - https://apacheignite.readme.io/v2.6/docs/events

Re: Apache ODE 2.0 Proposal

Posted by Sathwik B P <sa...@gmail.com>.
Hi Aaron,

Apologise for my late reply, I am returning from a long vacation on the
occasion of deepavali festival.

Good to see your proposal. Unfortunately we do not have an active PMC & Dev
community around ODE now.

I think it would be appropriate to take this new concept & proposal in the
Apache incubator [https://incubator.apache.org/policy/process.html], so
that it gives an opportunity to build a new community around it.

ODE 1.x has probably reached it's EOL. I suppose retiring it would be the
best option.

regards,
sathwik

On Fri, Nov 2, 2018 at 10:28 AM Aaron Anderson <aa...@acm.org>
wrote:

> Hi All,
> Here are my general thoughts on a possible architecture for a potential
> new version of ODE. It is by no means complete and any feedback is welcomed.
> Thanks,
> Aaron
>
> Apache ODE 2.0 Development Proposal
>
>
>
> Motivation
> There is a great demand in the marketplace for DigitalTransformation [1] –
> marrying best of breed Cloud based system toincrease automation and empower
> intelligent business decision making.An underlying facet of this drive
> towards automation is businessprocess management. While there are many
> available Open Source BPMplatforms many use legacy architectures and
> technologies that do notscale in a Cloud environment. Most are also often
> designed for aspecific process format such as BPMN and support for
> extension andalternative specifications are an afterthought. Commercial
> BPMsolutions are extremely costly. Apache ODE 2.0 can satisfy the
> marketdemand for a scalable and flexible Open Source process
> orchestrationplatform that many can benefit from.
>
>
> Guiding Principles
>
>    - Cloud Architecture – ODE 2.0 should adhere to the Cloud Architecture
> Goals listed here [2]. This goal can be realized using Apache Ignite as the
> foundation of the system. ODE developers should focus on BPM support and
> work with the Apache Ignite project identify and remedy any deficiencies or
> defects with performance and scalability if they occur.
>    - General Process Management – ODE 2.0’s main focus should be to
> provide generic framework of common business process features that can be
> used or not by extension modules. ODE 2.0 should not be an academic
> exercise in mapping processes to a universal process object model; instead
> it should be considered an extension of Apache Ignite for business process
> orchestration requirements.
>    - Extensibility – Apache ODE should not only be capable of providing
> support for existing BPM specifications but it also should be future proof
> and be capable of supporting emerging standards or proprietary
> orchestration requirements. If the question is “I would like to implement
> process management specification XYZ” the answer should be Apache ODE 2.0.
>
> Key Terminology
>
>    - Persistent Process – A long running process that has a beginning and
> end that can span multiple systems be activated and passivated.
>    - Dialect – A representation of process orchestration, such as BPEL,
> BPMN, SCXML, etc. Dialects typically need to be interpreted for execution.
>    - Identifier – Everything in ODE 2.0 is addressed using a Qname
> (namespace and local part) or generated GUID which can be used
> interchangeably.
>    - Module – An Apache ODE 2.0 extension that is bundled as a single JAR
> achive in the server classpath. A module may contain a dialect compiler,
> processor, numerous instructions, exchanges, CLI commands, etc. The ODE
> runtime will discover implementations and register them accordingly. A
> module may have dependencies on other modules and can include it’s own CDI
> extension to support it’s own extensions.
>    - Tenant – Support multitenancy [3] through isolated Ignite clusters
> [4].
>    - Endpoint Registry – Every process interacts with other processes or
> external systems. Managing which endpoints are bound to each other is a
> complex task and central registry should be maintained. The registry can
> map a logical endpoint address (Qname) to a physical URL.
>    - Exchange – ODE 2.0 processes will interact with external entities.
> Exchanges are mediation points were external requests and responses, like
> HTTP requests, are converted into Ignite computing requests that in turn
> are processed by ODE or module implementations.
>    - Assembly – A unit of deployment in a JAR format. It may contain Java
> code and libraries, static resources such as single page application HTML
> files, or dialect representations like executable BPMN xml files.
> Assemblies have a YAML descriptor file or necessary metadata is extracted
> through annotations and CDI inspection. Assemblies may have dependencies on
> other assemblies.
>    - Composite - A configuration of an assembly. Concrete bindings are
> provided for all of the registered assembly endpoints. All required
> assembly configuration is also provided. Composites are environment
> specific so there may be a dev/qa/prod composite variants.
>    - Compiler – A dialect specific lexical analyzer that converts a
> dialect file such as BPMN into a sequence of instructions. As the compiler
> parses the dialect file extensions can indicate or override how the
> instructions are ordered and configured.
>    - Instruction – A CDI annotated Java class that interacts through
> Ignite with ODE or assembly services to create, mutate, or destroy
> persisted state in Ignite. Instructions should be single purpose so that
> they can be rearanged in different execution order. Instructions will be
> provided contextual references through CDI or Ignite lookups to perform
> discrete processing tasks. Instructions should be annotated with source
> line and column numbers to easily correlate them back dialect source files
> for troubleshooting or diagnostic purposes.
>    - Executable – A namespace aware XML representation of a sequence of
> instructions arranged in blocks. EXI or FI may be used to compress the
> representation for performance purposes.
>    - Processor – A dialect specific interpreter that is activated on
> demand that executes instructions in sequence. The processor sets up the
> executational context the instructions are invoked in and acts on
> directives issued by the instructions, like create a process, destroy it,
> activate, passivate, etc.
>
> Major Features
>
>    - Virtual Processor – Implements a primitive metaphor everyone in
> computing is familiar with. A dialect implementer is free to fracture [5]
> source lines through a dialect specific compiler into a sequence of
> instruction blocks. Upon activation through an enpoint the dialect specific
> processor is activated and executes each instruction in sequence. There is
> no ODE 2.0 standard for process storage in ignite, common instructions, or
> executable XML file format. Instead ODE 2.0 provides optional utilities for
> assisting with these tasks that can be extended or replaced. For instance a
> StAX xml parser that supports registered extensions for Qnames, Ignite
> storage functions for creating/updating/destroying processes, etc.
>    - Cloud Scale – Apache Ignite provides all the necessary capabilities
> and more.
>    - Modules – ODE or other module extensions that are added to the server
> classpath and are discovered and registered through CDI.
>    - Assembly – A bundle of executable code or content that is intended to
> be deployed as a service to ignite. Assemblies are jar files akin to Tomcat
> web applications. They are stored in Ignite as binary objects and are lazy
> loaded and extracted to a temp file on Ignite nodes after they are deployed
> for easy and efficient access. As dynamic deployable unit’s assemblies have
> their own classloader and CDI BeanManager. The assembly’s bean manager may
> delegate to the parent server classloader and bean manager to inject select
> API instances but in general the server classpath is shielded from the
> assembly [6]. Assemblies with dependencies will be loaded in the same
> classpath. This can help with deploying a single shared dependency like a
> jar file or WSDL file and have it shared across multiple assemblies.
> Assemblies are deployed, undeployed, released, and recalled. If the
> assembly contains a dialect source file it also supports build, clean, and
> verify.
>    - Composite – Composites are the configuration for one or more
> assemblies. They contain environment specific settings such as bindings for
> physical addresses and compiler preferences. Composites are created,
> updated, deleted, activated, and deactivated.
>    - Management – All management is performed through CLI like other Cloud
> service providers. The CLI bootstrap entry point creates a custom
> classloader and bean manager so only CLI specific instances are loaded. Any
> necessary configuration files are in the YAML format. Activated composites
> can be instrumented with diagnostic configuration such as state logging or
> break points.
>
>
>
>
>
> Technologies
>
>    - Apache Ignite - While Apache Ignite offers a variety of distributed
> computing functionality significant capabilities include persistence [7],
> indexing (ACID SQL 99 support) [8], and eventing [9].
>    - OpenWebBeans - CDI scopes, extensions
>    - Apache Tomcat - HTTP server
>    - Apache CXF – REST, OIDC support
>    - STaX and FastInfoset or Apache Avro for process executable format
>    - YAML - composite application descriptor
>    - Apache CLI
>
> References
>
>    -
> https://www.forbes.com/sites/jimmiller/2018/08/13/what-is-a-digital-transformation-and-why-should-you-care/#65ed41c57d35
>    -
> http://blog.grapeup.com/read/yet-another-look-at-cloud-native-apps-architecture-33
>
>    -  https://en.wikipedia.org/wiki/Multitenancy
>    -  https://apacheignite.readme.io/v2.6/docs/tcpip-discovery
>    -  http://ode.apache.org/developerguide/jacob.html
>    -
> http://mail-archives.apache.org/mod_mbox/openwebbeans-user/201810.mbox/browser
>
>    - https://apacheignite.readme.io/v2.6/docs/distributed-persistent-store
>
>    - https://apacheignite.readme.io/v2.6/docs/distributed-sql
>    - https://apacheignite.readme.io/v2.6/docs/events
>
>
>
>
>