You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Nicolas Malin <ni...@nereide.fr> on 2021/09/02 14:43:40 UTC
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Hello Eugen,
On 30/08/2021 11:48, Eugen Stan wrote:
> Hello,
>
> I've opened a new issue
> https://issues.apache.org/jira/browse/OFBIZ-12309 .
>
> I need community help with this.
I analyzed at fresh head, It's a hard task to remove the circular
dependency.
We have some dependencies that would be easily to remove (and some
introduce by myself I realized by misunderstanding or weakness of mind)
and some other that have their own reason.
If we want to go on increase the scalability, I search if the effort to
scale at each component is coherent with the most-valuable to win.
I have on my mind to elaborate a framework-core an acceptable minimal
circular dependencies and some other present as three dependency
* core : common - base - start - entity - service - security - webapp
* entityext : core
* datafile: core
* testtools: core
* catalina: core
* widget: core
* webtools: widget, testtools, datafile, catalina
I know this isn't your final objective but can it be an acceptable spot ?
Nicolas
>
> From the exploratory
> branch https://github.com/apache/ofbiz-framework/pull/319 I
> discovered there are a lot of circular dependencies between components
> (components depending on each other in order to build).
>
> I believe it would be very useful to have a logical dependency tree
> between components.
>
> As a developer working to make OFBiz usable as a framework I need to
> solve issues like circular dependencies between components
> (see https://issues.apache.org/jira/browse/OFBIZ-12308 ) .
> This should serve as a guide to help me decide how to solve the
> circular dependency issue.
>
> This is the current list of dependencies in framework (applications
> should be in another issue IMO).
>
> <load-component component-location="base"/>
> <load-component component-location="entity"/>
> <load-component component-location="security"/>
> <load-component component-location="datafile"/>
> <load-component component-location="minilang"/>
> <load-component component-location="common"/>
> <load-component component-location="service"/>
> <load-component component-location="catalina"/>
> <load-component component-location="entityext"/>
> <load-component component-location="webapp"/>
> <load-component component-location="widget"/>
> <load-component component-location="testtools"/>
> <load-component component-location="webtools"/>
>
>
> Related work:
>
> * https://issues.apache.org/jira/browse/OFBIZ-3500
> *
> https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+9.04
> *
> https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04
>
>
> Looking forward to feedback on this.
>
> Regards,
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Posted by Eugen Stan <eu...@netdava.com>.
Hello Taher,
Thanks for getting back.
I think we are very much aligned on where we want to go but we have
different perspectives on where we are in regards to OFBiz.
I will motivate my reply inline:
On 29.11.2021 10:36, Taher Alkhateeb wrote:
> Hello,
>
> I may be misunderstanding your point, but ofbiz is already broken down
> to sub-projects and the sub-projects are defined from the framework
> itself (check common.gradle). And it is possible to publish ofbiz parts
> separately, and also possible to isolate dependencies in each project
> and it is already done for some components. We also do already have a
> plugin mechanism that allows you to publish things separately.
>
I disagree with your statement above.
From my experience ofbiz is not split up in separate modules.
If you read https://issues.apache.org/jira/browse/OFBIZ-12308 and
comments from https://github.com/apache/ofbiz-framework/pull/319 you
will see what I mean.
There are circular dependencies between modules - so they are not split up.
Also if you look at https://issues.apache.org/jira/browse/OFBIZ-12263
you can't build plugins that use other repositories, unless you add
those repos (and I believe also dependencies) to OFBIz.
Basically you can't publish binary plugins easily IMO.
In order to fix these I believe we need to use gradle sub-projects and
have a full fledged build.gradle in each directory (full fledged gradle
project) complete with dependencies.
> Maybe there is little value now in working on Gradle beyond basic
> maintenance. The real hard-work that is probably badly needed is in
> breaking down the components themselves into layers that makes it easy
> to extend the framework at multiple levels. And to get rid of many
> things that piled up over the years. If you can get the system to work
> well at each of the below layers then extensibility might become an
> easier problem:
>
> 1. framework
>
> 2. data model -> framework
>
> 3. data-model services -> data -model
>
> 5. data-model UI -> data-model & data-model services
>
> 6. applications (I don't mean accounting, party, but rather a full
> application that utilizes the above layers)
>
> 7. plugins (pick and choose whatever layers needed)
>
> My 2 cents.
I do agree we should allow multiple aggregation of these modules in
different configurations.
That will allow us to build custom OFBiz distributions.
As I stated in https://github.com/apache/ofbiz-framework/pull/319 , this
is not possible since modules are interconnected (circular dependencies)
and you can't split them apart.
You can't publish and independently use framework/entities or services
or base because they use classes from each other.
Also the gradle build system is highly customized and hence not very
friendly to new developers because - since they need to understand the
1000+ lines of custom build code.
My proposal on this part is to:
* Drop dynamic project add via activeComponents (maybe keep for plugins ?!)
* Use standard build.gradle in every project - static project composition
* Gradually refactor code -> move dependencies into each sub-project
* Deprecate framework/start and replace with top level ofbiz-start
gradle project that will depend and aggregate ofbiz modules and build
the app. This will be the deliverable and also example on how to build
your own ofbiz distribution.
People should be able to customize or create their own copy of this
project to have a custom OFBiz distribution with whatever modules and
configuration they desire. If we publish jars for modules, this project
could be anywhere on git(hub).
>
> Cheers,
>
> Taher Alkhateeb
>
Regards,
--
Eugen Stan
+40770 941 271 / https://www.netdava.com
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Posted by Taher Alkhateeb <ta...@pythys.com.INVALID>.
Hello,
I may be misunderstanding your point, but ofbiz is already broken down
to sub-projects and the sub-projects are defined from the framework
itself (check common.gradle). And it is possible to publish ofbiz parts
separately, and also possible to isolate dependencies in each project
and it is already done for some components. We also do already have a
plugin mechanism that allows you to publish things separately.
Maybe there is little value now in working on Gradle beyond basic
maintenance. The real hard-work that is probably badly needed is in
breaking down the components themselves into layers that makes it easy
to extend the framework at multiple levels. And to get rid of many
things that piled up over the years. If you can get the system to work
well at each of the below layers then extensibility might become an
easier problem:
1. framework
2. data model -> framework
3. data-model services -> data -model
5. data-model UI -> data-model & data-model services
6. applications (I don't mean accounting, party, but rather a full
application that utilizes the above layers)
7. plugins (pick and choose whatever layers needed)
My 2 cents.
Cheers,
Taher Alkhateeb
On 11/29/21 03:28, Eugen Stan wrote:
> Hi,
>
> I think I made some progress while reverse engineering how ofbiz is
> built and I decided to share this so on the issue
> https://issues.apache.org/jira/browse/OFBIZ-3500
>
> My train of thought is to split the current setup first into 4 static
> gradle sub-projects:
> * framework
> * application
> * themes
> * plugins - we should add plugins dynamically first.
>
> Each sub-project should have it's own dependencies {}, source-sets {}
> etc.
>
> We can then depend these sub-projects in the main project.
>
> Benefits we get from this change:
> * we get closer to publish ofbiz parts as libraries .
> * we get closer to smaller components - which are easier to maintain
> * we get better dependency management - and closer into splitting the
> ofbiz monolith into smaller services
>
>
> Please let me know how you feel about this.
>
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Posted by Eugen Stan <eu...@netdava.com>.
Hi,
I think I made some progress while reverse engineering how ofbiz is
built and I decided to share this so on the issue
https://issues.apache.org/jira/browse/OFBIZ-3500
My train of thought is to split the current setup first into 4 static
gradle sub-projects:
* framework
* application
* themes
* plugins - we should add plugins dynamically first
Each sub-project should have it's own dependencies {}, source-sets {} etc.
We can then depend these sub-projects in the main project.
Benefits we get from this change:
* we get closer to publish ofbiz parts as libraries .
* we get closer to smaller components - which are easier to maintain
* we get better dependency management - and closer into splitting the
ofbiz monolith into smaller services
Please let me know how you feel about this.
--
Eugen Stan
+40770 941 271 / https://www.netdava.com
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Posted by Gil Portenseigne <gi...@nereide.fr>.
Hello Eugen,
[...]
> > I have on my mind to elaborate a framework-core an acceptable minimal
> > circular dependencies and some other present as three dependency
> >
> > * core : common - base - start - entity - service - security - webapp
> > * entityext : core
> > * datafile: core
> > * testtools: core
> > * catalina: core
> > * widget: core
> > * webtools: widget, testtools, datafile, catalina
> >
> > I know this isn't your final objective but can it be an acceptable spot ?
>
> Thanks. It's a very good place to start.
> We might be able to introduce a 'core' gradle project that contains those
> and depend on that one in the others.
>
> I would move out start module.
> It seems to be responsible for starting up the app.
> I imagine components have an interface (the Container).
> All ofbiz plugins/modules implement this so there should not be a dependency
> on implementations.
> Perhaps this is a good point to start - separating the container and
> depending on it in other modules.
>
> start will start ofbiz and load modules list from xml + look them up on the
> classpath.
>
> WDYT? How would the list above change with a container API module?
About dependency between base and start. Today, Start is packaged in
`org.apache.ofbiz.base`, it seems to have been designed to be the only
alternative for OFBiz server launch (no other implementation possible).
Indeed, this component is the one that stock the launch information
(portoffset etc.). To make it happen, there have to be defined interfaces
from which actual codebase would be one implementation.
Is that what you envision, if so this could be a good first example of
circular dependency removal.
Other example, regarding common component, that contains common elements
applicable to any other component like Unit of measure, lookups, emails.
For those, the services can be based upon screen to generate the desired
results inducing a dependency toward the widget component.
Moreover, lots of display information are propagated by Visual theme,
(during login, emails, freemarker engine preparation). We might need to
integrate all of these into base component using interface OR split into
the good/new component OR decide that OFBiz nature is to live with
widgets in its core.
>
> Another helpful thing is to establish the boundaries of each project.
> What should each module do and what it should NOT do?
* service : contains all soa configuration and engine to manage services
* entity : contains all database / entity management
* common : contains functional widget, data, services that can be used
by all other components
* base : contains technical low level code for all OFBiz components
* start : launching, stopping OFBiz server instance, managing start
configuration
* security : as is named, tools to manage user security
* webapp : engine to manage exchange between servlet and controller
definition
* entityext : container that extend the entity engine on some element
not vital for OFBiz to run like entity sync, cache distribution, ...
* datafile: tools to create/read big data file base on xml definition
* testtools: integration test engine
* catalina: container for embedded tomcat
* widget: screen rendering based on xml definition
* webtools: web interface to analyse and manage different OFBiz engines
Thanks Eugen,
Nicolas and Gil
Re: Logical dependency tree for ofbiz components OFBIZ-12309
Posted by Eugen Stan <eu...@netdava.com>.
Hello Nicolas,
Thank you for looking into this.
I know it's not an easy task.
I believe that where is a will, there is a way.
Now let's see if the OFBiz community has a strong will in this direction.
Let's discuss about your proposal and let's see where it goes.
See my comments inline:
On 02.09.2021 17:43, Nicolas Malin wrote:
> Hello Eugen,
>
> On 30/08/2021 11:48, Eugen Stan wrote:
>> Hello,
>>
>> I've opened a new issue
>> https://issues.apache.org/jira/browse/OFBIZ-12309 .
>>
>> I need community help with this.
>
> I analyzed at fresh head, It's a hard task to remove the circular
> dependency.
>
Yes it is :(.
> We have some dependencies that would be easily to remove (and some
> introduce by myself I realized by misunderstanding or weakness of mind)
> and some other that have their own reason.
>
> If we want to go on increase the scalability, I search if the effort to
> scale at each component is coherent with the most-valuable to win.
>
I'm mostly concerned with project maintainability and innovation - both
are hard to do in a large project with unclear dependencies.
> I have on my mind to elaborate a framework-core an acceptable minimal
> circular dependencies and some other present as three dependency
>
> * core : common - base - start - entity - service - security - webapp
> * entityext : core
> * datafile: core
> * testtools: core
> * catalina: core
> * widget: core
> * webtools: widget, testtools, datafile, catalina
>
> I know this isn't your final objective but can it be an acceptable spot ?
Thanks. It's a very good place to start.
We might be able to introduce a 'core' gradle project that contains
those and depend on that one in the others.
I would move out start module.
It seems to be responsible for starting up the app.
I imagine components have an interface (the Container).
All ofbiz plugins/modules implement this so there should not be a
dependency on implementations.
Perhaps this is a good point to start - separating the container and
depending on it in other modules.
start will start ofbiz and load modules list from xml + look them up on
the classpath.
WDYT? How would the list above change with a container API module?
Another helpful thing is to establish the boundaries of each project.
What should each module do and what it should NOT do?
I think a lot of things will clear up once that is established.
The functionality that does not fit in the module/project should be
moved elsewhere.
Can you (or anyone else) (please) provide some lines as to what should
each module in framework do and NOT do?
Eugen
>
> Nicolas
>
>>
>> From the exploratory
>> branch https://github.com/apache/ofbiz-framework/pull/319 I
>> discovered there are a lot of circular dependencies between components
>> (components depending on each other in order to build).
>>
>> I believe it would be very useful to have a logical dependency tree
>> between components.
>>
>> As a developer working to make OFBiz usable as a framework I need to
>> solve issues like circular dependencies between components
>> (see https://issues.apache.org/jira/browse/OFBIZ-12308 ) .
>> This should serve as a guide to help me decide how to solve the
>> circular dependency issue.
>>
>> This is the current list of dependencies in framework (applications
>> should be in another issue IMO).
>>
>> <load-component component-location="base"/>
>> <load-component component-location="entity"/>
>> <load-component component-location="security"/>
>> <load-component component-location="datafile"/>
>> <load-component component-location="minilang"/>
>> <load-component component-location="common"/>
>> <load-component component-location="service"/>
>> <load-component component-location="catalina"/>
>> <load-component component-location="entityext"/>
>> <load-component component-location="webapp"/>
>> <load-component component-location="widget"/>
>> <load-component component-location="testtools"/>
>> <load-component component-location="webtools"/>
>>
>>
>> Related work:
>>
>> * https://issues.apache.org/jira/browse/OFBIZ-3500
>> *
>> https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+9.04
>> *
>> https://cwiki.apache.org/confluence/display/OFBIZ/Ofbiz+as+a+development+framework+-+release+10.04
>>
>>
>> Looking forward to feedback on this.
>>
>> Regards,
>
--
Eugen Stan
+40720 898 747 / netdava.com