Re: OFBiz re-architecture thoughts

[Jacques Le Roux <ja...@les7arts.com>]

Hi,

If we want to improve OFBiz w/o changing too much the current OFBiz why not create an official OFBiz2 fork and work to this goal there?

Disclaimer: I don't think of doing something like that, if people are ready for it, why not?

Jacques

Le 28/12/2021 à 18:18, Sakthivel Vellingiri a écrit :
> Thanks Taher and Michael for detailed insights and Sathish for sharing your
> thoughts on upgrades.
>
> Just to introduce myself, we are using OFBiz for little over 7 years for a
> Life Sciences Product, and we consider ourselves as end user and haven't
> inclined to make hardcore changes to the framework as yet.
>
> 1. Lack of Code Isolation_:::
>       Our approach has been as outlined by Michael, we have custom plugins
> for each application and in some cases custom application plugins for each
> customer which are overrides and extensions of the OOTB OFBiz application
> plugins and we more or less leave the framework untouched except for a few
> issues we have discovered, and we are guilty of not contributing the fix
> back to OFBIZ and hence maintaining in our own branch, but without taking
> an inventory now, my recollection is that the changes to the framework is
> minimal. we merge the changes from our customer application plugin branch
> to *our* common application plugin branch that way *our* common application
> plugin branch at a given point in time has most of *our* custom application
> plugin changes.
>
> 2. Rest Interface:::
>      We are currently using HTML based web UI and relying on
> widget/freemarker templating for the changes. We are inclined to client
> rendering using Vue Js, and having Rest API integrated in the core
> framework would be a huge plus. but we are open to using Rest API as a
> plugin to begin with, I know there has been some communication around Rest
> based webservice api in the email thread, I have to admit, we haven't
> explored that all that much yet. But if a Rest API plugin is made
> available, we will be happy to utilize that and share feedback.
>
> 3. UI Screens:::
>      As experienced by new users and outlined in this thread, we end up
> customizing the UI to minimize the field display to our user needs either
> by making changes in minilang or ftl, Perhaps it might be good idea to have
> database tables to control what is rendered in the UI and perhaps rendering
> based on database configuration can include support for role/permission
> based rendering to show different fields for different
> roles/SecurityPermission.
>
> 4. Webapps :::
>     As of right now, we switch off the webapps that we do not need
> currently, At less frequent times we see application logouts when switching
> from one webapp to another, i'm not sure if this caused by having different
> plugins as different webapps and passing the session id between them and if
> this be solved by having single webapp, but i do not understand enough
> about the merits of having a single webapp at this time to comment more on
> this.
>
> I do not have strong opinions about the other points other than what is
> discussed already and i will skip responding to them,
>
> In my opinion replacing OFBiz framework code with established open source
> which is  purpose-built for a specific use case like Apache Shiro for
> Security, eHCache for caching could  bring down OFBiz framework code and
> thereby maintenance of OFBiz, framework however i also recognize and
> reflect the thoughts outlined in this thread that we need to be very
> careful about bringing such hardcore changes with careful planning &
> quality control not to disrupt the existing implementations.
>
> Thanks to everyone sharing their view points and starting a conversation in
> lieu of the future of OFBiz.
>
> just my 2 cents from the end user perspective.
>
> regards
> Sakthi
>
>
> On Mon, Dec 27, 2021 at 1:09 PM Taher Alkhateeb <ta...@pythys.com.invalid>
> wrote:
>
>> Hi Michael,
>>
>> Thank you so much for spending your time reading and sifting through
>> everything, I really appreciate and value your feedback.
>>
>> I need to highlight that we deployed mixed and somewhat crazy solutions
>> including multiple mobile apps, live video / chat streaming systems and
>> other projects that are booming in Kuwait and exploding both in user
>> bases and technical requirements which is why we cannot operate some of
>> these projects the way we used to in the past.
>>
>> Anyway I think I'm not going to elaborate too much as we wait for more
>> opinions from the community to see if this is worth pushing. However, I
>> want to reply to a few points you raised:
>>
>> - There is a huge difference between _few_ modifications, and ZERO
>> modifications to the framework or core components. This way you don't
>> even need to freeze a version of OFBiz to develop against at all!
>> - Breaking to multiple git repositories means having something like
>> ./gradlew gitPullAll to update and you're done. Nothing breaks, no
>> surprises. You have to try it to believe it :) I have client projects
>> that have 15 git repos and I can upgrade with my eyes closed. Also there
>> is always entanglement that you don't feel or care for until you
>> actually cut the repo into pieces and then realize we have too much
>> entanglement. Having many small git repositories is in fact the most
>> enjoyable part of how we're writing software now and I can assign
>> different modules to different teams and they work independent of each
>> other. It's just a totally different ballgame.
>> - All or nothing in terms of code means you have to onboard a lot of
>> code that you don't need. I'm not saying you cannot it turn it off, I'm
>> saying it should be opt-in, not opt-out. All this code is potential
>> maintenance, bugs, security, and so on that you shouldn't carry on if
>> you don't want it. This makes things the messiest in terms of seed /
>> demo data but also in other entanglements.
>> - Yes entities and services are re-usable but you're limited to only
>> that. Powerful re-usability is applied to everything (configurations,
>> routers, widgets, scripts, etc ...). What is mostly re-usable in OFBiz
>> is entities and services, the others are mostly "usable" not "reusable".
>> It would take too long to explain and provide examples so I'll delay
>> this part.
>> - It's so awesome to use groovy exactly where you need it, without
>> having to jump into yet another file to only write 2 lines of code. I
>> have a service that needs 5-6 lines of code, why do I have to point to
>> an external script. Put the implementation RIGHT THERE next to the
>> definition of your service for example, the same is true everywhere you
>> need XML actions. So scripting in lacking in XML actions, and also they
>> are inconsistent (works differently in different places).
>> - Mixing XML with groovy is just part of the flexibility and having your
>> code where you need it. Even debugging XML code improves by putting
>> groovy snippets above / below to investigate the context. Nobody would
>> want to do that by opening a new file just to print something from the
>> context for example.
>> - Services are pure business logic, controller is routing logic. Routing
>> logic does not mean NO logic, it means logic that is focused on
>> coordination and decisions on where to go and what to do and putting
>> that logic in services is actually mixing layers and making the code
>> harder to understand / debug. I cannot show you how severely limited
>> routing logic is in OFBiz is without showing you 5 / 6 examples of
>> powerful things we've done that we simply can not do in OFBiz.
>> - We should IMHO by default have ONLY one web-app with the ability to
>> add more in an "opt-in" fashion. However, this means we need to be able
>> to break the screen definitions and routes, routing needs a major change
>> to achieve this objective. The controller.xml would simply not cut it.
>> Again I need examples to really highlight what I mean and I might do it
>> later.
>> - It is not enough to integrate REST to be able to have dynamic
>> libraries integrated with OFBiz if you want that to work with your
>> widgets and other constructs. You also need multiple rendering modes
>> that allow you to select from a variety of different technologies /
>> options with different macros and different theming constructs. Again I
>> cannot say more without examples to illustrate clearly.
>> - Finally, many of the changes that I'm proposing require substantial
>> changes to the framework and redesigning it. Well we cannot do that
>> without first refactoring it, which means we have to get rid of / update
>> lots of code. which means we need to start using libraries to control
>> the large size of that thing. It's a chain, one thing pulling another
>> thing, which makes it inevitable that a major shakeup of the core
>> framework is necessary. I'm not proposing to change it just to change
>> it. I'm proposing changing it because without that many of the things
>> I'm proposing simply won't happen.
>>
>> I can understand why you might not agree with some of the points. In a
>> way it's kind of hard to imagine a different way of doing things when
>> we're used to working a certain way, but once we get exposed we realize
>> that OH .. I've been torturing myself :). However, I do so your
>> very-good point on being conservative because there is substantial risk
>> in doing everything that I suggest.
>>
>> Anyway opening this to the rest of the community and thank you in
>> advance for sharing your thoughts.
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> On 12/27/21 17:20, Michael Brohl wrote:
>>> Hi Taher,
>>>
>>> thanks for bringing this up! I will try to briefly explain my views on
>>> the raised topics.
>>>
>>> My perspective is from nearly 20 years of intensive work with OFBiz,
>>> both for internal and external projects. Those projects have/had a
>>> volume reaching from about 50 to over 1000  project days and we have
>>> active customer projects which are running for almost 20 years now,
>>> still being developed further with the business. OFBiz often became a
>>> crucial part of the business ecosystem of those users. It is also the
>>> central toolbox for our services around development of web based
>>> software solutions and system integration in our own company since 7
>>> years now. I consider myself both a user of OFBiz and a service
>>> provider/integrator.
>>>
>>> After so many years of working with OFBiz if have learned that - from
>>> a business users perspective - the functionality and integration
>>> aspects along with the powerful datamodel, services etc. as well as
>>> flexibility, performance and scalability is way more important than
>>> detailed technical aspects (in the sense of “modern” or “newest
>>> technology”).
>>>
>>> As a service provider for OFBiz, we are feeling responsible to provide
>>> a reliable, constant and upgradeable software ecosystem which is why
>>> we are on a more "conservative" path regarding big or long running
>>> changes in the project. I have seen several new, ambititious
>>> contributors who wanted to change everything, using the newest
>>> frameworks and tools around etc., but often did not have an extensive
>>> background of real world projects or who are not able to explain why
>>> they want to use other tools and mechanisms and which user problems
>>> they really want to solve.
>>>
>>> Most of these people are not active anymore or unfinished solutions
>>> are put into the codebase  which is another reason why I am really
>>> careful when it comes to massive changes. I think the project needs
>>> good planning, a good migration path and constant support when those
>>> changes are decided to take place.
>>>
>>> All in all, I do not feel such a great pain as you seem to feel but
>>> there is definetely much room for improvement, for sure.
>>>
>>> I will try to give my views on the different topics inline. They are
>>> brief and need/deserve more in-depth preparation from my side which I
>>> cannot spare at the moment.
>>>
>>> For further discussions I recommend to split those topics in different
>>> threads because they would become unreadable soon with more
>>> participants. Also writing up the outcome of the discussions and the
>>> further roadmap in the wiki might help.
>>>
>>> Let's see how it works.
>>>
>>> Michael Brohl
>>>
>>> ecomify GmbH - www.ecomify.de
>>>
>>>
>>> Am 22.12.21 um 18:23 schrieb Taher Alkhateeb:
>>>> Problems
>>> ...
>>>> _1. Lack of Code Isolation_:It is unlikely to do anything substantial
>>>> in OFBiz without touching the core repositories. Compare this with
>>>> moqui or some other web frameworks like say Ruby-On-Rails or django
>>>> and you will see that you can work independently of the development
>>>> path in these frameworks. This makes upgrades quite painfulwith
>>>> patches and manual interventions. Some in the community even
>>>> introduced a whole patching framework in the past.
>>> We do not have this experience, at least not as much as you seem to
>>> have. Customer projects are developed completely in plugins, using the
>>> framework and avoiding changes there. There ARE some changes tagged as
>>> mods but not too much to be a real pain during a vendor update. Some
>>> mods are also contributed back to OFBiz so these mods disappear during
>>> the next vendor import.
>>>
>>> We are not directly using the original framework but rather use the
>>> original as an upstream branch for our own projects. This makes it
>>> easy to track changes and merge them into our codebase.
>>>
>>> It would be helpful to have some specific examples on where you are
>>> having those painful changes.
>>>
>>>> _2. All or nothing:_In OFBiz, if you want to use the system, either
>>>> take the whole thing or take nothing. What if I don't want any user
>>>> interface for example, and just want to utilize the entities and
>>>> services?
>>> We are running projects which do not use the original user interface
>>> or are being headless / just REST interfaces. You are not forced to
>>> use the UI and you can switch off the web apps, do we really have a
>>> problem there?
>>>
>>>> _3. Limited Re-usability:_Because of the way current components are
>>>> designed, it's difficult to take advantage of components in other
>>>> components. Sharing artifacts is pretty much limited to entities and
>>>> services. Even then they're not very shareable because they're
>>>> designed in a non-usable way for a certain context and certain
>>>> settings and database parameters.*
>>> Datamodel and services, the core business artifacts, are shareable and
>>> in my view that's what's needed mostly. Can you be more specific
>>> what's missing or designed in a way that it is not (re)usable ?
>>>
>>>
>>>> *
>>>>
>>>> _4. Fragmented Actions:_**Instead of having unified XML actions, you
>>>> have fragmentation across different contexts (screens, services,
>>>> SECAs, etc). Furthermore, you cannot mix XML actions with groovy or
>>>> other artifacts except by calling into external scripts and things
>>>> like that. In comparison to moqui which is more flexible in XML
>>>> actions I find it restrictive to work with OFBiz actions and it's
>>>> hard to review, debug, or modify. I'm listing something below as an
>>>> example for comparison [1]*
>>> I will have to investigate more how Moqui does those things. I'm not
>>> sure what the example shows. Is it part of the controller logic?
>>>
>>> Generally, I think that it is a strength to have these tasks separated
>>> in an MVC pattern and not being able to mix controller logic with
>>> business logic.
>>>
>>> When it comes to debugging: how do you debug the shown XML code?
>>>
>>>
>>>> *
>>>>
>>>> _5. Limited Routing capability:_**Routing is very minimal
>>>> (controller.xml). It cannot contain business logic and is limited
>>>> either to chaining requests or rendering screens. Sophisticated
>>>> routing logic is important for making rich systems that react to
>>>> differing circumstances.
>>>> **
>>> See above: the controller part should NOT contain too much business
>>> logic. If the business logic would be mixed with the controller logic,
>>> how would you then make the business logic more shareable (see point 3
>>> above)?
>>>
>>> Maybe I'm missing something and some examples would help me understand
>>> the point.
>>>
>>>
>>>> _6. Core framework difficult to change:_**OFBiz core was written
>>>> a/long/time ago, and back then you didn't have many libraries for
>>>> caching, transactions, DB pooling, Concurrency stuff, and others, so
>>>> a lot was hand crafted, and we have a lots of interconnected API that
>>>> is difficult to change, and most of these APIs depend on static util
>>>> files that just kept piling and piling over the years. Where is the
>>>> service engine implementation exactly? What about the entity engine?
>>>> What about widgets? They are all over the place! Pieces for XML
>>>> parsing, others for DOM models, and so on. The code is scattered and
>>>> fixing it means going through a maze of many classes that are
>>>> interconnected and all referencing each other. It's also hard to
>>>> identify the signature or figure out the "big picture" since you have
>>>> essentially minimal interfaces and no clear public API that you can
>>>> refactor against. Pretty much the whole thing is public API**[2].
>>>> **
>>> That's a big field and I think we should break this into subtopics to
>>> discuss. Can you be more specific, for me this is too broad to
>>> discuss. What is the real problem, what would be a better solution, ...?
>>>
>>> In general, a framework like OFBiz IS complex, it will never be easy
>>> to read/understand. And of course, there is code which should be
>>> refactored and better maintained (deprecation etc.). It would maybe be
>>> a good start to identify those and do refactorings there first to
>>> reduce complexity and pave the path for bigger refactorings?
>>>
>>>  From my experience, e.g. the caching capabilities of OFBiz are a big
>>> strength and to change those parts, we need some really good proof of
>>> concepts to check if other solutions are really better. I've had
>>> several bad experiences with, for example, object relational mapping
>>> frameworks (performance, complexity) and I'm still a fan of the entity
>>> engine.
>>>
>>>
>>>> _7. Web-apps design problem__:_ A big problem (in my opinion) in the
>>>> design is around having multiple web-apps, one for each component (or
>>>> more). This interrupts the flow of the app and makes it difficult to
>>>> come up with smooth solutions that work across the various screens
>>>> and instead we need to pass a key between the components (and face
>>>> other problems around session management). This restricts you in what
>>>> you can offer as a user experience.*
>>>> *
>>> Do you have a solution in mind to solve this and also keep those
>>> applications separated (see above)? Single Sign On?
>>>
>>>
>>>> _8. Problematic UI design__:_ Another/big/problem (in my opinion) is
>>>> having the UI show everything (reference implementation). I never
>>>> understood the value of showing/everything/! If I want to understand
>>>> the data model, I will just look at the data model. The purpose of
>>>> the UI is not to show everything, but to implement specific useful
>>>> use-cases. The reference implementation we have is not doing that,
>>>> and not offering something very useful, instead it is showing
>>>> everything! To improve this, we need to switch from data-model based
>>>> interface (party, order, etc ...) to use-case based interface
>>>> (project management, e-commerce, ERP, CRM, etc ...)*
>>>> *
>>> That's indeed a big issue and part of the discussions with every new
>>> user who wants to utilize the existing web apps. On one hand it is
>>> great to be able to show the potential of the ecosystem, on the other
>>> hand it's too complex, especially for new users.
>>>
>>> IMO this is not so much a problem of the framework in general, which
>>> does not prevent us from developing better UI and workflows.
>>> Personally, I see OFBiz as an ERP framework which has to be customized
>>> to match the target users's needs and not so much as an ready-to-use ERP.
>>>
>>> But yes, this can and should be improved with additional web apps or
>>> even ONE web app which tightly integrates all core applications into a
>>> one-stop ERP solution which is more user-friendly than the base
>>> applications. On the other hand, this will make separation more
>>> difficult...
>>>
>>>
>>>> _9. Limited client rendering:_Because it is difficult to refactor the
>>>> framework, it's hard to pin-point where to go exactly to refactor all
>>>> the code needed to support more client-rendering technology (react,
>>>> vue, angular, etc ...) and make it interact smoothly with the
>>>> back-end (drop-downs, complex forms, etc ...). A lot of work needs to
>>>> happen both to the /themes and to /framework to allow and provide
>>>> more flexibility (also moving things to DB). If this step is not done
>>>> you will be stuck with your UI capabilities and won't catch up
>>>> (unless you develop your own UI which makes the UI of the community
>>>> redundant and you let go of the whole widget system anyway).*
>>>> *
>>> I guess it would help a lot if we have REST integrated into the core
>>> framework and expose a proper REST API. For the HTML based web UI, I
>>> have not much to change as the widget/freemarker templating gives us
>>> much flexibility.
>>>
>>> What do you miss exactly / in which direction do you want to do changes?
>>>
>>>
>>>> _10. Security is difficult to implement:_One of my favorite features
>>>> in moqui is having security baked in the artifacts themselves to
>>>> avoid sprinkling all screens / services with security logic. I never
>>>> even worry about writing security related stuff until way close to
>>>> the end of delivering the project. So there needs to be a whole
>>>> security layer that sits below all the artifacts such that we don't
>>>> even need to write security code in the various screens and services
>>>> and around certain entities.*
>>>> *
>>> Can you elaborate more on how you want to achieve fine grained
>>> permissions for screens and services if not tied to the screen/service
>>> configurations (having in mind that they should be reusable)?
>>>
>>>
>>>> _11. Limited Integration Solutions__:_ I know we have now a REST
>>>> component, but this is not the same as being a core feature of the
>>>> framework (essential for any modern web framework). Why should it be
>>>> tightly integrated? Many reasons: security, localization, mapping to
>>>> service engine, mapping to entities, etc ...*
>>>> *
>>> I'm currently working on the rest-api plugin and I agree that it would
>>> be great to have it in the framework instead of being a plugin.
>>>
>>>
>>>> _12. No Specialization:_Because of the big repository, and having
>>>> everything point to everything, it's hard to specialize. Now compare
>>>> this to the approach we have in our work which is easier to
>>>> specialize in. For example, I have a component for payment gateway
>>>> integration, another one for twilio (SMS integration), and I have
>>>> application components for each solution in its own repository (HR,
>>>> insurance stuff, many others) and each is living in its own git repo
>>>> yet they work together just fine and without touching one line of
>>>> code in the framework or even core components (despite depending on
>>>> them and using pieces of them). I can jump into new versions of the
>>>> framework everyday without breaking a sweat. When I work on component
>>>> X, I'm only thinking of that and nothing else. I don't have to worry
>>>> about changing something that would crash another component or bring
>>>> the system down (happened to me many times in the past).
>>> For our projects, we were going the other way: we have framework +
>>> plugins in the same repository instead of handling different
>>> repositories. I think it would be a real pain to split the project in
>>> dozens of different repositories only to bring them together to
>>> integrate during development or deployment again.
>>>
>>> Can you describe how you are handling those different repositories in
>>> an efficient way to get a better picture?
>>>
>>> As long as your work in a component/application/plugin and don't touch
>>> the framework code I also do not see why you should break something
>>> for other components? Of course, there ARE applications which are more
>>> fundamental than other (e.g. party, security) and if those are changed
>>> they might break other functionality. I see those applications
>>> belonging to the framework more than being independent applications.
>>>
>>> I agree that we should try to identify false or unnecessary
>>> dependencies though.
>>>
>>>
>>>>
>>>>     Solutions
>>>>
>>>> Whatever solutions we come up with I think they're going to be
>>>> difficult and would require community buy-in and support. I'm
>>>> breaking this down into things to keep, update, add and remove as
>>>> follows:
>>>>
>>>>
>>>>       Keep it
>>>>
>>>> - XSD signatures (so entity definitions, service definitions, XML
>>>> actions though should expand)
>>>> - Basic constructs of what is a component (with basic directory
>>>> structure)
>>>> - Data model
>>>> - Existing services (core data-model services)
>>>> - Java / groovy code that doesn't use core framework constructs or
>>>> goes too deep into the internals.
>>>> - Most used and useful APIs (Making queries, calling services, etc
>>>> ...) although it should all be converted to interfaces with separate
>>>> implementations where it's lacking.
>>>>
>>> Agree to all above.
>>>
>>>
>>>>       Improve It
>>>>
>>>> - Break the project to smaller git repositories.
>>>>    - framework
>>>>    - data model
>>>>    - services
>>>>    - common UI
>>>>    - use-case applications (ERP, CRM, e-commerce, etc ...)
>>>>    - plugins
>>>> - entire framework directory needs refurbishment and big chunks
>>>> probably require a delete / rewrite
>>>> - the web architecture (get rid of web-apps)
>>>> - the tests (lots of them need to be deleted anyway)
>>>> - Routing needs to be programmatic with ability to inject code.
>>>>
>>>>
>>>>       Add It
>>>>
>>>> - Configuration override mechanism to remove the need to touch the
>>>> core. Overrides include everything including routing, component
>>>> registration, etc ...
>>>> - REST integration into the core
>>>> - Artifact based security that maps users / groups to artifacts
>>>> without the need to inject security code into every screen
>>>> - A super-set of XML actions that can be applied in all contexts and
>>>> can embed groovy code
>>>> - New, simple use-case based user interface. It needs to be designed
>>>> such that it is easy to extend to avoid the all-or-nothing UI that
>>>> currently exists.
>>>>
>>>>
>>>>       Remove It / Deprecate It / Overwrite It
>>>>
>>>> - Most of the existing UI (needs to be replaced with simpler use-case
>>>> based interface)
>>>> - Lots of the core framework code (needs major rewrites, needs
>>>> interfaces, import libraries for things like transactions, security,
>>>> concurrency, etc ...)
>>>> - Probably some of the existing plugins might need to be deprecated.
>>> See my questions above for some of the topics. I will add more details
>>> and my point-of-view once we've split up the topics.
>>>
>>> The overall change would be huge and we should paint a clear picture
>>> of what we want to achieve and which way to go in detail.
>>>
>>> We'll have to prioritize with a good cost/effort consideration and do
>>> implementations step-by-step with a proper migration plan. Maybe
>>> having feature branches for a longer time to be able to put out
>>> releases if the development gets stuck or lasts longer.
>>>
>>>
>>>>
>>>>       Quality Control
>>>>
>>>> Investing in such a huge effort requires that we change the way we
>>>> view the project after / while work is being completed. Code should
>>>> be approached with greater care and a lot of emphasis should be put
>>>> on high quality, readability, avoiding anti-patterns, enforcing good
>>>> patterns and best-practices. Some of the daily workflows need to
>>>> improve / change, more thorough auditing and peer-reviews, and so on.
>>> A big +1 for quality control. This will be a real challenge for the
>>> project as it requires dedication and enough ressources for review.
>>>
>>>
>>>>
>>>>     Tradeoffs
>>>>
>>>> It goes without saying that the above is a _lot_ of work and
>>>> commitment and requires a lot of effort. That's why I hesitated in
>>>> starting this thread, but I thought maybe showing the tradeoffs would
>>>> make it easier to weigh things out and decide on a direction to push.
>>>>
>>>>
>>>>       Pros
>>>>
>>>> - Pretty much most of the above mentioned problems will be resolved.
>>>> - The framework would be resilient and future proof.
>>>> - We will have more options and capabilities in the future, it would
>>>> be easier to incorporate new technologies as they emerge
>>>> - Development would go faster
>>>> - Technical Debt will be reduced
>>>>
>>>>
>>>>       Cons
>>>>
>>>> - Huge (maybe too huge) upfront costs in time and effort to make the
>>>> switch
>>>> - Extra effort needed to ensure a smooth upgrade path for existing
>>>> users / adopters / system integrators. A whole layer of code needs to
>>>> be written to ensure the ability to upgrade without massive rewrites.
>>>> - Risk of failure, exhaustion, inability to push through to the end
>>>> - Inconvenience to those with existing systems / solutions
>>>>
>>>>
>>>>     Summary
>>>>
>>>> Change is the only constant. Inability to adapt to the future means
>>>> not surviving. I'm not confident what is the best change possible or
>>>> at what pace, but I do know that lack of change is a death sentence
>>>> to any project. Maybe it's time to reflect and think of where we'd
>>>> like to go and this thread is sort of a brain-dump of possible ideas.
>>>>
>>>> I look forward to hearing your thoughts.
>>> Thanks, Taher, for the thoughtful write-up of your view on the project
>>> and the fields of change you see to be tackled. I do not agree with
>>> all of them and I have a sensible view on cost/effort, thorough
>>> planning and the need to always have existing users/projects in mind.
>>> But I hope that we/the project will find a way to constantly work on
>>> improvements.
>>>
>>> Again, I would propose to split up the most important topics into
>>> threads with accompanying documentation in the wiki to help avoid
>>> confusion.
>>>
>>> I am curious what others have to say and if we can activate enough
>>> interest to improve.
>>>
>>>
>>>> Cheers,
>>>>
>>>> Taher Alkhateeb
>>>>
>>>>
>>>> [1] Example mixed XML actions and groovy from moqui
>>>>
>>>> <if condition="settlementTerm?.orderPmtServiceRegisterId">
>>>>      <then>
>>>>          <service-call name="update#mantle.order.OrderPart"
>>>>                  in-map="[orderId:orderId,
>>>> orderPartSeqId:orderPartSeqId, settlementTermId:settlementTermId]"/>
>>>>
>>>>          <entity-find-one entity-name="moqui.service.ServiceRegister"
>>>> value-field="serviceRegister">
>>>>              <field-map field-name="serviceRegisterId"
>>>> from="settlementTerm.orderPmtServiceRegisterId"/>
>>>>          </entity-find-one>
>>>>          <script><![CDATA[
>>>>              try {
>>>>
>> ec.service.sync().name(serviceRegister.serviceName).parameters(context).softValidate(true).disableAuthz().call()
>>
>>>>                  if (ec.message.hasError()) return
>>>>              } catch (Throwable t) {
>>>>                  ec.logger.log(300, "Error calling auto order payments
>>>> service for SettlementTerm ${settlementTermId}", t)
>>>>              }
>>>>          ]]></script>
>>>>      </then>
>>>>      <else>
>>>>          <service-call
>>>> name="mantle.account.PaymentServices.create#Payment" out-map="context"
>>>>                  in-map="context + [amount:amount,
>>>> amountUomId:amountUomId, fromPartyId:fromPartyId,
>>>> toPartyId:toPartyId]"/>
>>>>      </else>
>>>> </if>
>>>>
>>>> [2] https://ci.apache.org/projects/ofbiz/site/trunk/javadocs/
>>>>

Re: OFBiz re-architecture thoughts

[Jacques Le Roux <ja...@les7arts.com>]

Hi Nicola,

I think  Moqui is near from that.

Though not totally started from OFBiz it's quite inspired from David's long experience with OFBiz

Jacques

Le 02/04/2022 à 19:29, Nicola Mazzoni a écrit :
> I think it would be very interesting if some community developer were interested in creating a * small * POC of a DDD project. Using the OFBiz 
> Entity Engine by exploiting the DDD concepts and therefore the generation of entity objects, value objects, repositories ... We could take a 
> "domain", then a small part of OFBiz data-model and experiment with this approach. Leaving aside the graphical interface part, focusing only on the 
> core.
>
> What do you think? 

Re: OFBiz re-architecture thoughts

[Nicola Mazzoni <ni...@mpstyle.it>]

Hi Taher,
 From my experience I have chosen OFBiz because the only "coherent" 
framework with a data model and related services. Completely different 
from all frameworks that are architectural only but remain an "empty 
box" for building applications from scratch. I think there are many 
developers who need a business application framework (with an underlying 
data model) but they don't come close to OFBiz, I think because of the 
complexity and being, paradoxically, too complete. For example, the zip 
distribution already suggests an "all or nothing" use. Perhaps these 
things do not bring new potential users of OFBiz closer.
I think that an OOP logic is perfectly compatible with the entity and 
service engines, xml-dsl etc etc; indeed, they can be used as a basis to 
create a higher level of abstraction, which through OOP makes it more 
accessible.
With this I don't think of "changing" OFBiz for what it is, for the 
backward compatibility and for the solidity of the solutions based on 
OFBiz. It could be interesting, if there are interested community 
members, to try a different path, with the solid foundations of OFBiz 
but on a different paradigm. I don't know if this POC could be 
interesting enough to become a sub-project or it could simply remain a 
POC; but if anyone is interested it would be interesting to try!

Regards,

Nicola



On 03/04/22 19:12, Taher Alkhateeb wrote:
> Hi Nicola,
>
> Thank you for sharing your thoughts. It's good to have a practical 
> perspective.
>
> If we try to maybe list what OFBiz offers as highest value (guessing 
> on behalf of folks) I would think it would include:
>
> - The entity and service engines with their APIs
> - The data model represented by entities
> - The services that serve the data model
> - The XML DSL that abstracts away the underlying technologies 
> (widgets, actions, entities, etc ...)
>
> So OFBiz in that sense is a radical departure from OO since it all 
> depends on the abstractions of services (SAO) with an XML-based DSL to 
> manage everything. Maybe this is what people like or want the most out 
> of it.
>
> So whether or not we should attempt a PoC depends on whether people 
> actually want that different architecture since a DDD perspective is a 
> significant shift (especially with bounded contexts in an OO style) 
> from what OFBiz offers.
>
> Maybe more opinions here might shed light.
>
> Cheers,
>
> Taher
>
> On 4/2/22 20:29, Nicola Mazzoni wrote:
>> Hi Taher,
>> I think it's a very interesting discussion. She made me think a lot. 
>> I agree with your thoughts, but working as OFBiz "system integrator" 
>> I fully agree with Michael.
>> OFBiz is not a ready-to-use ERP system but it is a fantastic 
>> framework for developing custom ERP solutions. ERPs have a long, very 
>> long life cycle and are less sensitive to technological innovations. 
>> Often ERP and legacy are synonymous, while all the surrounding 
>> applications change, the ERPs remain. My clients in Erp, as in the 
>> "traditional" management of existing business processes, do not ask 
>> for a "cutting-edge" system. Also working on the IBM mainframe I know 
>> the subject, and I often suffer the consequences, not being able to 
>> "move forward"
>>
>> Since the beginning of 2022 I have had to take over the design "from 
>> scratch" of a business application and lead a small DevOps team to 
>> follow the development. I've waited 3 months to reply to your email 
>> to share this new experience with a quarter of feedback behind it. 
>> The project is not a "complete" ERP, but it is part of it. I didn't 
>> start from OFBiz, it was a solution with technological requirements 
>> very far from OFBiz and too tied to an already existing datamodel. 
>> But most of all, I wanted to experiment with a new (for me) approach. 
>> Just as OFBiz is closely linked to Len Silverstone's fantastic 
>> "Datamodel resource book", I wanted to find Eric Evans' "Domain 
>> Driven Design" as a "literary reference".
>> I don't like ORMs and therefore a very light persistance framework 
>> (like MyBatis) has been used, keeping the queries "handwritten" and 
>> not generated by the system. The goal was to make a completely object 
>> oriented, clean and simple system, not heavily tied to any existing 
>> framework. I followed the DDD approach (not in a dogmatic way ...) 
>> avoiding any third party libraries (excluding Jackson, Apache 
>> Commons, Apache Commons Pool, Apache CXF). I could have used Spring / 
>> Spring data; but I didn't. I have nothing against Spring, but I would 
>> like to use only frameworks that I know well, otherwise it is easy to 
>> become a victim. I would never want to tell a client "you know, my 
>> program is broken because there's something under the hood that I 
>> don't know that doesn't make it work anymore"
>>
>> In short? It was very interesting! The most significant effect (for 
>> the development team / qa) was certainly the simplicity with which 
>> the junior developers were able to be productive (having already the 
>> object model done) by working on a clean and very readable code. In 
>> general, the only drawback is a "structured" approach at the 
>> beginning which certainly requires considerable design experience and 
>> time. Precisely this, however, then allows an "agile" and very fast 
>> approach in tests, in customer feedback, in reworks. I don't think 
>> DDD (neither microservices, nor ORMs, nor noSql ...) is the silver 
>> bullet that solves all problems. But we can intelligently take 
>> advantage of these approaches without marrying them.
>>
>> I think it would be very interesting if some community developer were 
>> interested in creating a * small * POC of a DDD project. Using the 
>> OFBiz Entity Engine by exploiting the DDD concepts and therefore the 
>> generation of entity objects, value objects, repositories ... We 
>> could take a "domain", then a small part of OFBiz data-model and 
>> experiment with this approach. Leaving aside the graphical interface 
>> part, focusing only on the core.
>>
>> What do you think?
>>
>> Regards,
>>
>> Nicola
>>
>> On 01/03/22 04:56, Aditya Sharma wrote:
>>> Hi Taher,
>>>
>>> Thanks for this initiative!
>>>
>>> How about incubating it and adopting it as a subproject?
>>>
>>> Thanks and Regards,
>>> Aditya Sharma
>>>
>>> On Tue, Jan 25, 2022 at 10:28 PM Eugen Stan<eu...@netdava.com>  
>>> wrote:
>>>
>>>> Hello Taher,
>>>>
>>>> Thank you for this initiative and sorry for not replying sooner.
>>>> I meant to reply for so long but life ...
>>>>
>>>> I think most / all agrees that change is needed in OFBiz but I 
>>>> guess we
>>>> have different ideas on how to proceed.
>>>>
>>>> I don't think I can add a lot to what was already said.
>>>>
>>>> I do think keeping the current number of git repositories is good.
>>>> More will add overhead.
>>>> I do believe that ofbiz should provide stable extension points so 
>>>> people
>>>> can deploy their own plugins and functionality.
>>>>
>>>> I'm also working on some ideas around OFBiz - I believe I can
>>>> re-implement OFBiz entity engine on top of relational algebra using
>>>> Calcite.
>>>>
>>>> More on this soon on the mailing list.
>>>>
>>>> Looking forward to seeing a PoC from you.
>>>> I hope we do get to improve OFBiz and make it even more appealing.
>>>>
>>>>
>>>> On 25.01.2022 16:29, Taher Alkhateeb wrote:
>>>>> Thank you everyone for your kind feedback and sharing your 
>>>>> thoughts on
>>>>> this initiative.
>>>>>
>>>>> I didn't get enough feedback or momentum to give me the impression 
>>>>> that
>>>>> this can be a community initiative. I think I made the discussion too
>>>>> early before coming up with a PoC. So I will attempt something 
>>>>> privately
>>>>> and come back perhaps when it is in a more ready state and see if it
>>>>> garners any momentum. IF people are interested in collaborating, then
>>>>> perhaps you can approach me and we can team up.
>>>>>
>>>>> Thank you again.
>>>> Regards,
>>>>
>>>> -- 
>>>> Eugen Stan
>>>>
>>>> +40770 941 271  /https://www.netdava.com
-- 

*Nicola Mazzoni* | /CEO/CTO/

Mobile: +39 347 990 5529

*Mp Style srl*

Via Meucci, 37 | 41019 Limidi di Soliera (MO)

p.i / c.f IT03679300362

Re: OFBiz re-architecture thoughts

[Taher Alkhateeb <ta...@pythys.com.INVALID>]

Hi Nicola,

Thank you for sharing your thoughts. It's good to have a practical 
perspective.

If we try to maybe list what OFBiz offers as highest value (guessing on 
behalf of folks) I would think it would include:

- The entity and service engines with their APIs
- The data model represented by entities
- The services that serve the data model
- The XML DSL that abstracts away the underlying technologies (widgets, 
actions, entities, etc ...)

So OFBiz in that sense is a radical departure from OO since it all 
depends on the abstractions of services (SAO) with an XML-based DSL to 
manage everything. Maybe this is what people like or want the most out 
of it.

So whether or not we should attempt a PoC depends on whether people 
actually want that different architecture since a DDD perspective is a 
significant shift (especially with bounded contexts in an OO style) from 
what OFBiz offers.

Maybe more opinions here might shed light.

Cheers,

Taher

On 4/2/22 20:29, Nicola Mazzoni wrote:
> Hi Taher,
> I think it's a very interesting discussion. She made me think a lot. I 
> agree with your thoughts, but working as OFBiz "system integrator" I 
> fully agree with Michael.
> OFBiz is not a ready-to-use ERP system but it is a fantastic framework 
> for developing custom ERP solutions. ERPs have a long, very long life 
> cycle and are less sensitive to technological innovations. Often ERP 
> and legacy are synonymous, while all the surrounding applications 
> change, the ERPs remain. My clients in Erp, as in the "traditional" 
> management of existing business processes, do not ask for a 
> "cutting-edge" system. Also working on the IBM mainframe I know the 
> subject, and I often suffer the consequences, not being able to "move 
> forward"
>
> Since the beginning of 2022 I have had to take over the design "from 
> scratch" of a business application and lead a small DevOps team to 
> follow the development. I've waited 3 months to reply to your email to 
> share this new experience with a quarter of feedback behind it. The 
> project is not a "complete" ERP, but it is part of it. I didn't start 
> from OFBiz, it was a solution with technological requirements very far 
> from OFBiz and too tied to an already existing datamodel. But most of 
> all, I wanted to experiment with a new (for me) approach. Just as 
> OFBiz is closely linked to Len Silverstone's fantastic "Datamodel 
> resource book", I wanted to find Eric Evans' "Domain Driven Design" as 
> a "literary reference".
> I don't like ORMs and therefore a very light persistance framework 
> (like MyBatis) has been used, keeping the queries "handwritten" and 
> not generated by the system. The goal was to make a completely object 
> oriented, clean and simple system, not heavily tied to any existing 
> framework. I followed the DDD approach (not in a dogmatic way ...) 
> avoiding any third party libraries (excluding Jackson, Apache Commons, 
> Apache Commons Pool, Apache CXF). I could have used Spring / Spring 
> data; but I didn't. I have nothing against Spring, but I would like to 
> use only frameworks that I know well, otherwise it is easy to become a 
> victim. I would never want to tell a client "you know, my program is 
> broken because there's something under the hood that I don't know that 
> doesn't make it work anymore"
>
> In short? It was very interesting! The most significant effect (for 
> the development team / qa) was certainly the simplicity with which the 
> junior developers were able to be productive (having already the 
> object model done) by working on a clean and very readable code. In 
> general, the only drawback is a "structured" approach at the beginning 
> which certainly requires considerable design experience and time. 
> Precisely this, however, then allows an "agile" and very fast approach 
> in tests, in customer feedback, in reworks. I don't think DDD (neither 
> microservices, nor ORMs, nor noSql ...) is the silver bullet that 
> solves all problems. But we can intelligently take advantage of these 
> approaches without marrying them.
>
> I think it would be very interesting if some community developer were 
> interested in creating a * small * POC of a DDD project. Using the 
> OFBiz Entity Engine by exploiting the DDD concepts and therefore the 
> generation of entity objects, value objects, repositories ... We could 
> take a "domain", then a small part of OFBiz data-model and experiment 
> with this approach. Leaving aside the graphical interface part, 
> focusing only on the core.
>
> What do you think?
>
> Regards,
>
> Nicola
>
> On 01/03/22 04:56, Aditya Sharma wrote:
>> Hi Taher,
>>
>> Thanks for this initiative!
>>
>> How about incubating it and adopting it as a subproject?
>>
>> Thanks and Regards,
>> Aditya Sharma
>>
>> On Tue, Jan 25, 2022 at 10:28 PM Eugen Stan<eu...@netdava.com>  
>> wrote:
>>
>>> Hello Taher,
>>>
>>> Thank you for this initiative and sorry for not replying sooner.
>>> I meant to reply for so long but life ...
>>>
>>> I think most / all agrees that change is needed in OFBiz but I guess we
>>> have different ideas on how to proceed.
>>>
>>> I don't think I can add a lot to what was already said.
>>>
>>> I do think keeping the current number of git repositories is good.
>>> More will add overhead.
>>> I do believe that ofbiz should provide stable extension points so 
>>> people
>>> can deploy their own plugins and functionality.
>>>
>>> I'm also working on some ideas around OFBiz - I believe I can
>>> re-implement OFBiz entity engine on top of relational algebra using
>>> Calcite.
>>>
>>> More on this soon on the mailing list.
>>>
>>> Looking forward to seeing a PoC from you.
>>> I hope we do get to improve OFBiz and make it even more appealing.
>>>
>>>
>>> On 25.01.2022 16:29, Taher Alkhateeb wrote:
>>>> Thank you everyone for your kind feedback and sharing your thoughts on
>>>> this initiative.
>>>>
>>>> I didn't get enough feedback or momentum to give me the impression 
>>>> that
>>>> this can be a community initiative. I think I made the discussion too
>>>> early before coming up with a PoC. So I will attempt something 
>>>> privately
>>>> and come back perhaps when it is in a more ready state and see if it
>>>> garners any momentum. IF people are interested in collaborating, then
>>>> perhaps you can approach me and we can team up.
>>>>
>>>> Thank you again.
>>> Regards,
>>>
>>> -- 
>>> Eugen Stan
>>>
>>> +40770 941 271  /https://www.netdava.com

Re: OFBiz re-architecture thoughts

[Nicola Mazzoni <ni...@mpstyle.it>]

Hi Taher,
I think it's a very interesting discussion. She made me think a lot. I 
agree with your thoughts, but working as OFBiz "system integrator" I 
fully agree with Michael.
OFBiz is not a ready-to-use ERP system but it is a fantastic framework 
for developing custom ERP solutions. ERPs have a long, very long life 
cycle and are less sensitive to technological innovations. Often ERP and 
legacy are synonymous, while all the surrounding applications change, 
the ERPs remain. My clients in Erp, as in the "traditional" management 
of existing business processes, do not ask for a "cutting-edge" system. 
Also working on the IBM mainframe I know the subject, and I often suffer 
the consequences, not being able to "move forward"

Since the beginning of 2022 I have had to take over the design "from 
scratch" of a business application and lead a small DevOps team to 
follow the development. I've waited 3 months to reply to your email to 
share this new experience with a quarter of feedback behind it. The 
project is not a "complete" ERP, but it is part of it. I didn't start 
from OFBiz, it was a solution with technological requirements very far 
from OFBiz and too tied to an already existing datamodel. But most of 
all, I wanted to experiment with a new (for me) approach. Just as OFBiz 
is closely linked to Len Silverstone's fantastic "Datamodel resource 
book", I wanted to find Eric Evans' "Domain Driven Design" as a 
"literary reference".
I don't like ORMs and therefore a very light persistance framework (like 
MyBatis) has been used, keeping the queries "handwritten" and not 
generated by the system. The goal was to make a completely object 
oriented, clean and simple system, not heavily tied to any existing 
framework. I followed the DDD approach (not in a dogmatic way ...) 
avoiding any third party libraries (excluding Jackson, Apache Commons, 
Apache Commons Pool, Apache CXF). I could have used Spring / Spring 
data; but I didn't. I have nothing against Spring, but I would like to 
use only frameworks that I know well, otherwise it is easy to become a 
victim. I would never want to tell a client "you know, my program is 
broken because there's something under the hood that I don't know that 
doesn't make it work anymore"

In short? It was very interesting! The most significant effect (for the 
development team / qa) was certainly the simplicity with which the 
junior developers were able to be productive (having already the object 
model done) by working on a clean and very readable code. In general, 
the only drawback is a "structured" approach at the beginning which 
certainly requires considerable design experience and time. Precisely 
this, however, then allows an "agile" and very fast approach in tests, 
in customer feedback, in reworks. I don't think DDD (neither 
microservices, nor ORMs, nor noSql ...) is the silver bullet that solves 
all problems. But we can intelligently take advantage of these 
approaches without marrying them.

I think it would be very interesting if some community developer were 
interested in creating a * small * POC of a DDD project. Using the OFBiz 
Entity Engine by exploiting the DDD concepts and therefore the 
generation of entity objects, value objects, repositories ... We could 
take a "domain", then a small part of OFBiz data-model and experiment 
with this approach. Leaving aside the graphical interface part, focusing 
only on the core.

What do you think?

Regards,

Nicola

On 01/03/22 04:56, Aditya Sharma wrote:
> Hi Taher,
>
> Thanks for this initiative!
>
> How about incubating it and adopting it as a subproject?
>
> Thanks and Regards,
> Aditya Sharma
>
> On Tue, Jan 25, 2022 at 10:28 PM Eugen Stan<eu...@netdava.com>  wrote:
>
>> Hello Taher,
>>
>> Thank you for this initiative and sorry for not replying sooner.
>> I meant to reply for so long but life ...
>>
>> I think most / all agrees that change is needed in OFBiz but I guess we
>> have different ideas on how to proceed.
>>
>> I don't think I can add a lot to what was already said.
>>
>> I do think keeping the current number of git repositories is good.
>> More will add overhead.
>> I do believe that ofbiz should provide stable extension points so people
>> can deploy their own plugins and functionality.
>>
>> I'm also working on some ideas around OFBiz - I believe I can
>> re-implement OFBiz entity engine on top of relational algebra using
>> Calcite.
>>
>> More on this soon on the mailing list.
>>
>> Looking forward to seeing a PoC from you.
>> I hope we do get to improve OFBiz and make it even more appealing.
>>
>>
>> On 25.01.2022 16:29, Taher Alkhateeb wrote:
>>> Thank you everyone for your kind feedback and sharing your thoughts on
>>> this initiative.
>>>
>>> I didn't get enough feedback or momentum to give me the impression that
>>> this can be a community initiative. I think I made the discussion too
>>> early before coming up with a PoC. So I will attempt something privately
>>> and come back perhaps when it is in a more ready state and see if it
>>> garners any momentum. IF people are interested in collaborating, then
>>> perhaps you can approach me and we can team up.
>>>
>>> Thank you again.
>> Regards,
>>
>> --
>> Eugen Stan
>>
>> +40770 941 271  /https://www.netdava.com
-- 

*Nicola Mazzoni* | /CEO/CTO/

Mobile: +39 347 990 5529

*Mp Style srl*

Via Meucci, 37 | 41019 Limidi di Soliera (MO)

p.i / c.f IT03679300362

Re: OFBiz re-architecture thoughts

[Aditya Sharma <ad...@apache.org>]

Hi Taher,

Thanks for this initiative!

How about incubating it and adopting it as a subproject?

Thanks and Regards,
Aditya Sharma

On Tue, Jan 25, 2022 at 10:28 PM Eugen Stan <eu...@netdava.com> wrote:

> Hello Taher,
>
> Thank you for this initiative and sorry for not replying sooner.
> I meant to reply for so long but life ...
>
> I think most / all agrees that change is needed in OFBiz but I guess we
> have different ideas on how to proceed.
>
> I don't think I can add a lot to what was already said.
>
> I do think keeping the current number of git repositories is good.
> More will add overhead.
> I do believe that ofbiz should provide stable extension points so people
> can deploy their own plugins and functionality.
>
> I'm also working on some ideas around OFBiz - I believe I can
> re-implement OFBiz entity engine on top of relational algebra using
> Calcite.
>
> More on this soon on the mailing list.
>
> Looking forward to seeing a PoC from you.
> I hope we do get to improve OFBiz and make it even more appealing.
>
>
> On 25.01.2022 16:29, Taher Alkhateeb wrote:
> > Thank you everyone for your kind feedback and sharing your thoughts on
> > this initiative.
> >
> > I didn't get enough feedback or momentum to give me the impression that
> > this can be a community initiative. I think I made the discussion too
> > early before coming up with a PoC. So I will attempt something privately
> > and come back perhaps when it is in a more ready state and see if it
> > garners any momentum. IF people are interested in collaborating, then
> > perhaps you can approach me and we can team up.
> >
> > Thank you again.
>
> Regards,
>
> --
> Eugen Stan
>
> +40770 941 271  / https://www.netdava.com

Re: OFBiz re-architecture thoughts

[Eugen Stan <eu...@netdava.com>]

Hello Taher,

Thank you for this initiative and sorry for not replying sooner.
I meant to reply for so long but life ...

I think most / all agrees that change is needed in OFBiz but I guess we 
have different ideas on how to proceed.

I don't think I can add a lot to what was already said.

I do think keeping the current number of git repositories is good.
More will add overhead.
I do believe that ofbiz should provide stable extension points so people 
can deploy their own plugins and functionality.

I'm also working on some ideas around OFBiz - I believe I can 
re-implement OFBiz entity engine on top of relational algebra using Calcite.

More on this soon on the mailing list.

Looking forward to seeing a PoC from you.
I hope we do get to improve OFBiz and make it even more appealing.


On 25.01.2022 16:29, Taher Alkhateeb wrote:
> Thank you everyone for your kind feedback and sharing your thoughts on 
> this initiative.
> 
> I didn't get enough feedback or momentum to give me the impression that 
> this can be a community initiative. I think I made the discussion too 
> early before coming up with a PoC. So I will attempt something privately 
> and come back perhaps when it is in a more ready state and see if it 
> garners any momentum. IF people are interested in collaborating, then 
> perhaps you can approach me and we can team up.
> 
> Thank you again.

Regards,

-- 
Eugen Stan

+40770 941 271  / https://www.netdava.com

Re: OFBiz re-architecture thoughts

[Taher Alkhateeb <ta...@pythys.com.INVALID>]

Thank you everyone for your kind feedback and sharing your thoughts on 
this initiative.

I didn't get enough feedback or momentum to give me the impression that 
this can be a community initiative. I think I made the discussion too 
early before coming up with a PoC. So I will attempt something privately 
and come back perhaps when it is in a more ready state and see if it 
garners any momentum. IF people are interested in collaborating, then 
perhaps you can approach me and we can team up.

Thank you again.

On 1/9/22 11:16, Jacques Le Roux wrote:
> Hi,
>
> If we want to improve OFBiz w/o changing too much the current OFBiz 
> why not create an official OFBiz2 fork and work to this goal there?
>
> Disclaimer: I don't think of doing something like that, if people are 
> ready for it, why not?
>
> Jacques
>
> Le 28/12/2021 à 18:18, Sakthivel Vellingiri a écrit :
>> Thanks Taher and Michael for detailed insights and Sathish for 
>> sharing your
>> thoughts on upgrades.
>>
>> Just to introduce myself, we are using OFBiz for little over 7 years 
>> for a
>> Life Sciences Product, and we consider ourselves as end user and haven't
>> inclined to make hardcore changes to the framework as yet.
>>
>> 1. Lack of Code Isolation_:::
>>       Our approach has been as outlined by Michael, we have custom 
>> plugins
>> for each application and in some cases custom application plugins for 
>> each
>> customer which are overrides and extensions of the OOTB OFBiz 
>> application
>> plugins and we more or less leave the framework untouched except for 
>> a few
>> issues we have discovered, and we are guilty of not contributing the fix
>> back to OFBIZ and hence maintaining in our own branch, but without 
>> taking
>> an inventory now, my recollection is that the changes to the 
>> framework is
>> minimal. we merge the changes from our customer application plugin 
>> branch
>> to *our* common application plugin branch that way *our* common 
>> application
>> plugin branch at a given point in time has most of *our* custom 
>> application
>> plugin changes.
>>
>> 2. Rest Interface:::
>>      We are currently using HTML based web UI and relying on
>> widget/freemarker templating for the changes. We are inclined to client
>> rendering using Vue Js, and having Rest API integrated in the core
>> framework would be a huge plus. but we are open to using Rest API as a
>> plugin to begin with, I know there has been some communication around 
>> Rest
>> based webservice api in the email thread, I have to admit, we haven't
>> explored that all that much yet. But if a Rest API plugin is made
>> available, we will be happy to utilize that and share feedback.
>>
>> 3. UI Screens:::
>>      As experienced by new users and outlined in this thread, we end up
>> customizing the UI to minimize the field display to our user needs 
>> either
>> by making changes in minilang or ftl, Perhaps it might be good idea 
>> to have
>> database tables to control what is rendered in the UI and perhaps 
>> rendering
>> based on database configuration can include support for role/permission
>> based rendering to show different fields for different
>> roles/SecurityPermission.
>>
>> 4. Webapps :::
>>     As of right now, we switch off the webapps that we do not need
>> currently, At less frequent times we see application logouts when 
>> switching
>> from one webapp to another, i'm not sure if this caused by having 
>> different
>> plugins as different webapps and passing the session id between them 
>> and if
>> this be solved by having single webapp, but i do not understand enough
>> about the merits of having a single webapp at this time to comment 
>> more on
>> this.
>>
>> I do not have strong opinions about the other points other than what is
>> discussed already and i will skip responding to them,
>>
>> In my opinion replacing OFBiz framework code with established open 
>> source
>> which is  purpose-built for a specific use case like Apache Shiro for
>> Security, eHCache for caching could  bring down OFBiz framework code and
>> thereby maintenance of OFBiz, framework however i also recognize and
>> reflect the thoughts outlined in this thread that we need to be very
>> careful about bringing such hardcore changes with careful planning &
>> quality control not to disrupt the existing implementations.
>>
>> Thanks to everyone sharing their view points and starting a 
>> conversation in
>> lieu of the future of OFBiz.
>>
>> just my 2 cents from the end user perspective.
>>
>> regards
>> Sakthi
>>
>>
>> On Mon, Dec 27, 2021 at 1:09 PM Taher Alkhateeb 
>> <ta...@pythys.com.invalid>
>> wrote:
>>
>>> Hi Michael,
>>>
>>> Thank you so much for spending your time reading and sifting through
>>> everything, I really appreciate and value your feedback.
>>>
>>> I need to highlight that we deployed mixed and somewhat crazy solutions
>>> including multiple mobile apps, live video / chat streaming systems and
>>> other projects that are booming in Kuwait and exploding both in user
>>> bases and technical requirements which is why we cannot operate some of
>>> these projects the way we used to in the past.
>>>
>>> Anyway I think I'm not going to elaborate too much as we wait for more
>>> opinions from the community to see if this is worth pushing. However, I
>>> want to reply to a few points you raised:
>>>
>>> - There is a huge difference between _few_ modifications, and ZERO
>>> modifications to the framework or core components. This way you don't
>>> even need to freeze a version of OFBiz to develop against at all!
>>> - Breaking to multiple git repositories means having something like
>>> ./gradlew gitPullAll to update and you're done. Nothing breaks, no
>>> surprises. You have to try it to believe it :) I have client projects
>>> that have 15 git repos and I can upgrade with my eyes closed. Also 
>>> there
>>> is always entanglement that you don't feel or care for until you
>>> actually cut the repo into pieces and then realize we have too much
>>> entanglement. Having many small git repositories is in fact the most
>>> enjoyable part of how we're writing software now and I can assign
>>> different modules to different teams and they work independent of each
>>> other. It's just a totally different ballgame.
>>> - All or nothing in terms of code means you have to onboard a lot of
>>> code that you don't need. I'm not saying you cannot it turn it off, I'm
>>> saying it should be opt-in, not opt-out. All this code is potential
>>> maintenance, bugs, security, and so on that you shouldn't carry on if
>>> you don't want it. This makes things the messiest in terms of seed /
>>> demo data but also in other entanglements.
>>> - Yes entities and services are re-usable but you're limited to only
>>> that. Powerful re-usability is applied to everything (configurations,
>>> routers, widgets, scripts, etc ...). What is mostly re-usable in OFBiz
>>> is entities and services, the others are mostly "usable" not 
>>> "reusable".
>>> It would take too long to explain and provide examples so I'll delay
>>> this part.
>>> - It's so awesome to use groovy exactly where you need it, without
>>> having to jump into yet another file to only write 2 lines of code. I
>>> have a service that needs 5-6 lines of code, why do I have to point to
>>> an external script. Put the implementation RIGHT THERE next to the
>>> definition of your service for example, the same is true everywhere you
>>> need XML actions. So scripting in lacking in XML actions, and also they
>>> are inconsistent (works differently in different places).
>>> - Mixing XML with groovy is just part of the flexibility and having 
>>> your
>>> code where you need it. Even debugging XML code improves by putting
>>> groovy snippets above / below to investigate the context. Nobody would
>>> want to do that by opening a new file just to print something from the
>>> context for example.
>>> - Services are pure business logic, controller is routing logic. 
>>> Routing
>>> logic does not mean NO logic, it means logic that is focused on
>>> coordination and decisions on where to go and what to do and putting
>>> that logic in services is actually mixing layers and making the code
>>> harder to understand / debug. I cannot show you how severely limited
>>> routing logic is in OFBiz is without showing you 5 / 6 examples of
>>> powerful things we've done that we simply can not do in OFBiz.
>>> - We should IMHO by default have ONLY one web-app with the ability to
>>> add more in an "opt-in" fashion. However, this means we need to be able
>>> to break the screen definitions and routes, routing needs a major 
>>> change
>>> to achieve this objective. The controller.xml would simply not cut it.
>>> Again I need examples to really highlight what I mean and I might do it
>>> later.
>>> - It is not enough to integrate REST to be able to have dynamic
>>> libraries integrated with OFBiz if you want that to work with your
>>> widgets and other constructs. You also need multiple rendering modes
>>> that allow you to select from a variety of different technologies /
>>> options with different macros and different theming constructs. Again I
>>> cannot say more without examples to illustrate clearly.
>>> - Finally, many of the changes that I'm proposing require substantial
>>> changes to the framework and redesigning it. Well we cannot do that
>>> without first refactoring it, which means we have to get rid of / 
>>> update
>>> lots of code. which means we need to start using libraries to control
>>> the large size of that thing. It's a chain, one thing pulling another
>>> thing, which makes it inevitable that a major shakeup of the core
>>> framework is necessary. I'm not proposing to change it just to change
>>> it. I'm proposing changing it because without that many of the things
>>> I'm proposing simply won't happen.
>>>
>>> I can understand why you might not agree with some of the points. In a
>>> way it's kind of hard to imagine a different way of doing things when
>>> we're used to working a certain way, but once we get exposed we realize
>>> that OH .. I've been torturing myself :). However, I do so your
>>> very-good point on being conservative because there is substantial risk
>>> in doing everything that I suggest.
>>>
>>> Anyway opening this to the rest of the community and thank you in
>>> advance for sharing your thoughts.
>>>
>>> Cheers,
>>>
>>> Taher Alkhateeb
>>>
>>> On 12/27/21 17:20, Michael Brohl wrote:
>>>> Hi Taher,
>>>>
>>>> thanks for bringing this up! I will try to briefly explain my views on
>>>> the raised topics.
>>>>
>>>> My perspective is from nearly 20 years of intensive work with OFBiz,
>>>> both for internal and external projects. Those projects have/had a
>>>> volume reaching from about 50 to over 1000  project days and we have
>>>> active customer projects which are running for almost 20 years now,
>>>> still being developed further with the business. OFBiz often became a
>>>> crucial part of the business ecosystem of those users. It is also the
>>>> central toolbox for our services around development of web based
>>>> software solutions and system integration in our own company since 7
>>>> years now. I consider myself both a user of OFBiz and a service
>>>> provider/integrator.
>>>>
>>>> After so many years of working with OFBiz if have learned that - from
>>>> a business users perspective - the functionality and integration
>>>> aspects along with the powerful datamodel, services etc. as well as
>>>> flexibility, performance and scalability is way more important than
>>>> detailed technical aspects (in the sense of “modern” or “newest
>>>> technology”).
>>>>
>>>> As a service provider for OFBiz, we are feeling responsible to provide
>>>> a reliable, constant and upgradeable software ecosystem which is why
>>>> we are on a more "conservative" path regarding big or long running
>>>> changes in the project. I have seen several new, ambititious
>>>> contributors who wanted to change everything, using the newest
>>>> frameworks and tools around etc., but often did not have an extensive
>>>> background of real world projects or who are not able to explain why
>>>> they want to use other tools and mechanisms and which user problems
>>>> they really want to solve.
>>>>
>>>> Most of these people are not active anymore or unfinished solutions
>>>> are put into the codebase  which is another reason why I am really
>>>> careful when it comes to massive changes. I think the project needs
>>>> good planning, a good migration path and constant support when those
>>>> changes are decided to take place.
>>>>
>>>> All in all, I do not feel such a great pain as you seem to feel but
>>>> there is definetely much room for improvement, for sure.
>>>>
>>>> I will try to give my views on the different topics inline. They are
>>>> brief and need/deserve more in-depth preparation from my side which I
>>>> cannot spare at the moment.
>>>>
>>>> For further discussions I recommend to split those topics in different
>>>> threads because they would become unreadable soon with more
>>>> participants. Also writing up the outcome of the discussions and the
>>>> further roadmap in the wiki might help.
>>>>
>>>> Let's see how it works.
>>>>
>>>> Michael Brohl
>>>>
>>>> ecomify GmbH - www.ecomify.de
>>>>
>>>>
>>>> Am 22.12.21 um 18:23 schrieb Taher Alkhateeb:
>>>>> Problems
>>>> ...
>>>>> _1. Lack of Code Isolation_:It is unlikely to do anything substantial
>>>>> in OFBiz without touching the core repositories. Compare this with
>>>>> moqui or some other web frameworks like say Ruby-On-Rails or django
>>>>> and you will see that you can work independently of the development
>>>>> path in these frameworks. This makes upgrades quite painfulwith
>>>>> patches and manual interventions. Some in the community even
>>>>> introduced a whole patching framework in the past.
>>>> We do not have this experience, at least not as much as you seem to
>>>> have. Customer projects are developed completely in plugins, using the
>>>> framework and avoiding changes there. There ARE some changes tagged as
>>>> mods but not too much to be a real pain during a vendor update. Some
>>>> mods are also contributed back to OFBiz so these mods disappear during
>>>> the next vendor import.
>>>>
>>>> We are not directly using the original framework but rather use the
>>>> original as an upstream branch for our own projects. This makes it
>>>> easy to track changes and merge them into our codebase.
>>>>
>>>> It would be helpful to have some specific examples on where you are
>>>> having those painful changes.
>>>>
>>>>> _2. All or nothing:_In OFBiz, if you want to use the system, either
>>>>> take the whole thing or take nothing. What if I don't want any user
>>>>> interface for example, and just want to utilize the entities and
>>>>> services?
>>>> We are running projects which do not use the original user interface
>>>> or are being headless / just REST interfaces. You are not forced to
>>>> use the UI and you can switch off the web apps, do we really have a
>>>> problem there?
>>>>
>>>>> _3. Limited Re-usability:_Because of the way current components are
>>>>> designed, it's difficult to take advantage of components in other
>>>>> components. Sharing artifacts is pretty much limited to entities and
>>>>> services. Even then they're not very shareable because they're
>>>>> designed in a non-usable way for a certain context and certain
>>>>> settings and database parameters.*
>>>> Datamodel and services, the core business artifacts, are shareable and
>>>> in my view that's what's needed mostly. Can you be more specific
>>>> what's missing or designed in a way that it is not (re)usable ?
>>>>
>>>>
>>>>> *
>>>>>
>>>>> _4. Fragmented Actions:_**Instead of having unified XML actions, you
>>>>> have fragmentation across different contexts (screens, services,
>>>>> SECAs, etc). Furthermore, you cannot mix XML actions with groovy or
>>>>> other artifacts except by calling into external scripts and things
>>>>> like that. In comparison to moqui which is more flexible in XML
>>>>> actions I find it restrictive to work with OFBiz actions and it's
>>>>> hard to review, debug, or modify. I'm listing something below as an
>>>>> example for comparison [1]*
>>>> I will have to investigate more how Moqui does those things. I'm not
>>>> sure what the example shows. Is it part of the controller logic?
>>>>
>>>> Generally, I think that it is a strength to have these tasks separated
>>>> in an MVC pattern and not being able to mix controller logic with
>>>> business logic.
>>>>
>>>> When it comes to debugging: how do you debug the shown XML code?
>>>>
>>>>
>>>>> *
>>>>>
>>>>> _5. Limited Routing capability:_**Routing is very minimal
>>>>> (controller.xml). It cannot contain business logic and is limited
>>>>> either to chaining requests or rendering screens. Sophisticated
>>>>> routing logic is important for making rich systems that react to
>>>>> differing circumstances.
>>>>> **
>>>> See above: the controller part should NOT contain too much business
>>>> logic. If the business logic would be mixed with the controller logic,
>>>> how would you then make the business logic more shareable (see point 3
>>>> above)?
>>>>
>>>> Maybe I'm missing something and some examples would help me understand
>>>> the point.
>>>>
>>>>
>>>>> _6. Core framework difficult to change:_**OFBiz core was written
>>>>> a/long/time ago, and back then you didn't have many libraries for
>>>>> caching, transactions, DB pooling, Concurrency stuff, and others, so
>>>>> a lot was hand crafted, and we have a lots of interconnected API that
>>>>> is difficult to change, and most of these APIs depend on static util
>>>>> files that just kept piling and piling over the years. Where is the
>>>>> service engine implementation exactly? What about the entity engine?
>>>>> What about widgets? They are all over the place! Pieces for XML
>>>>> parsing, others for DOM models, and so on. The code is scattered and
>>>>> fixing it means going through a maze of many classes that are
>>>>> interconnected and all referencing each other. It's also hard to
>>>>> identify the signature or figure out the "big picture" since you have
>>>>> essentially minimal interfaces and no clear public API that you can
>>>>> refactor against. Pretty much the whole thing is public API**[2].
>>>>> **
>>>> That's a big field and I think we should break this into subtopics to
>>>> discuss. Can you be more specific, for me this is too broad to
>>>> discuss. What is the real problem, what would be a better solution, 
>>>> ...?
>>>>
>>>> In general, a framework like OFBiz IS complex, it will never be easy
>>>> to read/understand. And of course, there is code which should be
>>>> refactored and better maintained (deprecation etc.). It would maybe be
>>>> a good start to identify those and do refactorings there first to
>>>> reduce complexity and pave the path for bigger refactorings?
>>>>
>>>>  From my experience, e.g. the caching capabilities of OFBiz are a big
>>>> strength and to change those parts, we need some really good proof of
>>>> concepts to check if other solutions are really better. I've had
>>>> several bad experiences with, for example, object relational mapping
>>>> frameworks (performance, complexity) and I'm still a fan of the entity
>>>> engine.
>>>>
>>>>
>>>>> _7. Web-apps design problem__:_ A big problem (in my opinion) in the
>>>>> design is around having multiple web-apps, one for each component (or
>>>>> more). This interrupts the flow of the app and makes it difficult to
>>>>> come up with smooth solutions that work across the various screens
>>>>> and instead we need to pass a key between the components (and face
>>>>> other problems around session management). This restricts you in what
>>>>> you can offer as a user experience.*
>>>>> *
>>>> Do you have a solution in mind to solve this and also keep those
>>>> applications separated (see above)? Single Sign On?
>>>>
>>>>
>>>>> _8. Problematic UI design__:_ Another/big/problem (in my opinion) is
>>>>> having the UI show everything (reference implementation). I never
>>>>> understood the value of showing/everything/! If I want to understand
>>>>> the data model, I will just look at the data model. The purpose of
>>>>> the UI is not to show everything, but to implement specific useful
>>>>> use-cases. The reference implementation we have is not doing that,
>>>>> and not offering something very useful, instead it is showing
>>>>> everything! To improve this, we need to switch from data-model based
>>>>> interface (party, order, etc ...) to use-case based interface
>>>>> (project management, e-commerce, ERP, CRM, etc ...)*
>>>>> *
>>>> That's indeed a big issue and part of the discussions with every new
>>>> user who wants to utilize the existing web apps. On one hand it is
>>>> great to be able to show the potential of the ecosystem, on the other
>>>> hand it's too complex, especially for new users.
>>>>
>>>> IMO this is not so much a problem of the framework in general, which
>>>> does not prevent us from developing better UI and workflows.
>>>> Personally, I see OFBiz as an ERP framework which has to be customized
>>>> to match the target users's needs and not so much as an 
>>>> ready-to-use ERP.
>>>>
>>>> But yes, this can and should be improved with additional web apps or
>>>> even ONE web app which tightly integrates all core applications into a
>>>> one-stop ERP solution which is more user-friendly than the base
>>>> applications. On the other hand, this will make separation more
>>>> difficult...
>>>>
>>>>
>>>>> _9. Limited client rendering:_Because it is difficult to refactor the
>>>>> framework, it's hard to pin-point where to go exactly to refactor all
>>>>> the code needed to support more client-rendering technology (react,
>>>>> vue, angular, etc ...) and make it interact smoothly with the
>>>>> back-end (drop-downs, complex forms, etc ...). A lot of work needs to
>>>>> happen both to the /themes and to /framework to allow and provide
>>>>> more flexibility (also moving things to DB). If this step is not done
>>>>> you will be stuck with your UI capabilities and won't catch up
>>>>> (unless you develop your own UI which makes the UI of the community
>>>>> redundant and you let go of the whole widget system anyway).*
>>>>> *
>>>> I guess it would help a lot if we have REST integrated into the core
>>>> framework and expose a proper REST API. For the HTML based web UI, I
>>>> have not much to change as the widget/freemarker templating gives us
>>>> much flexibility.
>>>>
>>>> What do you miss exactly / in which direction do you want to do 
>>>> changes?
>>>>
>>>>
>>>>> _10. Security is difficult to implement:_One of my favorite features
>>>>> in moqui is having security baked in the artifacts themselves to
>>>>> avoid sprinkling all screens / services with security logic. I never
>>>>> even worry about writing security related stuff until way close to
>>>>> the end of delivering the project. So there needs to be a whole
>>>>> security layer that sits below all the artifacts such that we don't
>>>>> even need to write security code in the various screens and services
>>>>> and around certain entities.*
>>>>> *
>>>> Can you elaborate more on how you want to achieve fine grained
>>>> permissions for screens and services if not tied to the screen/service
>>>> configurations (having in mind that they should be reusable)?
>>>>
>>>>
>>>>> _11. Limited Integration Solutions__:_ I know we have now a REST
>>>>> component, but this is not the same as being a core feature of the
>>>>> framework (essential for any modern web framework). Why should it be
>>>>> tightly integrated? Many reasons: security, localization, mapping to
>>>>> service engine, mapping to entities, etc ...*
>>>>> *
>>>> I'm currently working on the rest-api plugin and I agree that it would
>>>> be great to have it in the framework instead of being a plugin.
>>>>
>>>>
>>>>> _12. No Specialization:_Because of the big repository, and having
>>>>> everything point to everything, it's hard to specialize. Now compare
>>>>> this to the approach we have in our work which is easier to
>>>>> specialize in. For example, I have a component for payment gateway
>>>>> integration, another one for twilio (SMS integration), and I have
>>>>> application components for each solution in its own repository (HR,
>>>>> insurance stuff, many others) and each is living in its own git repo
>>>>> yet they work together just fine and without touching one line of
>>>>> code in the framework or even core components (despite depending on
>>>>> them and using pieces of them). I can jump into new versions of the
>>>>> framework everyday without breaking a sweat. When I work on component
>>>>> X, I'm only thinking of that and nothing else. I don't have to worry
>>>>> about changing something that would crash another component or bring
>>>>> the system down (happened to me many times in the past).
>>>> For our projects, we were going the other way: we have framework +
>>>> plugins in the same repository instead of handling different
>>>> repositories. I think it would be a real pain to split the project in
>>>> dozens of different repositories only to bring them together to
>>>> integrate during development or deployment again.
>>>>
>>>> Can you describe how you are handling those different repositories in
>>>> an efficient way to get a better picture?
>>>>
>>>> As long as your work in a component/application/plugin and don't touch
>>>> the framework code I also do not see why you should break something
>>>> for other components? Of course, there ARE applications which are more
>>>> fundamental than other (e.g. party, security) and if those are changed
>>>> they might break other functionality. I see those applications
>>>> belonging to the framework more than being independent applications.
>>>>
>>>> I agree that we should try to identify false or unnecessary
>>>> dependencies though.
>>>>
>>>>
>>>>>
>>>>>     Solutions
>>>>>
>>>>> Whatever solutions we come up with I think they're going to be
>>>>> difficult and would require community buy-in and support. I'm
>>>>> breaking this down into things to keep, update, add and remove as
>>>>> follows:
>>>>>
>>>>>
>>>>>       Keep it
>>>>>
>>>>> - XSD signatures (so entity definitions, service definitions, XML
>>>>> actions though should expand)
>>>>> - Basic constructs of what is a component (with basic directory
>>>>> structure)
>>>>> - Data model
>>>>> - Existing services (core data-model services)
>>>>> - Java / groovy code that doesn't use core framework constructs or
>>>>> goes too deep into the internals.
>>>>> - Most used and useful APIs (Making queries, calling services, etc
>>>>> ...) although it should all be converted to interfaces with separate
>>>>> implementations where it's lacking.
>>>>>
>>>> Agree to all above.
>>>>
>>>>
>>>>>       Improve It
>>>>>
>>>>> - Break the project to smaller git repositories.
>>>>>    - framework
>>>>>    - data model
>>>>>    - services
>>>>>    - common UI
>>>>>    - use-case applications (ERP, CRM, e-commerce, etc ...)
>>>>>    - plugins
>>>>> - entire framework directory needs refurbishment and big chunks
>>>>> probably require a delete / rewrite
>>>>> - the web architecture (get rid of web-apps)
>>>>> - the tests (lots of them need to be deleted anyway)
>>>>> - Routing needs to be programmatic with ability to inject code.
>>>>>
>>>>>
>>>>>       Add It
>>>>>
>>>>> - Configuration override mechanism to remove the need to touch the
>>>>> core. Overrides include everything including routing, component
>>>>> registration, etc ...
>>>>> - REST integration into the core
>>>>> - Artifact based security that maps users / groups to artifacts
>>>>> without the need to inject security code into every screen
>>>>> - A super-set of XML actions that can be applied in all contexts and
>>>>> can embed groovy code
>>>>> - New, simple use-case based user interface. It needs to be designed
>>>>> such that it is easy to extend to avoid the all-or-nothing UI that
>>>>> currently exists.
>>>>>
>>>>>
>>>>>       Remove It / Deprecate It / Overwrite It
>>>>>
>>>>> - Most of the existing UI (needs to be replaced with simpler use-case
>>>>> based interface)
>>>>> - Lots of the core framework code (needs major rewrites, needs
>>>>> interfaces, import libraries for things like transactions, security,
>>>>> concurrency, etc ...)
>>>>> - Probably some of the existing plugins might need to be deprecated.
>>>> See my questions above for some of the topics. I will add more details
>>>> and my point-of-view once we've split up the topics.
>>>>
>>>> The overall change would be huge and we should paint a clear picture
>>>> of what we want to achieve and which way to go in detail.
>>>>
>>>> We'll have to prioritize with a good cost/effort consideration and do
>>>> implementations step-by-step with a proper migration plan. Maybe
>>>> having feature branches for a longer time to be able to put out
>>>> releases if the development gets stuck or lasts longer.
>>>>
>>>>
>>>>>
>>>>>       Quality Control
>>>>>
>>>>> Investing in such a huge effort requires that we change the way we
>>>>> view the project after / while work is being completed. Code should
>>>>> be approached with greater care and a lot of emphasis should be put
>>>>> on high quality, readability, avoiding anti-patterns, enforcing good
>>>>> patterns and best-practices. Some of the daily workflows need to
>>>>> improve / change, more thorough auditing and peer-reviews, and so on.
>>>> A big +1 for quality control. This will be a real challenge for the
>>>> project as it requires dedication and enough ressources for review.
>>>>
>>>>
>>>>>
>>>>>     Tradeoffs
>>>>>
>>>>> It goes without saying that the above is a _lot_ of work and
>>>>> commitment and requires a lot of effort. That's why I hesitated in
>>>>> starting this thread, but I thought maybe showing the tradeoffs would
>>>>> make it easier to weigh things out and decide on a direction to push.
>>>>>
>>>>>
>>>>>       Pros
>>>>>
>>>>> - Pretty much most of the above mentioned problems will be resolved.
>>>>> - The framework would be resilient and future proof.
>>>>> - We will have more options and capabilities in the future, it would
>>>>> be easier to incorporate new technologies as they emerge
>>>>> - Development would go faster
>>>>> - Technical Debt will be reduced
>>>>>
>>>>>
>>>>>       Cons
>>>>>
>>>>> - Huge (maybe too huge) upfront costs in time and effort to make the
>>>>> switch
>>>>> - Extra effort needed to ensure a smooth upgrade path for existing
>>>>> users / adopters / system integrators. A whole layer of code needs to
>>>>> be written to ensure the ability to upgrade without massive rewrites.
>>>>> - Risk of failure, exhaustion, inability to push through to the end
>>>>> - Inconvenience to those with existing systems / solutions
>>>>>
>>>>>
>>>>>     Summary
>>>>>
>>>>> Change is the only constant. Inability to adapt to the future means
>>>>> not surviving. I'm not confident what is the best change possible or
>>>>> at what pace, but I do know that lack of change is a death sentence
>>>>> to any project. Maybe it's time to reflect and think of where we'd
>>>>> like to go and this thread is sort of a brain-dump of possible ideas.
>>>>>
>>>>> I look forward to hearing your thoughts.
>>>> Thanks, Taher, for the thoughtful write-up of your view on the project
>>>> and the fields of change you see to be tackled. I do not agree with
>>>> all of them and I have a sensible view on cost/effort, thorough
>>>> planning and the need to always have existing users/projects in mind.
>>>> But I hope that we/the project will find a way to constantly work on
>>>> improvements.
>>>>
>>>> Again, I would propose to split up the most important topics into
>>>> threads with accompanying documentation in the wiki to help avoid
>>>> confusion.
>>>>
>>>> I am curious what others have to say and if we can activate enough
>>>> interest to improve.
>>>>
>>>>
>>>>> Cheers,
>>>>>
>>>>> Taher Alkhateeb
>>>>>
>>>>>
>>>>> [1] Example mixed XML actions and groovy from moqui
>>>>>
>>>>> <if condition="settlementTerm?.orderPmtServiceRegisterId">
>>>>>      <then>
>>>>>          <service-call name="update#mantle.order.OrderPart"
>>>>>                  in-map="[orderId:orderId,
>>>>> orderPartSeqId:orderPartSeqId, settlementTermId:settlementTermId]"/>
>>>>>
>>>>>          <entity-find-one entity-name="moqui.service.ServiceRegister"
>>>>> value-field="serviceRegister">
>>>>>              <field-map field-name="serviceRegisterId"
>>>>> from="settlementTerm.orderPmtServiceRegisterId"/>
>>>>>          </entity-find-one>
>>>>>          <script><![CDATA[
>>>>>              try {
>>>>>
>>> ec.service.sync().name(serviceRegister.serviceName).parameters(context).softValidate(true).disableAuthz().call() 
>>>
>>>
>>>>>                  if (ec.message.hasError()) return
>>>>>              } catch (Throwable t) {
>>>>>                  ec.logger.log(300, "Error calling auto order 
>>>>> payments
>>>>> service for SettlementTerm ${settlementTermId}", t)
>>>>>              }
>>>>>          ]]></script>
>>>>>      </then>
>>>>>      <else>
>>>>>          <service-call
>>>>> name="mantle.account.PaymentServices.create#Payment" 
>>>>> out-map="context"
>>>>>                  in-map="context + [amount:amount,
>>>>> amountUomId:amountUomId, fromPartyId:fromPartyId,
>>>>> toPartyId:toPartyId]"/>
>>>>>      </else>
>>>>> </if>
>>>>>
>>>>> [2] https://ci.apache.org/projects/ofbiz/site/trunk/javadocs/
>>>>>