Re: Framework refactor (Moqui-/Mantle-specific)

[David E Jones <de...@me.com>]

I think the Moqui Framework is already to a point where migration of
OFBiz business-level artifacts could begin immediately.

Doing a migration like this would bring up other issues... including
whether or not to clean up the data model and services while at it,
especially rewriting messier parts of OFBiz like the ShoppingCart*
objects and order processing stuff in general.

If you want to stick with everything as-is and just do a migration, that
should actually be much less work than you might think. I have some FTL
templates to transform entity and service definitions, and a partial
template for simple-methods. Similar templates could be made for screens
and forms, though they are more complicated and may require more manual
work. Things like groovy scripts and Java classes will also require a
bit more work, but many patterns are similar so it would be much less
work than moving to an object-relational sort of framework (ie
Hibernate/JPA type of thing).

While on the topic, some notes about things I saw in some related threads:

1. the DSL/Helper class effort is in principle just an improved API for
framework tools; the OFBiz framework tools were originally implemented
independently, and based on a LOT of static methods and static
initialization, resulting in an "API" that is made up of a bunch of
independent objects that I've found are opaque even to developers with
years of OFBiz experience (ie they are not aware of functionality
because it is so hard to find); the Moqui API takes care of this problem
with an API designed to actually be an API as opposed to a bunch of
independent classes that evolved; for more info on that, see:

http://www.moqui.org/apiJavadoc/index.html

... and the central object of the API (that also has contextual details
for the current request/user/etc):

http://www.moqui.org/apiJavadoc/org/moqui/context/ExecutionContext.html

Here are some quick examples of how this looks in actual use (from the
sendEmailTemplate.groovy script; sorry for the wrapping):

def emailTemplate =
ec.entity.makeFind("moqui.basic.email.EmailTemplate").condition("emailTemplateId",
emailTemplateId).one()

String bodyText =
ec.screen.makeRender().rootScreen(emailTemplate.bodyScreenLocation).renderMode("text").render()

Map cemParms = [sentDate:ec.user.nowTimestamp, subject:subject,
body:bodyHtml, fromAddress:emailTemplate.fromAddress,
toAddresses:toAddresses, ccAddresses:emailTemplate.ccAddresses,
bccAddresses:emailTemplate.bccAddresses, contentType:"text/html",
emailTemplateId:emailTemplateId,fromUserId:ec.user.userId]
ec.service.sync().name("create",
"moqui.basic.email.EmailMessage").parameters(cemParms).call()

2. consolidating OFBiz data model and services into a single component:
this is the way the Mantle UDM and USL are currently organized, though
they are restricted to very generic stuff and application-specific
extensions would live in the corresponding application

3. running in a single webapp: while this isn't necessary with Moqui,
the Moqui Screens are a combination of the controller.xml entries for
the particular screen and the OFBiz Screen Widget, and are hierarchical
instead of being flat like the request-map URIs in OFBiz; this allows
apps to plug into a screen hierarchy from separate places (by explicit
inclusion, a database record, or implicitly by directory structure)

================================

This might actually be good in a separate email, but anyway...

The Moqui Framework would be a separate project from OFBiz, and while
some OFBiz committers could certainly become moderators for Moqui that
would be based on their personal efforts and merits and not anything to
do with what happens or has happened in OFBiz.

Moqui has a lot more extension points built-in than the OFBiz Framework,
so you can actually do quite a bit without changing the framework
itself. For cases where changes are needed (or wanted) to Moqui itself
they would go through the normal feature request and patch submission
process for Moqui to be reviewed by a moderator. Initially this would be
me, and I obviously have some skin invested in an effort like this so
I'd try to be responsive as possible (I have been so far with Moqui, but
most of the time it has had light traffic so I've been able to get
things reviewed and changed/fixed within a couple of days).

One of the easiest ways to do that is with a pull request on GitHub. In
fact, if the OFBiz variation of the Moqui Framework was managed on
GitHub you could customize things there as desired while still being
able to submit changes back to Moqui and get updates from Moqui so that
OFBiz can more easily participate in the Moqui community. In git
terminology this would actually be a "fork", and I think is what Jacopo
was talking about.

Still, for the most part my guess is that this would not be needed and
I'd recommend starting with just the jar files (or even just the war
file) and not set things up for easily changing the framework until the
need for doing so is well established.

For this interested in considering Moqui for use in OFBiz, the tutorial
and framework intro docs are probably the best places to start, and the
Run/Deploy doc is also helpful (BTW, the moqui.org site is actually
running on Moqui, and the tutorial and run/deploy docs are actually
cwiki documents (confluence-style wiki text)):

http://www.moqui.org/framework/docs/Tutorial.html

http://sourceforge.net/projects/moqui/files/IntroductionToMoquiFramework-1.0.1.pdf/download

http://www.moqui.org/framework/docs/RunDeploy.html

Anyway, sorry for the delay in responding to this. I'm still a bit busy
on contracts and I only catch up on the OFBiz mailing lists every week
or so.

-David




Jacopo Cappellato wrote:
> On Mar 2, 2012, at 7:50 AM, Hans Bakker wrote:
> 
>> Jacopo,
>>
>> You would even consider forking?
> 
> From Wikipedia [*]:
> 
> "[...] More recently, distributed revision control (DVCS) tools have popularised a less emotive use of the term "fork", blurring the distinction with "branch". With a DVCS such as Mercurial or Git, the normal way to contribute to a project is to first branch the repository, and later seek to have your changes integrated with the main repository. Sites such as Github, Bitbucket and Launchpad provide free DVCS hosting expressly supporting independent branches, such that the technical, social and financial barriers to forking a source code repository are massively reduced."
> 
> In order of preference (descending), here are the options I see for the future of the OFBiz framework:
> 
> 1) develop a great Apache OFBiz framework 2.0 within the OFBiz community; then release it separately from the Apache OFBiz ERP
> 2) greatly clean up and improve the existing framework (I was not sure if this could go at #1)
> 3) if the above will not be possible (frankly speaking, in the committers group, apart from David, none of us ever implemented with success an open source framework) we should also consider to drop the existing code and have our community focusing on the ERP part (as Hans seems to advocate); at this point Moqui would be the most natural choice; if we will ever go with this path a great exchange of information will have to happen between the two projects: for example OFBiz will probably have to ask the Moqui framework to evolve some of its features; given the current nature of the Moqui project, I doubt that the OFBiz committers will be ever invited as committers there; if Moqui will be our choice, I see two possibilities:
> 3.a) we base the Apache OFBiz ERP on a release of Moqui: this will only be possible if Moqui release will have all the features we need (and if Moqui community will be interested in getting contribution to evolve in the direction required by OFBzi)
> 3.b) if 3.a will not be possible because OFBiz will need some features that Moqui community will not consider as a good fit for Moqui, then, under the guidance and bless of David, we could work on a fork: get the code from a Moqui release, import in our repository and add to it, in a controlled way, the features we need; of course this should be always kept as close as possible to the original code; we could synch our custom code with every new Moqui release; I was not thinking about *stealing* code to Moqui and the fact that David is both the founder of OFBiz and of Moqui and he is both in the OFBiz PMC and the leader of the Moqui project will definitely facilitate this; but it will be still an ugly solution but for example when you said: "My proposal is that Apache OFBiz will be in the future just the ERP system based on many opensource products like birt and also Moqui...." you are actually implying that the ERP applications will be able to use Birt... but this requires 
some sort of framework and what would you do if Moqui will not think that Birt is a good fit for them?
> 4) if Moqui will not be a good option we may consider other frameworks (?), but it will be difficult, or continue with what we have
> 
> Jacopo
> 
> 
> 
> [*]: http://en.wikipedia.org/wiki/Fork_(software_development)

Re: Framework refactor (Moqui-/Mantle-specific)

[David E Jones <de...@me.com>]

NOTE: replying to this in multiple messages for better digestibility. :)

Jacopo Cappellato wrote:
> Thank you David,
> 
> please see inline:
> 
> On Mar 15, 2012, at 6:08 AM, David E Jones wrote:
> 
>> I think the Moqui Framework is already to a point where migration
>> of OFBiz business-level artifacts could begin immediately.
> 
> Of course I would be happy to see (and help with) an effort to create
> a relatively complex proof of concepts for the migration. Then we
> will have a better ground to evaluate pros and cons of the two
> frameworks.

There is an example application in Moqui that is similar to the one in
OFBiz, though focusing on the main screens and not including all of the
JS and other feature demo screens.

The online demo (running on Amazon Elastic Beanstalk) is here:

http://demo.moqui.org/apps/example/Example

The source code can be viewed through GitHub here:

https://github.com/jonesde/moqui/tree/master/runtime/component/example

-David

Re: Framework refactor (Moqui-/Mantle-specific)

[Adrian Crum <ad...@sandglass-software.com>]

On 3/15/2012 9:20 AM, Jacopo Cappellato wrote:
> On Mar 15, 2012, at 9:46 AM, Jacopo Cappellato wrote:
>
>> Adrian, it was a joke not real criticism on you.
> And to be very clear: I appreciate all the work you did to add support to JSR-223

We're cool. ;-)

-Adrian

Re: Framework refactor (Moqui-/Mantle-specific)

[Jacopo Cappellato <ja...@hotwaxmedia.com>]

On Mar 15, 2012, at 9:46 AM, Jacopo Cappellato wrote:

> Adrian, it was a joke not real criticism on you.

And to be very clear: I appreciate all the work you did to add support to JSR-223

Jacopo

Re: Framework refactor (Moqui-/Mantle-specific)

[Jacopo Cappellato <ja...@hotwaxmedia.com>]

On Mar 15, 2012, at 9:38 AM, Adrian Crum wrote:

> On 3/15/2012 7:57 AM, Jacopo Cappellato wrote:
>> Not only this; well, yes it is simply this at the technical level but it is much much more in the end result and approach: in fact in my poc I have laid out some simple but very effective best practices for programming services and events: the code I added is minimal (a few lines... that after Adrian's refactoring became 10 times more... but the OFBiz community likes heavy stuff :-) )
> 
> The refactoring might have added a few lines to the Groovy implementation, but it also eliminates the need for these classes:
> 
> BshUtil.java
> GroovyUtil.java
> BeanShellEngine.java
> BSFEngine.java
> GroovyEngine.java
> BsfEventHandler.java
> GroovyEventHandler.java
> 
> and when mini-language implements JSR-223, these classes will not be needed:
> 
> SimpleEventHandler.java
> SimpleServiceEngine.java
> 
> The refactoring also adds support for many more scripting languages.
> 
> So, I'm having a hard time understanding how the elimination of nine classes (1100 lines of code) while adding support for more scripting languages can be considered "heavy stuff."

Adrian, it was a joke not real criticism on you.
However I was not talking about the JSR-223 work in general but about the conversion of the specific DSL language class into the helper/factory pattern; that added a lot of code to a poc class that was intended to enhance Groovy for OFBiz. But you are right, in *theory* 100 different languages could use the new helper classes now and at that point the overhead would make a lot of sense: but we will never really know if they will be used out of OFBiz.

Jacopo

> 
> -Adrian
> 

Re: Framework refactor (Moqui-/Mantle-specific)

[Adrian Crum <ad...@sandglass-software.com>]

On 3/15/2012 7:57 AM, Jacopo Cappellato wrote:
> Not only this; well, yes it is simply this at the technical level but 
> it is much much more in the end result and approach: in fact in my poc 
> I have laid out some simple but very effective best practices for 
> programming services and events: the code I added is minimal (a few 
> lines... that after Adrian's refactoring became 10 times more... but 
> the OFBiz community likes heavy stuff :-) )

The refactoring might have added a few lines to the Groovy 
implementation, but it also eliminates the need for these classes:

BshUtil.java
GroovyUtil.java
BeanShellEngine.java
BSFEngine.java
GroovyEngine.java
BsfEventHandler.java
GroovyEventHandler.java

and when mini-language implements JSR-223, these classes will not be needed:

SimpleEventHandler.java
SimpleServiceEngine.java

The refactoring also adds support for many more scripting languages.

So, I'm having a hard time understanding how the elimination of nine 
classes (1100 lines of code) while adding support for more scripting 
languages can be considered "heavy stuff."

-Adrian

Re: Framework refactor (Moqui-/Mantle-specific)

[Adrian Crum <ad...@sandglass-software.com>]

The mini-lang overhaul only effects the grammar, not the performance. No 
one can comment on the framework performance until a decision is made 
about the direction it will take.

-Adrian

On 3/16/2012 8:09 AM, Pierre Smits wrote:
> Hi all,
>
> Currently there are several threads regarding the overhaul of the
> framework, the mini-lang, and other structures. But how will all these
> affect the performance?
>
> I haven't seen anything on that subject.
>
> Regards,
>
> Pierre
>
> Op 16 maart 2012 02:15 schreef David E Jones<de...@me.com>  het volgende:
>
>>
>> Jacopo Cappellato wrote:
>>>> 3. running in a single webapp: while this isn't necessary with
>>>> Moqui, the Moqui Screens are a combination of the controller.xml
>>>> entries for the particular screen and the OFBiz Screen Widget, and
>>>> are hierarchical instead of being flat like the request-map URIs in
>>>> OFBiz; this allows apps to plug into a screen hierarchy from
>>>> separate places (by explicit inclusion, a database record, or
>>>> implicitly by directory structure)
>>> The existing OFBiz screen/form widget is an area where OFBiz should
>>> be greatly improved in my opinion. Some of the things I don't like: *
>>> the "configuration by exception" patter should be used more
>>> extensively (most common screen definitions, controller entries
>>> etc... should not be required) * it is difficult to build complex
>>> Ajax screens * I don't like the xml code for screen/form actions: it
>>> is limited and it is an additional programming language to learn * in
>>> simpler screen it is an overkill to maintain in separate files the
>>> data preparation script, the form definition and the screen
>>> definition * the html code that is produced by the widgets is ugly
>>> and difficult to maintain
>>>
>>> It seems that some of the points above have been addressed by Moqui;
>>> David, did you ever consider to integrate other ui frameworks in
>>> Moqui? I know that in Moqui there is a more powerful mechanism to
>>> automatically render ui elements (e.g. submenu items for subscreens
>>> etc...) but frankly speaking I am less interested in them because I
>>> have learned that these tools are great to cover the 80% of the
>>> requirements of a ui but the work on the remaining 20% still has to
>>> be done in each and every screen to make it really usable. But I
>>> would be really interested to see a rather complex Moqui screen that
>>> uses Ajax and calls events: this would help me to evaluate the
>>> advantages of the new solution.
>> I don't know of any magic formulas for web-based UI stuff. Even a lot of
>> the desktop UI stuff is quite a pain and only elaborate visual-design
>> tools make it easier... and when those aren't adequate for what you want
>> to build life gets hard in a hurry.
>>
>> One thing that the Moqui screens have that help with this is that it is
>> easier to extend the form and screen definitions by just adding macros
>> to an FTL template that extends and/or overrides the default templates.
>> You can add your new tags to an XSD file if you want so that things will
>> validate properly in your IDE, but that is not necessary for the new
>> tags to function.
>>
>> This still isn't a silver bullet, but at least it makes it easier to
>> incorporate consistent UI widgets based on jQuery plugins or your own
>> hand-rolled stuff or whatever so that it is not necessary to have HTML
>> and/or JavaScript snippets to use each of these on each screen/page that
>> needs them.
>>
>> It is also easier to inline HTML and JavaScript in a screen definition,
>> so that doesn't have to be maintained in a separate file... and have
>> have HTML inline in a form field and such. In other words, you can drop
>> it just about anywhere, making it much easier to do crazy stuff with the
>> screen and form XML files so you don't have to drop to FTL templates so
>> often.
>>
>> OOTB the Moqui XML Screens and XML Forms use more JavaScript and dynamic
>> stuff than the OFBiz ones, but there is still a LOT of room for
>> improvement and it is something I'd like to do more of, or work with
>> other who do it to get it into the project (any takers? :) ).
>>
>> If you have other ideas about what an artifact might look like that
>> would be easier for UI stuff, I would be very interested in discussing
>> it more. You have a lot of good ideas Jacopo, and I appreciate them a
>> great deal.
>>
>> -David
>>
>>

Re: Framework refactor (Moqui-/Mantle-specific)

[Pierre Smits <pi...@gmail.com>]

Hi all,

Currently there are several threads regarding the overhaul of the
framework, the mini-lang, and other structures. But how will all these
affect the performance?

I haven't seen anything on that subject.

Regards,

Pierre

Op 16 maart 2012 02:15 schreef David E Jones <de...@me.com> het volgende:

>
>
> Jacopo Cappellato wrote:
> >> 3. running in a single webapp: while this isn't necessary with
> >> Moqui, the Moqui Screens are a combination of the controller.xml
> >> entries for the particular screen and the OFBiz Screen Widget, and
> >> are hierarchical instead of being flat like the request-map URIs in
> >> OFBiz; this allows apps to plug into a screen hierarchy from
> >> separate places (by explicit inclusion, a database record, or
> >> implicitly by directory structure)
> >
> > The existing OFBiz screen/form widget is an area where OFBiz should
> > be greatly improved in my opinion. Some of the things I don't like: *
> > the "configuration by exception" patter should be used more
> > extensively (most common screen definitions, controller entries
> > etc... should not be required) * it is difficult to build complex
> > Ajax screens * I don't like the xml code for screen/form actions: it
> > is limited and it is an additional programming language to learn * in
> > simpler screen it is an overkill to maintain in separate files the
> > data preparation script, the form definition and the screen
> > definition * the html code that is produced by the widgets is ugly
> > and difficult to maintain
> >
> > It seems that some of the points above have been addressed by Moqui;
> > David, did you ever consider to integrate other ui frameworks in
> > Moqui? I know that in Moqui there is a more powerful mechanism to
> > automatically render ui elements (e.g. submenu items for subscreens
> > etc...) but frankly speaking I am less interested in them because I
> > have learned that these tools are great to cover the 80% of the
> > requirements of a ui but the work on the remaining 20% still has to
> > be done in each and every screen to make it really usable. But I
> > would be really interested to see a rather complex Moqui screen that
> > uses Ajax and calls events: this would help me to evaluate the
> > advantages of the new solution.
>
> I don't know of any magic formulas for web-based UI stuff. Even a lot of
> the desktop UI stuff is quite a pain and only elaborate visual-design
> tools make it easier... and when those aren't adequate for what you want
> to build life gets hard in a hurry.
>
> One thing that the Moqui screens have that help with this is that it is
> easier to extend the form and screen definitions by just adding macros
> to an FTL template that extends and/or overrides the default templates.
> You can add your new tags to an XSD file if you want so that things will
> validate properly in your IDE, but that is not necessary for the new
> tags to function.
>
> This still isn't a silver bullet, but at least it makes it easier to
> incorporate consistent UI widgets based on jQuery plugins or your own
> hand-rolled stuff or whatever so that it is not necessary to have HTML
> and/or JavaScript snippets to use each of these on each screen/page that
> needs them.
>
> It is also easier to inline HTML and JavaScript in a screen definition,
> so that doesn't have to be maintained in a separate file... and have
> have HTML inline in a form field and such. In other words, you can drop
> it just about anywhere, making it much easier to do crazy stuff with the
> screen and form XML files so you don't have to drop to FTL templates so
> often.
>
> OOTB the Moqui XML Screens and XML Forms use more JavaScript and dynamic
> stuff than the OFBiz ones, but there is still a LOT of room for
> improvement and it is something I'd like to do more of, or work with
> other who do it to get it into the project (any takers? :) ).
>
> If you have other ideas about what an artifact might look like that
> would be easier for UI stuff, I would be very interested in discussing
> it more. You have a lot of good ideas Jacopo, and I appreciate them a
> great deal.
>
> -David
>
>

Re: Framework refactor (Moqui-/Mantle-specific)

[David E Jones <de...@me.com>]

Jacopo Cappellato wrote:
>> 3. running in a single webapp: while this isn't necessary with
>> Moqui, the Moqui Screens are a combination of the controller.xml
>> entries for the particular screen and the OFBiz Screen Widget, and
>> are hierarchical instead of being flat like the request-map URIs in
>> OFBiz; this allows apps to plug into a screen hierarchy from
>> separate places (by explicit inclusion, a database record, or
>> implicitly by directory structure)
> 
> The existing OFBiz screen/form widget is an area where OFBiz should
> be greatly improved in my opinion. Some of the things I don't like: *
> the "configuration by exception" patter should be used more
> extensively (most common screen definitions, controller entries
> etc... should not be required) * it is difficult to build complex
> Ajax screens * I don't like the xml code for screen/form actions: it
> is limited and it is an additional programming language to learn * in
> simpler screen it is an overkill to maintain in separate files the
> data preparation script, the form definition and the screen
> definition * the html code that is produced by the widgets is ugly
> and difficult to maintain
> 
> It seems that some of the points above have been addressed by Moqui;
> David, did you ever consider to integrate other ui frameworks in
> Moqui? I know that in Moqui there is a more powerful mechanism to
> automatically render ui elements (e.g. submenu items for subscreens
> etc...) but frankly speaking I am less interested in them because I
> have learned that these tools are great to cover the 80% of the
> requirements of a ui but the work on the remaining 20% still has to
> be done in each and every screen to make it really usable. But I
> would be really interested to see a rather complex Moqui screen that
> uses Ajax and calls events: this would help me to evaluate the
> advantages of the new solution.

I don't know of any magic formulas for web-based UI stuff. Even a lot of
the desktop UI stuff is quite a pain and only elaborate visual-design
tools make it easier... and when those aren't adequate for what you want
to build life gets hard in a hurry.

One thing that the Moqui screens have that help with this is that it is
easier to extend the form and screen definitions by just adding macros
to an FTL template that extends and/or overrides the default templates.
You can add your new tags to an XSD file if you want so that things will
validate properly in your IDE, but that is not necessary for the new
tags to function.

This still isn't a silver bullet, but at least it makes it easier to
incorporate consistent UI widgets based on jQuery plugins or your own
hand-rolled stuff or whatever so that it is not necessary to have HTML
and/or JavaScript snippets to use each of these on each screen/page that
needs them.

It is also easier to inline HTML and JavaScript in a screen definition,
so that doesn't have to be maintained in a separate file... and have
have HTML inline in a form field and such. In other words, you can drop
it just about anywhere, making it much easier to do crazy stuff with the
screen and form XML files so you don't have to drop to FTL templates so
often.

OOTB the Moqui XML Screens and XML Forms use more JavaScript and dynamic
stuff than the OFBiz ones, but there is still a LOT of room for
improvement and it is something I'd like to do more of, or work with
other who do it to get it into the project (any takers? :) ).

If you have other ideas about what an artifact might look like that
would be easier for UI stuff, I would be very interested in discussing
it more. You have a lot of good ideas Jacopo, and I appreciate them a
great deal.

-David

Re: Framework refactor (Moqui-/Mantle-specific)

[David E Jones <de...@me.com>]

Jacopo Cappellato wrote:
>> Doing a migration like this would bring up other issues...
>> including whether or not to clean up the data model and services
>> while at it, especially rewriting messier parts of OFBiz like the
>> ShoppingCart* objects and order processing stuff in general.
>> 
> 
> It will be very difficult to rewrite existing stuff, and doing this
> before or after the migration wouldn't change its complexity. For
> this reason I think we can keep this topic out of this thread for
> now. However, instead of rewriting, we could clean up what we have in
> OFBiz before the migration... especially if this could simplify the
> migration or make the existing code better organized. For example, in
> another thread I was proposing to merge the existing application
> components into one, with possibly one webapp and less (and more
> consolidated) files: this may seem a huge effort but it may not be,
> because it could be done in baby steps with a lot of string
> replacements, moving/renaming files around etc... we could even
> create some scripts to simplify the effort. But again, also this
> could be done before/after the migration (or never).

Some of it could certainly be improved with a rewrite, or perhaps more
of a serious refactor. For example, using the database for shopping cart
data instead of an object in the session would be far more flexible for
a variety of order processing apps, especially outside of the normal
ecommerce submit-all-at-once flow.

But yes, this introduces another whole big set of issues and
discussions. Actually, frankly I'm surprised there is consideration of
any major framework changes, let alone moving to a new framework,
because so much effort is required to update or migrate all of the
existing business artifacts. That has killed a lot of framework
enhancement efforts over the years.

The main trick I'd see with combining webapps in the current
ofbiz/applications components is the request-map uri and view-map name
namespaces, ie the issue with duplicate names pointing to different
events and screens and such.

In Moqui these controller files would go away and the request-map and
view-map data goes into the screen definitions, but I can't think of any
changes that could be made now and supported by OFBiz to help move
toward this.

>> If you want to stick with everything as-is and just do a migration,
>> that should actually be much less work than you might think. I have
>> some FTL templates to transform entity and service definitions, and
>> a partial template for simple-methods. Similar templates could be
>> made for screens and forms, though they are more complicated and
>> may require more manual work. Things like groovy scripts and Java
>> classes will also require a bit more work, but many patterns are
>> similar so it would be much less work than moving to an
>> object-relational sort of framework (ie Hibernate/JPA type of
>> thing).
>
> This doesn't help much to make me comfortable :-) David, it may help
> if you could list a series of tasks (if any) we could perform in one
> of the OFBiz application components to make it more ready to be
> migrated: the tasks should ideally be doable in steps, and after each
> step the component should be usable in the existing OFBiz framework;
> but the code should be more compliant with the future layout of Moqui
> (or with what the migration scripts will expect); for example we
> could: * merge/split service, entity, screen/forms definitions in
> one/more files * merge Groovy scripts into one file with several
> methods (one per data preparation script) * merge/split Java classes
> for services * merge/split xml files with services * etc. etc..

As for what can be done in advance to prepare for it, I don't think that
splitting or combining files is too big a deal because that can be done
by a tool more easily than by a human.

The main area where Moqui differs from OFBiz Framework in terms of
artifacts per file is the screens. In Moqui there is one screen per file
as opposed to multiple screens per file, and they are hierarchical as
opposed to flat (or more of a network for decoration/inclusion/etc).
This could be done manually, but it could also be done by a tool and
perhaps more easily.

-David

Re: Framework refactor (Moqui-/Mantle-specific)

[Jacopo Cappellato <ja...@hotwaxmedia.com>]

Thank you David,

please see inline:

On Mar 15, 2012, at 6:08 AM, David E Jones wrote:

> 
> I think the Moqui Framework is already to a point where migration of
> OFBiz business-level artifacts could begin immediately.

Of course I would be happy to see (and help with) an effort to create a relatively complex proof of concepts for the migration. Then we will have a better ground to evaluate pros and cons of the two frameworks.

> 
> Doing a migration like this would bring up other issues... including
> whether or not to clean up the data model and services while at it,
> especially rewriting messier parts of OFBiz like the ShoppingCart*
> objects and order processing stuff in general.
> 

It will be very difficult to rewrite existing stuff, and doing this before or after the migration wouldn't change its complexity.
For this reason I think we can keep this topic out of this thread for now.
However, instead of rewriting, we could clean up what we have in OFBiz before the migration... especially if this could simplify the migration or make the existing code better organized.
For example, in another thread I was proposing to merge the existing application components into one, with possibly one webapp and less (and more consolidated) files: this may seem a huge effort but it may not be, because it could be done in baby steps with a lot of string replacements, moving/renaming files around etc... we could even create some scripts to simplify the effort.
But again, also this could be done before/after the migration (or never).

> If you want to stick with everything as-is and just do a migration, that
> should actually be much less work than you might think. I have some FTL
> templates to transform entity and service definitions, and a partial
> template for simple-methods. Similar templates could be made for screens
> and forms, though they are more complicated and may require more manual
> work. Things like groovy scripts and Java classes will also require a
> bit more work, but many patterns are similar so it would be much less
> work than moving to an object-relational sort of framework (ie
> Hibernate/JPA type of thing).

This doesn't help much to make me comfortable :-)
David, it may help if you could list a series of tasks (if any) we could perform in one of the OFBiz application components to make it more ready to be migrated: the tasks should ideally be doable in steps, and after each step the component should be usable in the existing OFBiz framework; but the code should be more compliant with the future layout of Moqui (or with what the migration scripts will expect); for example we could:
* merge/split service, entity, screen/forms definitions in one/more files
* merge Groovy scripts into one file with several methods (one per data preparation script)
* merge/split Java classes for services
* merge/split xml files with services
* etc. etc..

> 
> While on the topic, some notes about things I saw in some related threads:
> 
> 1. the DSL/Helper class effort is in principle just an improved API for
> framework tools;

Not only this; well, yes it is simply this at the technical level but it is much much more in the end result and approach: in fact in my poc I have laid out some simple but very effective best practices for programming services and events: the code I added is minimal (a few lines... that after Adrian's refactoring became 10 times more... but the OFBiz community likes heavy stuff :-) ) and in fact the building blocks of my work have been there since we introduced Groovy for scripts (to replace Beanshell); but since then no one really did this small step to enable Groovy (with a series of best practices and a small set of methods) to be used in a way that is perfect for OFBiz (and it was the reason that made Minilang such a productive tool).
I really think that the 50 lines of code I wrote, together with the best practices defined by the two examples I have provided (for error/event/message handling) would represent a huge step ahead if used extensively to create new code (or to convert old one).
But of course the same concepts can be easily applied to Moqui.

> the OFBiz framework tools were originally implemented
> independently, and based on a LOT of static methods and static
> initialization, resulting in an "API" that is made up of a bunch of
> independent objects that I've found are opaque even to developers with
> years of OFBiz experience (ie they are not aware of functionality
> because it is so hard to find); the Moqui API takes care of this problem
> with an API designed to actually be an API as opposed to a bunch of
> independent classes that evolved; for more info on that, see:
> 
> http://www.moqui.org/apiJavadoc/index.html
> 
> ... and the central object of the API (that also has contextual details
> for the current request/user/etc):
> 
> http://www.moqui.org/apiJavadoc/org/moqui/context/ExecutionContext.html
> 
> Here are some quick examples of how this looks in actual use (from the
> sendEmailTemplate.groovy script; sorry for the wrapping):
> 
> def emailTemplate =
> ec.entity.makeFind("moqui.basic.email.EmailTemplate").condition("emailTemplateId",
> emailTemplateId).one()
> 
> String bodyText =
> ec.screen.makeRender().rootScreen(emailTemplate.bodyScreenLocation).renderMode("text").render()
> 
> Map cemParms = [sentDate:ec.user.nowTimestamp, subject:subject,
> body:bodyHtml, fromAddress:emailTemplate.fromAddress,
> toAddresses:toAddresses, ccAddresses:emailTemplate.ccAddresses,
> bccAddresses:emailTemplate.bccAddresses, contentType:"text/html",
> emailTemplateId:emailTemplateId,fromUserId:ec.user.userId]
> ec.service.sync().name("create",
> "moqui.basic.email.EmailMessage").parameters(cemParms).call()
> 

The Moqui API is of course cleaner because it was implemented from scratch based on lesson learned so far in OFBiz, and this is a *great* reason for me to consider the migration to Moqui (another one is that you are the architect of Moqui).
However I would be really interested to see a service implementation in Moqui of one of the Java/Minilang services we have in OFBiz, like the poc I did to demonstrate my work on Groovy+DSL.

> 2. consolidating OFBiz data model and services into a single component:
> this is the way the Mantle UDM and USL are currently organized, though
> they are restricted to very generic stuff and application-specific
> extensions would live in the corresponding application
> 
> 3. running in a single webapp: while this isn't necessary with Moqui,
> the Moqui Screens are a combination of the controller.xml entries for
> the particular screen and the OFBiz Screen Widget, and are hierarchical
> instead of being flat like the request-map URIs in OFBiz; this allows
> apps to plug into a screen hierarchy from separate places (by explicit
> inclusion, a database record, or implicitly by directory structure)

The existing OFBiz screen/form widget is an area where OFBiz should be greatly improved in my opinion.
Some of the things I don't like:
* the "configuration by exception" patter should be used more extensively (most common screen definitions, controller entries etc... should not be required)
* it is difficult to build complex Ajax screens
* I don't like the xml code for screen/form actions: it is limited and it is an additional programming language to learn
* in simpler screen it is an overkill to maintain in separate files the data preparation script, the form definition and the screen definition
* the html code that is produced by the widgets is ugly and difficult to maintain

It seems that some of the points above have been addressed by Moqui; David, did you ever consider to integrate other ui frameworks in Moqui?
I know that in Moqui there is a more powerful mechanism to automatically render ui elements (e.g. submenu items for subscreens etc...) but frankly speaking I am less interested in them because I have learned that these tools are great to cover the 80% of the requirements of a ui but the work on the remaining 20% still has to be done in each and every screen to make it really usable.
But I would be really interested to see a rather complex Moqui screen that uses Ajax and calls events: this would help me to evaluate the advantages of the new solution.

Kind regards,

Jacopo

> 
> ================================
> 
> This might actually be good in a separate email, but anyway...
> 
> The Moqui Framework would be a separate project from OFBiz, and while
> some OFBiz committers could certainly become moderators for Moqui that
> would be based on their personal efforts and merits and not anything to
> do with what happens or has happened in OFBiz.
> 
> Moqui has a lot more extension points built-in than the OFBiz Framework,
> so you can actually do quite a bit without changing the framework
> itself. For cases where changes are needed (or wanted) to Moqui itself
> they would go through the normal feature request and patch submission
> process for Moqui to be reviewed by a moderator. Initially this would be
> me, and I obviously have some skin invested in an effort like this so
> I'd try to be responsive as possible (I have been so far with Moqui, but
> most of the time it has had light traffic so I've been able to get
> things reviewed and changed/fixed within a couple of days).
> 
> One of the easiest ways to do that is with a pull request on GitHub. In
> fact, if the OFBiz variation of the Moqui Framework was managed on
> GitHub you could customize things there as desired while still being
> able to submit changes back to Moqui and get updates from Moqui so that
> OFBiz can more easily participate in the Moqui community. In git
> terminology this would actually be a "fork", and I think is what Jacopo
> was talking about.
> 
> Still, for the most part my guess is that this would not be needed and
> I'd recommend starting with just the jar files (or even just the war
> file) and not set things up for easily changing the framework until the
> need for doing so is well established.
> 
> For this interested in considering Moqui for use in OFBiz, the tutorial
> and framework intro docs are probably the best places to start, and the
> Run/Deploy doc is also helpful (BTW, the moqui.org site is actually
> running on Moqui, and the tutorial and run/deploy docs are actually
> cwiki documents (confluence-style wiki text)):
> 
> http://www.moqui.org/framework/docs/Tutorial.html
> 
> http://sourceforge.net/projects/moqui/files/IntroductionToMoquiFramework-1.0.1.pdf/download
> 
> http://www.moqui.org/framework/docs/RunDeploy.html
> 
> Anyway, sorry for the delay in responding to this. I'm still a bit busy
> on contracts and I only catch up on the OFBiz mailing lists every week
> or so.
> 
> -David
> 
> 
> 
> 
> Jacopo Cappellato wrote:
>> On Mar 2, 2012, at 7:50 AM, Hans Bakker wrote:
>> 
>>> Jacopo,
>>> 
>>> You would even consider forking?
>> 
>> From Wikipedia [*]:
>> 
>> "[...] More recently, distributed revision control (DVCS) tools have popularised a less emotive use of the term "fork", blurring the distinction with "branch". With a DVCS such as Mercurial or Git, the normal way to contribute to a project is to first branch the repository, and later seek to have your changes integrated with the main repository. Sites such as Github, Bitbucket and Launchpad provide free DVCS hosting expressly supporting independent branches, such that the technical, social and financial barriers to forking a source code repository are massively reduced."
>> 
>> In order of preference (descending), here are the options I see for the future of the OFBiz framework:
>> 
>> 1) develop a great Apache OFBiz framework 2.0 within the OFBiz community; then release it separately from the Apache OFBiz ERP
>> 2) greatly clean up and improve the existing framework (I was not sure if this could go at #1)
>> 3) if the above will not be possible (frankly speaking, in the committers group, apart from David, none of us ever implemented with success an open source framework) we should also consider to drop the existing code and have our community focusing on the ERP part (as Hans seems to advocate); at this point Moqui would be the most natural choice; if we will ever go with this path a great exchange of information will have to happen between the two projects: for example OFBiz will probably have to ask the Moqui framework to evolve some of its features; given the current nature of the Moqui project, I doubt that the OFBiz committers will be ever invited as committers there; if Moqui will be our choice, I see two possibilities:
>> 3.a) we base the Apache OFBiz ERP on a release of Moqui: this will only be possible if Moqui release will have all the features we need (and if Moqui community will be interested in getting contribution to evolve in the direction required by OFBzi)
>> 3.b) if 3.a will not be possible because OFBiz will need some features that Moqui community will not consider as a good fit for Moqui, then, under the guidance and bless of David, we could work on a fork: get the code from a Moqui release, import in our repository and add to it, in a controlled way, the features we need; of course this should be always kept as close as possible to the original code; we could synch our custom code with every new Moqui release; I was not thinking about *stealing* code to Moqui and the fact that David is both the founder of OFBiz and of Moqui and he is both in the OFBiz PMC and the leader of the Moqui project will definitely facilitate this; but it will be still an ugly solution but for example when you said: "My proposal is that Apache OFBiz will be in the future just the ERP system based on many opensource products like birt and also Moqui...." you are actually implying that the ERP applications will be able to use Birt... but this requires 
> some sort of framework and what would you do if Moqui will not think that Birt is a good fit for them?
>> 4) if Moqui will not be a good option we may consider other frameworks (?), but it will be difficult, or continue with what we have
>> 
>> Jacopo
>> 
>> 
>> 
>> [*]: http://en.wikipedia.org/wiki/Fork_(software_development)

Re: Framework refactor (Moqui-/Mantle-specific)

[David E Jones <de...@me.com>]

The only library that is not friendly right now is the Redstone XMLRPC
library, which is LGPL licensed. That could be replaced by the Apache
XML-RPC project. The only reasons I used Redstone are that it is much
smaller (library bloat is a real problem), and it is a bit easier to use.

There are a couple of libraries that use CDDL (winstone servlet
container) and EPL (Mylin WikiText), but they are in binary form only so
should be fine.

If there is enough interest I'd be happy to change Moqui to use Apache
XML-RPC instead of Redstone.

-David


Jacopo Cappellato wrote:
> David,
> 
> a side note (but important): is Moqui using (or there are plans to use) jars or external tools whose license would prevent us from bundling Moqui in an OFBiz release under the ASL 2.0? This would be a show stopper...
> 
> Jacopo
> 
> On Mar 15, 2012, at 6:08 AM, David E Jones wrote:
> 
>> This might actually be good in a separate email, but anyway...
>>
>> The Moqui Framework would be a separate project from OFBiz, and while
>> some OFBiz committers could certainly become moderators for Moqui that
>> would be based on their personal efforts and merits and not anything to
>> do with what happens or has happened in OFBiz.
>>
>> Moqui has a lot more extension points built-in than the OFBiz Framework,
>> so you can actually do quite a bit without changing the framework
>> itself. For cases where changes are needed (or wanted) to Moqui itself
>> they would go through the normal feature request and patch submission
>> process for Moqui to be reviewed by a moderator. Initially this would be
>> me, and I obviously have some skin invested in an effort like this so
>> I'd try to be responsive as possible (I have been so far with Moqui, but
>> most of the time it has had light traffic so I've been able to get
>> things reviewed and changed/fixed within a couple of days).
>>
>> One of the easiest ways to do that is with a pull request on GitHub. In
>> fact, if the OFBiz variation of the Moqui Framework was managed on
>> GitHub you could customize things there as desired while still being
>> able to submit changes back to Moqui and get updates from Moqui so that
>> OFBiz can more easily participate in the Moqui community. In git
>> terminology this would actually be a "fork", and I think is what Jacopo
>> was talking about.
>>
>> Still, for the most part my guess is that this would not be needed and
>> I'd recommend starting with just the jar files (or even just the war
>> file) and not set things up for easily changing the framework until the
>> need for doing so is well established.
>>
>> For this interested in considering Moqui for use in OFBiz, the tutorial
>> and framework intro docs are probably the best places to start, and the
>> Run/Deploy doc is also helpful (BTW, the moqui.org site is actually
>> running on Moqui, and the tutorial and run/deploy docs are actually
>> cwiki documents (confluence-style wiki text)):
>>
>> http://www.moqui.org/framework/docs/Tutorial.html
>>
>> http://sourceforge.net/projects/moqui/files/IntroductionToMoquiFramework-1.0.1.pdf/download
>>
>> http://www.moqui.org/framework/docs/RunDeploy.html
>>
>> Anyway, sorry for the delay in responding to this. I'm still a bit busy
>> on contracts and I only catch up on the OFBiz mailing lists every week
>> or so.
>>
>> -David
> 

Re: Framework refactor (Moqui-/Mantle-specific)

[Jacopo Cappellato <ja...@hotwaxmedia.com>]

David,

a side note (but important): is Moqui using (or there are plans to use) jars or external tools whose license would prevent us from bundling Moqui in an OFBiz release under the ASL 2.0? This would be a show stopper...

Jacopo

On Mar 15, 2012, at 6:08 AM, David E Jones wrote:

> This might actually be good in a separate email, but anyway...
> 
> The Moqui Framework would be a separate project from OFBiz, and while
> some OFBiz committers could certainly become moderators for Moqui that
> would be based on their personal efforts and merits and not anything to
> do with what happens or has happened in OFBiz.
> 
> Moqui has a lot more extension points built-in than the OFBiz Framework,
> so you can actually do quite a bit without changing the framework
> itself. For cases where changes are needed (or wanted) to Moqui itself
> they would go through the normal feature request and patch submission
> process for Moqui to be reviewed by a moderator. Initially this would be
> me, and I obviously have some skin invested in an effort like this so
> I'd try to be responsive as possible (I have been so far with Moqui, but
> most of the time it has had light traffic so I've been able to get
> things reviewed and changed/fixed within a couple of days).
> 
> One of the easiest ways to do that is with a pull request on GitHub. In
> fact, if the OFBiz variation of the Moqui Framework was managed on
> GitHub you could customize things there as desired while still being
> able to submit changes back to Moqui and get updates from Moqui so that
> OFBiz can more easily participate in the Moqui community. In git
> terminology this would actually be a "fork", and I think is what Jacopo
> was talking about.
> 
> Still, for the most part my guess is that this would not be needed and
> I'd recommend starting with just the jar files (or even just the war
> file) and not set things up for easily changing the framework until the
> need for doing so is well established.
> 
> For this interested in considering Moqui for use in OFBiz, the tutorial
> and framework intro docs are probably the best places to start, and the
> Run/Deploy doc is also helpful (BTW, the moqui.org site is actually
> running on Moqui, and the tutorial and run/deploy docs are actually
> cwiki documents (confluence-style wiki text)):
> 
> http://www.moqui.org/framework/docs/Tutorial.html
> 
> http://sourceforge.net/projects/moqui/files/IntroductionToMoquiFramework-1.0.1.pdf/download
> 
> http://www.moqui.org/framework/docs/RunDeploy.html
> 
> Anyway, sorry for the delay in responding to this. I'm still a bit busy
> on contracts and I only catch up on the OFBiz mailing lists every week
> or so.
> 
> -David