You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Taher Alkhateeb <sl...@gmail.com> on 2017/07/14 09:59:24 UTC

[Proposal] moving all data in applications to the data-model component

 Hello Everyone,

Data loading in OFBiz is quite fragile. A small change could crash the
system and finding out the reason can be difficult. I really felt the
pain of this when trying to disentangle the framework from data
dependencies on plugins [1].

I've been thinking about a solution to reduce fragility, and here are
my thoughts:
- The less data files the better.
- The less data the better.
- The less distance (e.g. same directory) between data files the better.
- No circular dependencies (file dependencies should be like a tree).
- Fail fast [2].

Therefore, I suggest the following action points:
- Create a directory structure similar to [3] in the datamodel component.
- For any data that two or more files depend on, immediately move it
to the common-<reader>.xml file.
- For any data that does not belong to any domain (e.g. marketing,
content, etc ..) move it to the common-<reader>.xml.
- Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
- Gradually and carefully move data from all applications to the
datamodel component as per above guidelines.
- Cleanup and refactor the data as we move it including any tests that
depend on it.

What do you think of this proposal? Does it sound reasonable? Also
given the size of what I'm proposing I think we need multiple people
to work on this. So is anyone interested in taking part in this
initiative?

Cheers,

Taher Alkhateeb

[1] https://issues.apache.org/jira/browse/OFBIZ-9322
[2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
[3] directory structure:
data/
├── demo
│   ├── accounting-demo.xml
│   ├── common-demo.xml
│   ├── content-demo.xml
│   ├── humanres-demo.xml
│   ├── manufacturing-demo.xml
│   ├── marketing-demo.xml
│   ├── order-demo.xml
│   ├── party-demo.xml
│   ├── product-demo.xml
│   ├── shipment-demo.xml
│   └── workeffort-demo.xml
├── seed
│   ├── accounting-seed.xml
│   ├── common-seed.xml
│   ├── content-seed.xml
│   ├── humanres-seed.xml
│   ├── manufacturing-seed.xml
│   ├── marketing-seed.xml
│   ├── order-seed.xml
│   ├── party-seed.xml
│   ├── product-seed.xml
│   ├── shipment-seed.xml
│   └── workeffort-seed.xml
└── seed-initial
    ├── accounting-seed-initial.xml
    ├── common-seed-initial.xml
    ├── content-seed-initial.xml
    ├── humanres-seed-initial.xml
    ├── manufacturing-seed-initial.xml
    ├── marketing-seed-initial.xml
    ├── order-seed-initial.xml
    ├── party-seed-initial.xml
    ├── product-seed-initial.xml
    ├── shipment-seed-initial.xml
    └── workeffort-seed-initial.xml

Re: [Proposal] moving all data in applications to the data-model component

Posted by Ashish Vijaywargiya <as...@hotwaxsystems.com>.
+1

--
Kind Regards
Ashish Vijaywargiya
HotWax Systems - est. 1997 <http://www.hotwaxsystems.com/>


On Fri, Jul 14, 2017 at 3:29 PM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Devanshu Vyas <vy...@gmail.com>.
+1

Thanks & Regards,
Devanshu Vyas.

On Fri, Jul 14, 2017 at 3:29 PM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Taher Alkhateeb <sl...@gmail.com>.
Hi Paul,

I don't have an opinion on plugins so far as I didn't think enough
about it. I just thought since we centralized the entities in
"data-model" we might as well centralize the data itself. Hopefully
we'll discuss architectural proposals for plugins at some point but
I'm trying to focus on cleaning stuff from the inside out.

With that being said, I always welcome ideas to enrich this
discussion, and I hope more people find this thread interesting to
pitch in.

On Mon, Jul 17, 2017 at 9:00 AM, Paul Foxworthy <pa...@cohsoft.com.au> wrote:
> Hi Taher,
>
> Makes sense to me.
>
> Would any plugin defining custom entities also have a similar /data tree?
>
> Cheers
>
> Paul
>
>
> On 14 July 2017 at 19:59, Taher Alkhateeb <sl...@gmail.com>
> wrote:
>
>>  Hello Everyone,
>>
>> Data loading in OFBiz is quite fragile. A small change could crash the
>> system and finding out the reason can be difficult. I really felt the
>> pain of this when trying to disentangle the framework from data
>> dependencies on plugins [1].
>>
>> I've been thinking about a solution to reduce fragility, and here are
>> my thoughts:
>> - The less data files the better.
>> - The less data the better.
>> - The less distance (e.g. same directory) between data files the better.
>> - No circular dependencies (file dependencies should be like a tree).
>> - Fail fast [2].
>>
>> Therefore, I suggest the following action points:
>> - Create a directory structure similar to [3] in the datamodel component.
>> - For any data that two or more files depend on, immediately move it
>> to the common-<reader>.xml file.
>> - For any data that does not belong to any domain (e.g. marketing,
>> content, etc ..) move it to the common-<reader>.xml.
>> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
>> - Gradually and carefully move data from all applications to the
>> datamodel component as per above guidelines.
>> - Cleanup and refactor the data as we move it including any tests that
>> depend on it.
>>
>> What do you think of this proposal? Does it sound reasonable? Also
>> given the size of what I'm proposing I think we need multiple people
>> to work on this. So is anyone interested in taking part in this
>> initiative?
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
>> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> [3] directory structure:
>> data/
>> ├── demo
>> │   ├── accounting-demo.xml
>> │   ├── common-demo.xml
>> │   ├── content-demo.xml
>> │   ├── humanres-demo.xml
>> │   ├── manufacturing-demo.xml
>> │   ├── marketing-demo.xml
>> │   ├── order-demo.xml
>> │   ├── party-demo.xml
>> │   ├── product-demo.xml
>> │   ├── shipment-demo.xml
>> │   └── workeffort-demo.xml
>> ├── seed
>> │   ├── accounting-seed.xml
>> │   ├── common-seed.xml
>> │   ├── content-seed.xml
>> │   ├── humanres-seed.xml
>> │   ├── manufacturing-seed.xml
>> │   ├── marketing-seed.xml
>> │   ├── order-seed.xml
>> │   ├── party-seed.xml
>> │   ├── product-seed.xml
>> │   ├── shipment-seed.xml
>> │   └── workeffort-seed.xml
>> └── seed-initial
>>     ├── accounting-seed-initial.xml
>>     ├── common-seed-initial.xml
>>     ├── content-seed-initial.xml
>>     ├── humanres-seed-initial.xml
>>     ├── manufacturing-seed-initial.xml
>>     ├── marketing-seed-initial.xml
>>     ├── order-seed-initial.xml
>>     ├── party-seed-initial.xml
>>     ├── product-seed-initial.xml
>>     ├── shipment-seed-initial.xml
>>     └── workeffort-seed-initial.xml
>>
>
>
>
> --
> Coherent Software Australia Pty Ltd
> PO Box 2773
> Cheltenham Vic 3192
> Australia
>
> Phone: +61 3 9585 6788
> Web: http://www.coherentsoftware.com.au/
> Email: info@coherentsoftware.com.au

Re: [Proposal] moving all data in applications to the data-model component

Posted by Paul Foxworthy <pa...@cohsoft.com.au>.
Hi Taher,

Makes sense to me.

Would any plugin defining custom entities also have a similar /data tree?

Cheers

Paul


On 14 July 2017 at 19:59, Taher Alkhateeb <sl...@gmail.com>
wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>



-- 
Coherent Software Australia Pty Ltd
PO Box 2773
Cheltenham Vic 3192
Australia

Phone: +61 3 9585 6788
Web: http://www.coherentsoftware.com.au/
Email: info@coherentsoftware.com.au

Re: [Proposal] moving all data in applications to the data-model component

Posted by Michael Brohl <mi...@ecomify.de>.
+1

Michael

Am 14.07.17 um 11:59 schrieb Taher Alkhateeb:
>   Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>      ├── accounting-seed-initial.xml
>      ├── common-seed-initial.xml
>      ├── content-seed-initial.xml
>      ├── humanres-seed-initial.xml
>      ├── manufacturing-seed-initial.xml
>      ├── marketing-seed-initial.xml
>      ├── order-seed-initial.xml
>      ├── party-seed-initial.xml
>      ├── product-seed-initial.xml
>      ├── shipment-seed-initial.xml
>      └── workeffort-seed-initial.xml

Re: [Proposal] moving all data in applications to the data-model component

Posted by Taher Alkhateeb <sl...@gmail.com>.
That's super Swapnil, thank you :)

On Thu, Jul 20, 2017 at 10:47 AM, Swapnil Mane
<sw...@hotwaxsystems.com> wrote:
> Hello Taher,
>
> Thanks for the initiative, please count me in. I will help in providing
> patches for the same.
>
>
> - Best Regards,
> Swapnil M Mane
> www.hotwaxsystems.com
> www.hotwax.co
>
> On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com> wrote:
>
>> Thank you all for your feedback and votes. I just counted 12 positive
>> votes to go ahead.
>>
>> I have created a JIRA [1] for this task and I will work on it, but it
>> would be fantastic if someone is willing to take the lead on this so
>> that I can focus on refactoring the rest of the core framework. Also
>> any volunteers to commit or provide patches are more than welcomed.
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>>
>> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <ni...@nereide.fr>
>> wrote:
>> > +1
>> >
>> >
>> >
>> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
>> >>
>> >>   Hello Everyone,
>> >>
>> >> Data loading in OFBiz is quite fragile. A small change could crash the
>> >> system and finding out the reason can be difficult. I really felt the
>> >> pain of this when trying to disentangle the framework from data
>> >> dependencies on plugins [1].
>> >>
>> >> I've been thinking about a solution to reduce fragility, and here are
>> >> my thoughts:
>> >> - The less data files the better.
>> >> - The less data the better.
>> >> - The less distance (e.g. same directory) between data files the better.
>> >> - No circular dependencies (file dependencies should be like a tree).
>> >> - Fail fast [2].
>> >>
>> >> Therefore, I suggest the following action points:
>> >> - Create a directory structure similar to [3] in the datamodel
>> component.
>> >> - For any data that two or more files depend on, immediately move it
>> >> to the common-<reader>.xml file.
>> >> - For any data that does not belong to any domain (e.g. marketing,
>> >> content, etc ..) move it to the common-<reader>.xml.
>> >> - Maintain a strict dependency hierarchy of demo -> seed ->
>> seed-initial.
>> >> - Gradually and carefully move data from all applications to the
>> >> datamodel component as per above guidelines.
>> >> - Cleanup and refactor the data as we move it including any tests that
>> >> depend on it.
>> >>
>> >> What do you think of this proposal? Does it sound reasonable? Also
>> >> given the size of what I'm proposing I think we need multiple people
>> >> to work on this. So is anyone interested in taking part in this
>> >> initiative?
>> >>
>> >> Cheers,
>> >>
>> >> Taher Alkhateeb
>> >>
>> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> >> [2]
>> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
>> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> >> [3] directory structure:
>> >> data/
>> >> ├── demo
>> >> │   ├── accounting-demo.xml
>> >> │   ├── common-demo.xml
>> >> │   ├── content-demo.xml
>> >> │   ├── humanres-demo.xml
>> >> │   ├── manufacturing-demo.xml
>> >> │   ├── marketing-demo.xml
>> >> │   ├── order-demo.xml
>> >> │   ├── party-demo.xml
>> >> │   ├── product-demo.xml
>> >> │   ├── shipment-demo.xml
>> >> │   └── workeffort-demo.xml
>> >> ├── seed
>> >> │   ├── accounting-seed.xml
>> >> │   ├── common-seed.xml
>> >> │   ├── content-seed.xml
>> >> │   ├── humanres-seed.xml
>> >> │   ├── manufacturing-seed.xml
>> >> │   ├── marketing-seed.xml
>> >> │   ├── order-seed.xml
>> >> │   ├── party-seed.xml
>> >> │   ├── product-seed.xml
>> >> │   ├── shipment-seed.xml
>> >> │   └── workeffort-seed.xml
>> >> └── seed-initial
>> >>      ├── accounting-seed-initial.xml
>> >>      ├── common-seed-initial.xml
>> >>      ├── content-seed-initial.xml
>> >>      ├── humanres-seed-initial.xml
>> >>      ├── manufacturing-seed-initial.xml
>> >>      ├── marketing-seed-initial.xml
>> >>      ├── order-seed-initial.xml
>> >>      ├── party-seed-initial.xml
>> >>      ├── product-seed-initial.xml
>> >>      ├── shipment-seed-initial.xml
>> >>      └── workeffort-seed-initial.xml
>> >>
>> >
>>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Swapnil Mane <sw...@hotwaxsystems.com>.
Hello Taher,

Thanks for the initiative, please count me in. I will help in providing
patches for the same.


- Best Regards,
Swapnil M Mane
www.hotwaxsystems.com
www.hotwax.co

On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
slidingfilaments@gmail.com> wrote:

> Thank you all for your feedback and votes. I just counted 12 positive
> votes to go ahead.
>
> I have created a JIRA [1] for this task and I will work on it, but it
> would be fantastic if someone is willing to take the lead on this so
> that I can focus on refactoring the rest of the core framework. Also
> any volunteers to commit or provide patches are more than welcomed.
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>
> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <ni...@nereide.fr>
> wrote:
> > +1
> >
> >
> >
> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
> >>
> >>   Hello Everyone,
> >>
> >> Data loading in OFBiz is quite fragile. A small change could crash the
> >> system and finding out the reason can be difficult. I really felt the
> >> pain of this when trying to disentangle the framework from data
> >> dependencies on plugins [1].
> >>
> >> I've been thinking about a solution to reduce fragility, and here are
> >> my thoughts:
> >> - The less data files the better.
> >> - The less data the better.
> >> - The less distance (e.g. same directory) between data files the better.
> >> - No circular dependencies (file dependencies should be like a tree).
> >> - Fail fast [2].
> >>
> >> Therefore, I suggest the following action points:
> >> - Create a directory structure similar to [3] in the datamodel
> component.
> >> - For any data that two or more files depend on, immediately move it
> >> to the common-<reader>.xml file.
> >> - For any data that does not belong to any domain (e.g. marketing,
> >> content, etc ..) move it to the common-<reader>.xml.
> >> - Maintain a strict dependency hierarchy of demo -> seed ->
> seed-initial.
> >> - Gradually and carefully move data from all applications to the
> >> datamodel component as per above guidelines.
> >> - Cleanup and refactor the data as we move it including any tests that
> >> depend on it.
> >>
> >> What do you think of this proposal? Does it sound reasonable? Also
> >> given the size of what I'm proposing I think we need multiple people
> >> to work on this. So is anyone interested in taking part in this
> >> initiative?
> >>
> >> Cheers,
> >>
> >> Taher Alkhateeb
> >>
> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> >> [2]
> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> >> [3] directory structure:
> >> data/
> >> ├── demo
> >> │   ├── accounting-demo.xml
> >> │   ├── common-demo.xml
> >> │   ├── content-demo.xml
> >> │   ├── humanres-demo.xml
> >> │   ├── manufacturing-demo.xml
> >> │   ├── marketing-demo.xml
> >> │   ├── order-demo.xml
> >> │   ├── party-demo.xml
> >> │   ├── product-demo.xml
> >> │   ├── shipment-demo.xml
> >> │   └── workeffort-demo.xml
> >> ├── seed
> >> │   ├── accounting-seed.xml
> >> │   ├── common-seed.xml
> >> │   ├── content-seed.xml
> >> │   ├── humanres-seed.xml
> >> │   ├── manufacturing-seed.xml
> >> │   ├── marketing-seed.xml
> >> │   ├── order-seed.xml
> >> │   ├── party-seed.xml
> >> │   ├── product-seed.xml
> >> │   ├── shipment-seed.xml
> >> │   └── workeffort-seed.xml
> >> └── seed-initial
> >>      ├── accounting-seed-initial.xml
> >>      ├── common-seed-initial.xml
> >>      ├── content-seed-initial.xml
> >>      ├── humanres-seed-initial.xml
> >>      ├── manufacturing-seed-initial.xml
> >>      ├── marketing-seed-initial.xml
> >>      ├── order-seed-initial.xml
> >>      ├── party-seed-initial.xml
> >>      ├── product-seed-initial.xml
> >>      ├── shipment-seed-initial.xml
> >>      └── workeffort-seed-initial.xml
> >>
> >
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Taher Alkhateeb <sl...@gmail.com>.
Wonderful :) looking forward to getting this thing done

On Thu, Jul 20, 2017 at 11:31 AM, Devanshu Vyas
<vy...@gmail.com> wrote:
> Hello Taher,
>
> I would like to work on to provide patches for this effort.
>
> Thanks & Regards,
> Devanshu Vyas.
>
> On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com> wrote:
>
>> Thank you all for your feedback and votes. I just counted 12 positive
>> votes to go ahead.
>>
>> I have created a JIRA [1] for this task and I will work on it, but it
>> would be fantastic if someone is willing to take the lead on this so
>> that I can focus on refactoring the rest of the core framework. Also
>> any volunteers to commit or provide patches are more than welcomed.
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>>
>> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <ni...@nereide.fr>
>> wrote:
>> > +1
>> >
>> >
>> >
>> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
>> >>
>> >>   Hello Everyone,
>> >>
>> >> Data loading in OFBiz is quite fragile. A small change could crash the
>> >> system and finding out the reason can be difficult. I really felt the
>> >> pain of this when trying to disentangle the framework from data
>> >> dependencies on plugins [1].
>> >>
>> >> I've been thinking about a solution to reduce fragility, and here are
>> >> my thoughts:
>> >> - The less data files the better.
>> >> - The less data the better.
>> >> - The less distance (e.g. same directory) between data files the better.
>> >> - No circular dependencies (file dependencies should be like a tree).
>> >> - Fail fast [2].
>> >>
>> >> Therefore, I suggest the following action points:
>> >> - Create a directory structure similar to [3] in the datamodel
>> component.
>> >> - For any data that two or more files depend on, immediately move it
>> >> to the common-<reader>.xml file.
>> >> - For any data that does not belong to any domain (e.g. marketing,
>> >> content, etc ..) move it to the common-<reader>.xml.
>> >> - Maintain a strict dependency hierarchy of demo -> seed ->
>> seed-initial.
>> >> - Gradually and carefully move data from all applications to the
>> >> datamodel component as per above guidelines.
>> >> - Cleanup and refactor the data as we move it including any tests that
>> >> depend on it.
>> >>
>> >> What do you think of this proposal? Does it sound reasonable? Also
>> >> given the size of what I'm proposing I think we need multiple people
>> >> to work on this. So is anyone interested in taking part in this
>> >> initiative?
>> >>
>> >> Cheers,
>> >>
>> >> Taher Alkhateeb
>> >>
>> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> >> [2]
>> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
>> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> >> [3] directory structure:
>> >> data/
>> >> ├── demo
>> >> │   ├── accounting-demo.xml
>> >> │   ├── common-demo.xml
>> >> │   ├── content-demo.xml
>> >> │   ├── humanres-demo.xml
>> >> │   ├── manufacturing-demo.xml
>> >> │   ├── marketing-demo.xml
>> >> │   ├── order-demo.xml
>> >> │   ├── party-demo.xml
>> >> │   ├── product-demo.xml
>> >> │   ├── shipment-demo.xml
>> >> │   └── workeffort-demo.xml
>> >> ├── seed
>> >> │   ├── accounting-seed.xml
>> >> │   ├── common-seed.xml
>> >> │   ├── content-seed.xml
>> >> │   ├── humanres-seed.xml
>> >> │   ├── manufacturing-seed.xml
>> >> │   ├── marketing-seed.xml
>> >> │   ├── order-seed.xml
>> >> │   ├── party-seed.xml
>> >> │   ├── product-seed.xml
>> >> │   ├── shipment-seed.xml
>> >> │   └── workeffort-seed.xml
>> >> └── seed-initial
>> >>      ├── accounting-seed-initial.xml
>> >>      ├── common-seed-initial.xml
>> >>      ├── content-seed-initial.xml
>> >>      ├── humanres-seed-initial.xml
>> >>      ├── manufacturing-seed-initial.xml
>> >>      ├── marketing-seed-initial.xml
>> >>      ├── order-seed-initial.xml
>> >>      ├── party-seed-initial.xml
>> >>      ├── product-seed-initial.xml
>> >>      ├── shipment-seed-initial.xml
>> >>      └── workeffort-seed-initial.xml
>> >>
>> >
>>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Devanshu Vyas <vy...@gmail.com>.
Hello Taher,

I would like to work on to provide patches for this effort.

Thanks & Regards,
Devanshu Vyas.

On Thu, Jul 20, 2017 at 12:15 PM, Taher Alkhateeb <
slidingfilaments@gmail.com> wrote:

> Thank you all for your feedback and votes. I just counted 12 positive
> votes to go ahead.
>
> I have created a JIRA [1] for this task and I will work on it, but it
> would be fantastic if someone is willing to take the lead on this so
> that I can focus on refactoring the rest of the core framework. Also
> any volunteers to commit or provide patches are more than welcomed.
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9501
>
> On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <ni...@nereide.fr>
> wrote:
> > +1
> >
> >
> >
> > Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
> >>
> >>   Hello Everyone,
> >>
> >> Data loading in OFBiz is quite fragile. A small change could crash the
> >> system and finding out the reason can be difficult. I really felt the
> >> pain of this when trying to disentangle the framework from data
> >> dependencies on plugins [1].
> >>
> >> I've been thinking about a solution to reduce fragility, and here are
> >> my thoughts:
> >> - The less data files the better.
> >> - The less data the better.
> >> - The less distance (e.g. same directory) between data files the better.
> >> - No circular dependencies (file dependencies should be like a tree).
> >> - Fail fast [2].
> >>
> >> Therefore, I suggest the following action points:
> >> - Create a directory structure similar to [3] in the datamodel
> component.
> >> - For any data that two or more files depend on, immediately move it
> >> to the common-<reader>.xml file.
> >> - For any data that does not belong to any domain (e.g. marketing,
> >> content, etc ..) move it to the common-<reader>.xml.
> >> - Maintain a strict dependency hierarchy of demo -> seed ->
> seed-initial.
> >> - Gradually and carefully move data from all applications to the
> >> datamodel component as per above guidelines.
> >> - Cleanup and refactor the data as we move it including any tests that
> >> depend on it.
> >>
> >> What do you think of this proposal? Does it sound reasonable? Also
> >> given the size of what I'm proposing I think we need multiple people
> >> to work on this. So is anyone interested in taking part in this
> >> initiative?
> >>
> >> Cheers,
> >>
> >> Taher Alkhateeb
> >>
> >> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> >> [2]
> >> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> >> [3] directory structure:
> >> data/
> >> ├── demo
> >> │   ├── accounting-demo.xml
> >> │   ├── common-demo.xml
> >> │   ├── content-demo.xml
> >> │   ├── humanres-demo.xml
> >> │   ├── manufacturing-demo.xml
> >> │   ├── marketing-demo.xml
> >> │   ├── order-demo.xml
> >> │   ├── party-demo.xml
> >> │   ├── product-demo.xml
> >> │   ├── shipment-demo.xml
> >> │   └── workeffort-demo.xml
> >> ├── seed
> >> │   ├── accounting-seed.xml
> >> │   ├── common-seed.xml
> >> │   ├── content-seed.xml
> >> │   ├── humanres-seed.xml
> >> │   ├── manufacturing-seed.xml
> >> │   ├── marketing-seed.xml
> >> │   ├── order-seed.xml
> >> │   ├── party-seed.xml
> >> │   ├── product-seed.xml
> >> │   ├── shipment-seed.xml
> >> │   └── workeffort-seed.xml
> >> └── seed-initial
> >>      ├── accounting-seed-initial.xml
> >>      ├── common-seed-initial.xml
> >>      ├── content-seed-initial.xml
> >>      ├── humanres-seed-initial.xml
> >>      ├── manufacturing-seed-initial.xml
> >>      ├── marketing-seed-initial.xml
> >>      ├── order-seed-initial.xml
> >>      ├── party-seed-initial.xml
> >>      ├── product-seed-initial.xml
> >>      ├── shipment-seed-initial.xml
> >>      └── workeffort-seed-initial.xml
> >>
> >
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Taher Alkhateeb <sl...@gmail.com>.
Thank you all for your feedback and votes. I just counted 12 positive
votes to go ahead.

I have created a JIRA [1] for this task and I will work on it, but it
would be fantastic if someone is willing to take the lead on this so
that I can focus on refactoring the rest of the core framework. Also
any volunteers to commit or provide patches are more than welcomed.

[1] https://issues.apache.org/jira/browse/OFBIZ-9501

On Tue, Jul 18, 2017 at 6:57 PM, Nicolas Malin <ni...@nereide.fr> wrote:
> +1
>
>
>
> Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
>>
>>   Hello Everyone,
>>
>> Data loading in OFBiz is quite fragile. A small change could crash the
>> system and finding out the reason can be difficult. I really felt the
>> pain of this when trying to disentangle the framework from data
>> dependencies on plugins [1].
>>
>> I've been thinking about a solution to reduce fragility, and here are
>> my thoughts:
>> - The less data files the better.
>> - The less data the better.
>> - The less distance (e.g. same directory) between data files the better.
>> - No circular dependencies (file dependencies should be like a tree).
>> - Fail fast [2].
>>
>> Therefore, I suggest the following action points:
>> - Create a directory structure similar to [3] in the datamodel component.
>> - For any data that two or more files depend on, immediately move it
>> to the common-<reader>.xml file.
>> - For any data that does not belong to any domain (e.g. marketing,
>> content, etc ..) move it to the common-<reader>.xml.
>> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
>> - Gradually and carefully move data from all applications to the
>> datamodel component as per above guidelines.
>> - Cleanup and refactor the data as we move it including any tests that
>> depend on it.
>>
>> What do you think of this proposal? Does it sound reasonable? Also
>> given the size of what I'm proposing I think we need multiple people
>> to work on this. So is anyone interested in taking part in this
>> initiative?
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> [2]
>> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> [3] directory structure:
>> data/
>> ├── demo
>> │   ├── accounting-demo.xml
>> │   ├── common-demo.xml
>> │   ├── content-demo.xml
>> │   ├── humanres-demo.xml
>> │   ├── manufacturing-demo.xml
>> │   ├── marketing-demo.xml
>> │   ├── order-demo.xml
>> │   ├── party-demo.xml
>> │   ├── product-demo.xml
>> │   ├── shipment-demo.xml
>> │   └── workeffort-demo.xml
>> ├── seed
>> │   ├── accounting-seed.xml
>> │   ├── common-seed.xml
>> │   ├── content-seed.xml
>> │   ├── humanres-seed.xml
>> │   ├── manufacturing-seed.xml
>> │   ├── marketing-seed.xml
>> │   ├── order-seed.xml
>> │   ├── party-seed.xml
>> │   ├── product-seed.xml
>> │   ├── shipment-seed.xml
>> │   └── workeffort-seed.xml
>> └── seed-initial
>>      ├── accounting-seed-initial.xml
>>      ├── common-seed-initial.xml
>>      ├── content-seed-initial.xml
>>      ├── humanres-seed-initial.xml
>>      ├── manufacturing-seed-initial.xml
>>      ├── marketing-seed-initial.xml
>>      ├── order-seed-initial.xml
>>      ├── party-seed-initial.xml
>>      ├── product-seed-initial.xml
>>      ├── shipment-seed-initial.xml
>>      └── workeffort-seed-initial.xml
>>
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Nicolas Malin <ni...@nereide.fr>.
+1


Le 14/07/2017 à 11:59, Taher Alkhateeb a écrit :
>   Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>      ├── accounting-seed-initial.xml
>      ├── common-seed-initial.xml
>      ├── content-seed-initial.xml
>      ├── humanres-seed-initial.xml
>      ├── manufacturing-seed-initial.xml
>      ├── marketing-seed-initial.xml
>      ├── order-seed-initial.xml
>      ├── party-seed-initial.xml
>      ├── product-seed-initial.xml
>      ├── shipment-seed-initial.xml
>      └── workeffort-seed-initial.xml
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Jacopo Cappellato <ja...@hotwaxsystems.com>.
+1 to everything you wrote. I will do my best to help with the initiative.

Thank you Taher.

Jacopo

On Fri, Jul 14, 2017 at 11:59 AM, Taher Alkhateeb <
slidingfilaments@gmail.com> wrote:

>  Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f
> 8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Aditya Sharma <ad...@hotwaxsystems.com>.
+1

Thanks and Regards,
*Aditya Sharma* | Enterprise Software Engineer
HotWax Systems <http://www.hotwaxsystems.com/>

<https://www.linkedin.com/in/aditya-sharma-78291810a/>

On Tue, Jul 18, 2017 at 3:08 PM, Sharan Foga <sh...@gmail.com> wrote:

> +1
>
> Thanks
> Sharan
>
>
> On 14/07/17 11:59, Taher Alkhateeb wrote:
>
>>   Hello Everyone,
>>
>> Data loading in OFBiz is quite fragile. A small change could crash the
>> system and finding out the reason can be difficult. I really felt the
>> pain of this when trying to disentangle the framework from data
>> dependencies on plugins [1].
>>
>> I've been thinking about a solution to reduce fragility, and here are
>> my thoughts:
>> - The less data files the better.
>> - The less data the better.
>> - The less distance (e.g. same directory) between data files the better.
>> - No circular dependencies (file dependencies should be like a tree).
>> - Fail fast [2].
>>
>> Therefore, I suggest the following action points:
>> - Create a directory structure similar to [3] in the datamodel component.
>> - For any data that two or more files depend on, immediately move it
>> to the common-<reader>.xml file.
>> - For any data that does not belong to any domain (e.g. marketing,
>> content, etc ..) move it to the common-<reader>.xml.
>> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
>> - Gradually and carefully move data from all applications to the
>> datamodel component as per above guidelines.
>> - Cleanup and refactor the data as we move it including any tests that
>> depend on it.
>>
>> What do you think of this proposal? Does it sound reasonable? Also
>> given the size of what I'm proposing I think we need multiple people
>> to work on this. So is anyone interested in taking part in this
>> initiative?
>>
>> Cheers,
>>
>> Taher Alkhateeb
>>
>> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
>> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb95
>> 0999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
>> [3] directory structure:
>> data/
>> ├── demo
>> │   ├── accounting-demo.xml
>> │   ├── common-demo.xml
>> │   ├── content-demo.xml
>> │   ├── humanres-demo.xml
>> │   ├── manufacturing-demo.xml
>> │   ├── marketing-demo.xml
>> │   ├── order-demo.xml
>> │   ├── party-demo.xml
>> │   ├── product-demo.xml
>> │   ├── shipment-demo.xml
>> │   └── workeffort-demo.xml
>> ├── seed
>> │   ├── accounting-seed.xml
>> │   ├── common-seed.xml
>> │   ├── content-seed.xml
>> │   ├── humanres-seed.xml
>> │   ├── manufacturing-seed.xml
>> │   ├── marketing-seed.xml
>> │   ├── order-seed.xml
>> │   ├── party-seed.xml
>> │   ├── product-seed.xml
>> │   ├── shipment-seed.xml
>> │   └── workeffort-seed.xml
>> └── seed-initial
>>      ├── accounting-seed-initial.xml
>>      ├── common-seed-initial.xml
>>      ├── content-seed-initial.xml
>>      ├── humanres-seed-initial.xml
>>      ├── manufacturing-seed-initial.xml
>>      ├── marketing-seed-initial.xml
>>      ├── order-seed-initial.xml
>>      ├── party-seed-initial.xml
>>      ├── product-seed-initial.xml
>>      ├── shipment-seed-initial.xml
>>      └── workeffort-seed-initial.xml
>>
>
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by Sharan Foga <sh...@gmail.com>.
+1

Thanks
Sharan

On 14/07/17 11:59, Taher Alkhateeb wrote:
>   Hello Everyone,
>
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
>
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
>
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
>
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
>
> Cheers,
>
> Taher Alkhateeb
>
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>      ├── accounting-seed-initial.xml
>      ├── common-seed-initial.xml
>      ├── content-seed-initial.xml
>      ├── humanres-seed-initial.xml
>      ├── manufacturing-seed-initial.xml
>      ├── marketing-seed-initial.xml
>      ├── order-seed-initial.xml
>      ├── party-seed-initial.xml
>      ├── product-seed-initial.xml
>      ├── shipment-seed-initial.xml
>      └── workeffort-seed-initial.xml

Re: [Proposal] moving all data in applications to the data-model component

Posted by Deepak Dixit <de...@hotwaxsystems.com>.
+1 ,

Thanks & Regards
--
Deepak Dixit
www.hotwaxsystems.com
www.hotwax.co

On Tue, Jul 18, 2017 at 2:29 PM, Sakke Ferraris <sa...@ferrasys.com> wrote:

> Hi,
>
> I just joined, and I'd like to help on this.
>
> BR, Sakke Ferraris
>
> -----Ursprungligt meddelande-----
> Från: James Yong [mailto:jamesyong@apache.org]
> Skickat: den 18 juli 2017 10:45
> Till: dev@ofbiz.apache.org
> Ämne: Re: [Proposal] moving all data in applications to the data-model
> component
>
> +1
>
> Regards,
> James Yong
>
> On 2017-07-14 17:59 (+0800), Taher Alkhateeb <sl...@gmail.com>
> wrote:
> >  Hello Everyone,
> >
> > Data loading in OFBiz is quite fragile. A small change could crash the
> > system and finding out the reason can be difficult. I really felt the
> > pain of this when trying to disentangle the framework from data
> > dependencies on plugins [1].
> >
> > I've been thinking about a solution to reduce fragility, and here are
> > my thoughts:
> > - The less data files the better.
> > - The less data the better.
> > - The less distance (e.g. same directory) between data files the better.
> > - No circular dependencies (file dependencies should be like a tree).
> > - Fail fast [2].
> >
> > Therefore, I suggest the following action points:
> > - Create a directory structure similar to [3] in the datamodel component.
> > - For any data that two or more files depend on, immediately move it
> > to the common-<reader>.xml file.
> > - For any data that does not belong to any domain (e.g. marketing,
> > content, etc ..) move it to the common-<reader>.xml.
> > - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> > - Gradually and carefully move data from all applications to the
> > datamodel component as per above guidelines.
> > - Cleanup and refactor the data as we move it including any tests that
> > depend on it.
> >
> > What do you think of this proposal? Does it sound reasonable? Also
> > given the size of what I'm proposing I think we need multiple people
> > to work on this. So is anyone interested in taking part in this
> > initiative?
> >
> > Cheers,
> >
> > Taher Alkhateeb
> >
> > [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> > [2]
> > https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e8
> > 69a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> > [3] directory structure:
> > data/
> > ├── demo
> > │   ├── accounting-demo.xml
> > │   ├── common-demo.xml
> > │   ├── content-demo.xml
> > │   ├── humanres-demo.xml
> > │   ├── manufacturing-demo.xml
> > │   ├── marketing-demo.xml
> > │   ├── order-demo.xml
> > │   ├── party-demo.xml
> > │   ├── product-demo.xml
> > │   ├── shipment-demo.xml
> > │   └── workeffort-demo.xml
> > ├── seed
> > │   ├── accounting-seed.xml
> > │   ├── common-seed.xml
> > │   ├── content-seed.xml
> > │   ├── humanres-seed.xml
> > │   ├── manufacturing-seed.xml
> > │   ├── marketing-seed.xml
> > │   ├── order-seed.xml
> > │   ├── party-seed.xml
> > │   ├── product-seed.xml
> > │   ├── shipment-seed.xml
> > │   └── workeffort-seed.xml
> > └── seed-initial
> >     ├── accounting-seed-initial.xml
> >     ├── common-seed-initial.xml
> >     ├── content-seed-initial.xml
> >     ├── humanres-seed-initial.xml
> >     ├── manufacturing-seed-initial.xml
> >     ├── marketing-seed-initial.xml
> >     ├── order-seed-initial.xml
> >     ├── party-seed-initial.xml
> >     ├── product-seed-initial.xml
> >     ├── shipment-seed-initial.xml
> >     └── workeffort-seed-initial.xml
> >
>

SV: [Proposal] moving all data in applications to the data-model component

Posted by Sakke Ferraris <sa...@ferrasys.com>.
Hi,

I just joined, and I'd like to help on this.

BR, Sakke Ferraris

-----Ursprungligt meddelande-----
Från: James Yong [mailto:jamesyong@apache.org] 
Skickat: den 18 juli 2017 10:45
Till: dev@ofbiz.apache.org
Ämne: Re: [Proposal] moving all data in applications to the data-model component

+1

Regards,
James Yong

On 2017-07-14 17:59 (+0800), Taher Alkhateeb <sl...@gmail.com> wrote: 
>  Hello Everyone,
> 
> Data loading in OFBiz is quite fragile. A small change could crash the 
> system and finding out the reason can be difficult. I really felt the 
> pain of this when trying to disentangle the framework from data 
> dependencies on plugins [1].
> 
> I've been thinking about a solution to reduce fragility, and here are 
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
> 
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it 
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing, 
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the 
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that 
> depend on it.
> 
> What do you think of this proposal? Does it sound reasonable? Also 
> given the size of what I'm proposing I think we need multiple people 
> to work on this. So is anyone interested in taking part in this 
> initiative?
> 
> Cheers,
> 
> Taher Alkhateeb
> 
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] 
> https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e8
> 69a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>

Re: [Proposal] moving all data in applications to the data-model component

Posted by James Yong <ja...@apache.org>.
+1

Regards,
James Yong

On 2017-07-14 17:59 (+0800), Taher Alkhateeb <sl...@gmail.com> wrote: 
>  Hello Everyone,
> 
> Data loading in OFBiz is quite fragile. A small change could crash the
> system and finding out the reason can be difficult. I really felt the
> pain of this when trying to disentangle the framework from data
> dependencies on plugins [1].
> 
> I've been thinking about a solution to reduce fragility, and here are
> my thoughts:
> - The less data files the better.
> - The less data the better.
> - The less distance (e.g. same directory) between data files the better.
> - No circular dependencies (file dependencies should be like a tree).
> - Fail fast [2].
> 
> Therefore, I suggest the following action points:
> - Create a directory structure similar to [3] in the datamodel component.
> - For any data that two or more files depend on, immediately move it
> to the common-<reader>.xml file.
> - For any data that does not belong to any domain (e.g. marketing,
> content, etc ..) move it to the common-<reader>.xml.
> - Maintain a strict dependency hierarchy of demo -> seed -> seed-initial.
> - Gradually and carefully move data from all applications to the
> datamodel component as per above guidelines.
> - Cleanup and refactor the data as we move it including any tests that
> depend on it.
> 
> What do you think of this proposal? Does it sound reasonable? Also
> given the size of what I'm proposing I think we need multiple people
> to work on this. So is anyone interested in taking part in this
> initiative?
> 
> Cheers,
> 
> Taher Alkhateeb
> 
> [1] https://issues.apache.org/jira/browse/OFBIZ-9322
> [2] https://lists.apache.org/thread.html/a18b5d4086a07ff34a0cb950999e8f8e869a6aa7efa2fc866a75a7b6@%3Cdev.ofbiz.apache.org%3E
> [3] directory structure:
> data/
> ├── demo
> │   ├── accounting-demo.xml
> │   ├── common-demo.xml
> │   ├── content-demo.xml
> │   ├── humanres-demo.xml
> │   ├── manufacturing-demo.xml
> │   ├── marketing-demo.xml
> │   ├── order-demo.xml
> │   ├── party-demo.xml
> │   ├── product-demo.xml
> │   ├── shipment-demo.xml
> │   └── workeffort-demo.xml
> ├── seed
> │   ├── accounting-seed.xml
> │   ├── common-seed.xml
> │   ├── content-seed.xml
> │   ├── humanres-seed.xml
> │   ├── manufacturing-seed.xml
> │   ├── marketing-seed.xml
> │   ├── order-seed.xml
> │   ├── party-seed.xml
> │   ├── product-seed.xml
> │   ├── shipment-seed.xml
> │   └── workeffort-seed.xml
> └── seed-initial
>     ├── accounting-seed-initial.xml
>     ├── common-seed-initial.xml
>     ├── content-seed-initial.xml
>     ├── humanres-seed-initial.xml
>     ├── manufacturing-seed-initial.xml
>     ├── marketing-seed-initial.xml
>     ├── order-seed-initial.xml
>     ├── party-seed-initial.xml
>     ├── product-seed-initial.xml
>     ├── shipment-seed-initial.xml
>     └── workeffort-seed-initial.xml
>