You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Gareth Carter <Ga...@stannah.co.uk> on 2017/10/24 15:36:17 UTC

Updates to EntityQuery

Hi all

We have internally patched EntityQuery with some additional functionality and before I create a Jira was going to see what the community thinks.

New functionality:

-       New method getFieldMap which returns a Map object of selected fields from GenericValue objects, useful for creating cache map objects for lookup

-       Support forUpdate

-       forEach method to accept Consumer

We have found these useful and believe the project can benefit, let me know what you think

Regards

Gareth Carter


Software Development Analyst


Stannah Management Services Ltd


IT Department


Ext:


7036


DDI:


01264 364311




[http://logos.stannah.co.uk/stan150.jpg]


[http://logos.stannah.co.uk/enviro.jpg]Please consider the environment before printing this email.



This email is intended only for the above addressee. It may contain privileged information. If you are not the addressee you must not copy, distribute, disclose or use any of the information in it. If you have received it in error, please delete it and notify the sender.

Stannah Lift Holdings Ltd registered No. 686996, Stannah Management Services Ltd registered No. 2483693, Stannah Lift Services Ltd registered No. 1189799, Stannah Microlifts Ltd registered No. 964804, Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd registered No. 1401451, Global Upholstery Solutions Ltd registered No. 02452728.

All registered offices at Watt Close, East Portway, Andover, Hampshire, SP10 3SD, England.

All Registered in England and Wales.

This message has been scanned for malware by Websense. www.websense.com

Re: Updates to EntityQuery

Posted by Pritam Kute <pr...@hotwaxsystems.com>.
Hello Gareth,

Looks like a nice addition to EntityQuery API. Thanks for that. As
mentioned by Arun, please provide more details to understand about
"forUpdate" and "forEach" methods.

Regards
--
Pritam Kute

On Wed, Oct 25, 2017 at 10:18 AM, Arun Patidar <
arun.patidar@hotwaxsystems.com> wrote:

> Hello Gareth,
>
> Please provide some more details or patch to understand  - forUpdate
> and forEach
> method utility.
>
> getFieldMap method looks good to me.
>
>
>
> --
> Thanks & Regards
> ---
> Arun Patidar
> Manager, Enterprise Software Development
>
> HotWax Systems Pvt Ltd.
>
> www.hotwaxsystems.com
>
>
> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
> Gareth.Carter@stannah.co.uk>
> wrote:
>
> > Hi all
> >
> > We have internally patched EntityQuery with some additional functionality
> > and before I create a Jira was going to see what the community thinks.
> >
> > New functionality:
> >
> > -       New method getFieldMap which returns a Map object of selected
> > fields from GenericValue objects, useful for creating cache map objects
> for
> > lookup
> >
> > -       Support forUpdate
> >
> > -       forEach method to accept Consumer
> >
> > We have found these useful and believe the project can benefit, let me
> > know what you think
> >
> > Regards
> >
> > Gareth Carter
> >
> >
> > Software Development Analyst
> >
> >
> > Stannah Management Services Ltd
> >
> >
> > IT Department
> >
> >
> > Ext:
> >
> >
> > 7036
> >
> >
> > DDI:
> >
> >
> > 01264 364311
> >
> >
> >
> >
> > [http://logos.stannah.co.uk/stan150.jpg]
> >
> >
> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
> > before printing this email.
> >
> >
> >
> > This email is intended only for the above addressee. It may contain
> > privileged information. If you are not the addressee you must not copy,
> > distribute, disclose or use any of the information in it. If you have
> > received it in error, please delete it and notify the sender.
> >
> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd registered
> > No. 1189799, Stannah Microlifts Ltd registered No. 964804, Stannah Lifts
> > Ltd registered No. 1189836, Stannah Stairlifts Ltd registered No.
> 1401451,
> > Global Upholstery Solutions Ltd registered No. 02452728.
> >
> > All registered offices at Watt Close, East Portway, Andover, Hampshire,
> > SP10 3SD, England.
> >
> > All Registered in England and Wales.
> >
> > This message has been scanned for malware by Websense. www.websense.com
> >
>

RE: Updates to EntityQuery

Posted by Gareth Carter <Ga...@stannah.co.uk>.
Hi Paul

Problem with that, it's all or nothing. We only need this for specific use cases. Having forUpdate in EntityQuery gives us that flexibility. It could be changed to use either method which depends on the scenario you use it for
Gareth Carter
Software Development Analyst
Stannah Management Services Ltd
IT Department
Ext:
7036
DDI:
01264 364311


Please consider the environment before printing this email.

-----Original Message-----
From: Paul Foxworthy [mailto:paul@cohsoft.com.au]
Sent: 27 October 2017 12:42 AM
To: dev@ofbiz.apache.org
Subject: Re: Updates to EntityQuery

Hi all,

As far as I know, FOR UPDATE would only break SQLite, and I can live with that.

Gareth, would optimistic instead of pessimistic work for your situation?
The database does less of the work and it's more scalable. I haven't tested it out, but OFBiz provides optimistic locking if you turn on enable-lock.

Cheers

Paul Foxworthy



On 27 October 2017 at 03:49, Jacques Le Roux <ja...@les7arts.com>
wrote:

> I agree with Scott and Taher,
>
> About your question Gareth, we once had a hard coded  "for update"
> somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ).
> I put it in. Then Jacopo crossed an issue in a cluster and rightly
> changed it to something better See
> https://issues.apache.org/jira/browse/OFBIZ-2353 for more where Adrian
> gave some directions also.
>
> Since I did not read all in the thread and there is no patch yet I
> don't know if something alike applies to EntityQuery API. Putting a
> for update is certainly much simpler, so just saying for now.
>
> Jacques
>
>
>
> Le 26/10/2017 à 10:46, Gareth Carter a écrit :
>
>> HI all
>>
>> Thank for the response
>>
>> It could be considered a hint, if the underlying db does not support
>> it then don't use it. Adding an attribute on the datasource could be
>> the way to go but I also believe we need methods to tell the system
>> when to use it, you don't need it all the time only in selective cases.
>>
>> Until support is added I see ofbiz as purely RDMS. I don't believe
>> ofbiz should restrict itself on the assumption it might support
>> different databases such as nosql.
>>
>> There could be an alternative to forUpdate, anyone considered
>> implementing a distributed locking mechanism? The whole point for
>> using forUpdate is to prevent one or more code blocks changing the
>> same database records concurrently, db row locks help but are not
>> supported by all databases but an external locking mechanism may
>> provide a generic way of dealing with this. OFBiz already provides a
>> semaphore on services but I believe you need something at entity
>> level Gareth Carter Software Development Analyst Stannah Management
>> Services Ltd IT Department
>> Ext:
>> 7036
>> DDI:
>> 01264 364311
>>
>>
>> Please consider the environment before printing this email.
>>
>> -----Original Message-----
>> From: Taher Alkhateeb [mailto:slidingfilaments@gmail.com]
>> Sent: 26 October 2017 6:13 AM
>> To: OFBIZ Development Mailing List <de...@ofbiz.apache.org>
>> Subject: Re: Updates to EntityQuery
>>
>> Hmmm usually my first gut reaction would be "keep things consistent",
>> however Scott made a good argument for supporting non-common DB features.
>> So my suggestion would be to use something like the widget's
>> <platform-specific ...> tag (or equivalent in code) for any features
>> not supported in all DBMSs. By labeling it as such the user is aware
>> that the DB selection is restricted.
>>
>> Now with regards to your comment about NoSQL, such databases are
>> fundamentally different, and act more like a key-value based entities
>> without a schema or the resulting ACID properties. It is possible to
>> make it work with the entity engine, but then you'd lose the
>> flexibility it gives you because you want to restrict yourself to the
>> relational model that we apply in OFBiz. The idea of NoSQL is to
>> think at a different conceptual level of your domain model.
>>
>> On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray
>> <sc...@hotwaxsystems.com>
>> wrote:
>>
>>> I think it's better to have for-update available for use for the
>>> majority of use-cases than it is to exclude it altogether.  I'd
>>> rather lose the databases that don't support it than constrain the
>>> ones that do and are used most by pretty much everyone.  It would be
>>> nice to hear from anyone using a database that doesn't support it
>>> since I'm just guessing most of us use postgres or mysql.
>>>
>>> Has anyone actually looked at how NoSQL databases might work with
>>> the OFBiz entity engine or are the concepts too far removed from SQL
>>> for it to even be workable?
>>>
>>> If we did want to support for-update globally, we could always have
>>> the delegator simulate it by issuing an UPDATE to lock the rows
>>> prior to performing the SELECT.  Although it might not work
>>> perfectly in all cases, it could be a good compromise. e.g.
>>> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions --
>>> lock the rows SELECT * FROM table WHERE ...conditions -- select the
>>> rows
>>>
>>> Regards
>>> Scott
>>>
>>>
>>>
>>> On 26 October 2017 at 13:45, Paul Foxworthy <pa...@cohsoft.com.au> wrote:
>>>
>>> Hi Gareth,
>>>>
>>>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't
>>>> support
>>>> it:
>>>>
>>>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html
>>>> https://sqlite.org/lang_select.html
>>>>
>>>> FOR UPDATE only makes sense for relational databases. Adding
>>>> assumptions it's available would make it harder to use a NoSQL
>>>> database in future.
>>>>
>>>> In the case of SQLite, in effect it's got serialized isolation
>>>> anyway, so the FOR UPDATE is not strictly necessary.
>>>>
>>>> I suggest OFBiz shouldn't have a method that generates
>>>> syntactically incorrect SQL for any database we want to support,
>>>> and we claim to support several. Where there are variations between
>>>> SQL dialects, we have attributes in the data source, such as "join-style".
>>>>
>>>> So think our choices are:
>>>>
>>>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by
>>>> a data source. OFBiz would only add the "FOR UPDATE" if it is
>>>> supported. In the case of SQLite, I suspect we'd get away without
>>>> it, but there may be other databases where the absence of "FOR UPDATE"
>>>> might bring on unexpected behaviour.
>>>>
>>>> 2. Don't provide a method to expose FOR UPDATE.
>>>>
>>>> Could you achieve the results you want with a higher transaction
>>>> isolation level like repeatable read, or with optimistic locking
>>>> (enable-lock)?
>>>>
>>>> What do you think?
>>>>
>>>> Cheers
>>>>
>>>> Paul Foxworthy
>>>>
>>>>
>>>> On 26 October 2017 at 07:50, Gareth Carter
>>>> <Ga...@stannah.co.uk>
>>>> wrote:
>>>>
>>>> Ok I will create a Jira with patch for review.
>>>>>
>>>>>
>>>>>
>>>>> Sent from my Samsung Galaxy smartphone.
>>>>>
>>>>>
>>>>>
>>>>> Gareth Carter
>>>>>
>>>>>
>>>>> Software Development Analyst
>>>>>
>>>>>
>>>>> Stannah Management Services Ltd
>>>>>
>>>>>
>>>>> IT Department
>>>>>
>>>>>
>>>>> Ext:
>>>>>
>>>>>
>>>>> 7036
>>>>>
>>>>>
>>>>> DDI:
>>>>>
>>>>>
>>>>> 01264 364311
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [http://logos.stannah.co.uk/stan150.jpg]
>>>>>
>>>>>
>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>>>>> environment before printing this email.
>>>>>
>>>>>
>>>>> -------- Original message --------
>>>>> From: deepak nigam <de...@gmail.com>
>>>>> Date: 25/10/2017 14:06 (GMT+00:00)
>>>>> To: dev@ofbiz.apache.org
>>>>> Subject: Re: Updates to EntityQuery
>>>>>
>>>>> Looking good to me also. Let me know if I can be of any help.
>>>>>
>>>>>
>>>>> Thanks & Regards
>>>>> --
>>>>> Deepak Nigam
>>>>>
>>>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <
>>>>> yash.sharma@hotwaxsystems.com>
>>>>> wrote:
>>>>>
>>>>> I am all up for it, please let me know if I could help.
>>>>>>
>>>>>> Thanks & Regards,
>>>>>> --
>>>>>> *Pradhan Yash Sharma*
>>>>>> *HotWax Systems* |
>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>>>>>>
>>>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
>>>>>> yash.sharma@hotwaxsystems.com>
>>>>>> wrote:
>>>>>>
>>>>>> Yes, Usage of the stream will surely enhance performance to a
>>>>>>> certain extent and removes ceremony from the code base. I think
>>>>>>> Parallel
>>>>>>>
>>>>>> streams
>>>>>
>>>>>> will add enhancements in many folds as we are using a multicore
>>>>>>>
>>>>>> processor
>>>>>
>>>>>> (I have not tested yet), but the Functional approach seems promising.
>>>>>>>
>>>>>>> Thanks & Regards,
>>>>>>> --
>>>>>>> *Pradhan Yash Sharma*
>>>>>>> *HotWax Systems* | www.hotwaxsystems.com<http://
>>>>>>>
>>>>>> www.hotwaxsystems.com>
>>>>
>>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
>>>>>>> Gareth.Carter@stannah.co.uk> wrote:
>>>>>>>
>>>>>>> forUpdate
>>>>>>>>
>>>>>>>> We patched EntityFindOptions with a new field "forUpdate" with
>>>>>>>>
>>>>>>> shorthand
>>>>>
>>>>>> methods in EntityQuery to enable. We then made a change to
>>>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE"
>>>>>>>> on the
>>>>>>>>
>>>>>>> end
>>>>>
>>>>>> of the SQL select statement - this allows for DB row locks (we
>>>>>>>> use
>>>>>>>>
>>>>>>> postgres
>>>>>>
>>>>>>> and works but have not tested other databases). I believe
>>>>>>>> there may
>>>>>>>>
>>>>>>> have
>>>>>
>>>>>> been a discussion about this before
>>>>>>>>
>>>>>>>>
>>>>>>>> forEach on EntityQuery
>>>>>>>>
>>>>>>>> Use Consumer in java and groovy to iterate over a query. This
>>>>>>>> can
>>>>>>>>
>>>>>>> reduce
>>>>>
>>>>>> memory consumption (replacement for queryList()) and boiler
>>>>>>>> plate
>>>>>>>>
>>>>>>> code
>>>>
>>>>> (eg
>>>>>>
>>>>>>> queryIterator(), while loop and close)
>>>>>>>>
>>>>>>>> Example:
>>>>>>>>
>>>>>>>> EntityQuery.use(delegator).from("Foobar")
>>>>>>>> .forEach(item ->
>>>>>>>>      Debug.logInfo(item.toString(), module); );
>>>>>>>>
>>>>>>>> A further update could be to provide stream capabilities
>>>>>>>>
>>>>>>>> Hope this helps, I can provide a patch aswell
>>>>>>>>
>>>>>>>> Gareth Carter
>>>>>>>> Software Development Analyst
>>>>>>>> Stannah Management Services Ltd IT Department
>>>>>>>> Ext:
>>>>>>>> 7036
>>>>>>>> DDI:
>>>>>>>> 01264 364311
>>>>>>>>
>>>>>>>>
>>>>>>>> Please consider the environment before printing this email.
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
>>>>>>>> Sent: 25 October 2017 5:49 AM
>>>>>>>> To: dev@ofbiz.apache.org
>>>>>>>> Cc: arunpatidar@apache.org
>>>>>>>> Subject: Re: Updates to EntityQuery
>>>>>>>>
>>>>>>>> Hello Gareth,
>>>>>>>>
>>>>>>>> Please provide some more details or patch to understand  -
>>>>>>>> forUpdate
>>>>>>>>
>>>>>>> and
>>>>>
>>>>>> forEach method utility.
>>>>>>>>
>>>>>>>> getFieldMap method looks good to me.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks & Regards
>>>>>>>> ---
>>>>>>>> Arun Patidar
>>>>>>>> Manager, Enterprise Software Development
>>>>>>>>
>>>>>>>> HotWax Systems Pvt Ltd.
>>>>>>>>
>>>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
>>>>>>>> Gareth.Carter@stannah.co.uk>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi all
>>>>>>>>>
>>>>>>>>> We have internally patched EntityQuery with some additional
>>>>>>>>> functionality and before I create a Jira was going to see what
>>>>>>>>> the
>>>>>>>>>
>>>>>>>> community thinks.
>>>>>>>>
>>>>>>>>> New functionality:
>>>>>>>>>
>>>>>>>>> -       New method getFieldMap which returns a Map object of
>>>>>>>>>
>>>>>>>> selected
>>>>>
>>>>>> fields from GenericValue objects, useful for creating cache map
>>>>>>>>> objects for lookup
>>>>>>>>>
>>>>>>>>> -       Support forUpdate
>>>>>>>>>
>>>>>>>>> -       forEach method to accept Consumer
>>>>>>>>>
>>>>>>>>> We have found these useful and believe the project can
>>>>>>>>> benefit,
>>>>>>>>>
>>>>>>>> let
>>>>
>>>>> me
>>>>>
>>>>>> know what you think
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>>
>>>>>>>>> Gareth Carter
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Software Development Analyst
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Stannah Management Services Ltd
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> IT Department
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ext:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 7036
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> DDI:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 01264 364311
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [http://logos.stannah.co.uk/stan150.jpg]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>>>>>>>>>
>>>>>>>> environment
>>>>>>
>>>>>>> before printing this email.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This email is intended only for the above addressee. It may
>>>>>>>>>
>>>>>>>> contain
>>>>
>>>>> privileged information. If you are not the addressee you must not
>>>>>>>>> copy, distribute, disclose or use any of the information in
>>>>>>>>> it. If
>>>>>>>>>
>>>>>>>> you
>>>>>
>>>>>> have received it in error, please delete it and notify the sender.
>>>>>>>>>
>>>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah
>>>>>>>>>
>>>>>>>> Management
>>>>
>>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd
>>>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No.
>>>>>>>>>
>>>>>>>> 964804,
>>>>>
>>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
>>>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd
>>>>>>>>> registered
>>>>>>>>>
>>>>>>>> No.
>>>>>
>>>>>> 02452728.
>>>>>>>>
>>>>>>>>> All registered offices at Watt Close, East Portway, Andover,
>>>>>>>>> Hampshire,
>>>>>>>>> SP10 3SD, England.
>>>>>>>>>
>>>>>>>>> All Registered in England and Wales.
>>>>>>>>>
>>>>>>>>> This message has been scanned for malware by Websense.
>>>>>>>>> www.websense.com<http://www.websense.com>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> To report this email as spam, please send the original message,
>>>>>>>>
>>>>>>> complete
>>>>>
>>>>>> with headers to spam@websense.com
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> Coherent Software Australia Pty Ltd PO Box 2773 Cheltenham Vic 3192
>>>> Australia
>>>>
>>>> Phone: +61 3 9585 6788
>>>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxN
>>>> CsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A
>>>> -OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSA
>>>> Ux33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z
>>>> Email: info@coherentsoftware.com.au
>>>>
>>>>
>


--
Coherent Software Australia Pty Ltd
PO Box 2773
Cheltenham Vic 3192
Australia

Phone: +61 3 9585 6788
Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicY2BnsD7JwLD1IANDUU6lkXGiXnFRmV5uYmZOcn5eSVF-jl5yfi5DmWWgX3pwiLmBgZmhhQmDe2JRakmGnnNiUUlqkUNxSWJeXmIGUKFeaTZDRklJgZW-fnl5OVAgI7UoNa-kOD-tpByoBWSUXmKpPgMDw5sdDAwAvaYpRA&Z
Email: info@coherentsoftware.com.au

Re: Updates to EntityQuery

Posted by Paul Foxworthy <pa...@cohsoft.com.au>.
Hi all,

As far as I know, FOR UPDATE would only break SQLite, and I can live with
that.

Gareth, would optimistic instead of pessimistic work for your situation?
The database does less of the work and it's more scalable. I haven't tested
it out, but OFBiz provides optimistic locking if you turn on enable-lock.

Cheers

Paul Foxworthy



On 27 October 2017 at 03:49, Jacques Le Roux <ja...@les7arts.com>
wrote:

> I agree with Scott and Taher,
>
> About your question Gareth, we once had a hard coded  "for update"
> somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ).
> I put it in. Then Jacopo crossed an issue in a cluster and rightly changed
> it to something better
> See https://issues.apache.org/jira/browse/OFBIZ-2353 for more where
> Adrian gave some directions also.
>
> Since I did not read all in the thread and there is no patch yet I don't
> know if something alike applies to EntityQuery API. Putting a for update is
> certainly much simpler, so just saying for now.
>
> Jacques
>
>
>
> Le 26/10/2017 à 10:46, Gareth Carter a écrit :
>
>> HI all
>>
>> Thank for the response
>>
>> It could be considered a hint, if the underlying db does not support it
>> then don't use it. Adding an attribute on the datasource could be the way
>> to go but I also believe we need methods to tell the system when to use it,
>> you don't need it all the time only in selective cases.
>>
>> Until support is added I see ofbiz as purely RDMS. I don't believe ofbiz
>> should restrict itself on the assumption it might support different
>> databases such as nosql.
>>
>> There could be an alternative to forUpdate, anyone considered
>> implementing a distributed locking mechanism? The whole point for using
>> forUpdate is to prevent one or more code blocks changing the same database
>> records concurrently, db row locks help but are not supported by all
>> databases but an external locking mechanism may provide a generic way of
>> dealing with this. OFBiz already provides a semaphore on services but I
>> believe you need something at entity level
>> Gareth Carter
>> Software Development Analyst
>> Stannah Management Services Ltd
>> IT Department
>> Ext:
>> 7036
>> DDI:
>> 01264 364311
>>
>>
>> Please consider the environment before printing this email.
>>
>> -----Original Message-----
>> From: Taher Alkhateeb [mailto:slidingfilaments@gmail.com]
>> Sent: 26 October 2017 6:13 AM
>> To: OFBIZ Development Mailing List <de...@ofbiz.apache.org>
>> Subject: Re: Updates to EntityQuery
>>
>> Hmmm usually my first gut reaction would be "keep things consistent",
>> however Scott made a good argument for supporting non-common DB features.
>> So my suggestion would be to use something like the widget's
>> <platform-specific ...> tag (or equivalent in code) for any features not
>> supported in all DBMSs. By labeling it as such the user is aware that the
>> DB selection is restricted.
>>
>> Now with regards to your comment about NoSQL, such databases are
>> fundamentally different, and act more like a key-value based entities
>> without a schema or the resulting ACID properties. It is possible to make
>> it work with the entity engine, but then you'd lose the flexibility it
>> gives you because you want to restrict yourself to the relational model
>> that we apply in OFBiz. The idea of NoSQL is to think at a different
>> conceptual level of your domain model.
>>
>> On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <sc...@hotwaxsystems.com>
>> wrote:
>>
>>> I think it's better to have for-update available for use for the
>>> majority of use-cases than it is to exclude it altogether.  I'd rather
>>> lose the databases that don't support it than constrain the ones that
>>> do and are used most by pretty much everyone.  It would be nice to
>>> hear from anyone using a database that doesn't support it since I'm
>>> just guessing most of us use postgres or mysql.
>>>
>>> Has anyone actually looked at how NoSQL databases might work with the
>>> OFBiz entity engine or are the concepts too far removed from SQL for
>>> it to even be workable?
>>>
>>> If we did want to support for-update globally, we could always have
>>> the delegator simulate it by issuing an UPDATE to lock the rows prior
>>> to performing the SELECT.  Although it might not work perfectly in all
>>> cases, it could be a good compromise. e.g.
>>> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions --
>>> lock the rows SELECT * FROM table WHERE ...conditions -- select the
>>> rows
>>>
>>> Regards
>>> Scott
>>>
>>>
>>>
>>> On 26 October 2017 at 13:45, Paul Foxworthy <pa...@cohsoft.com.au> wrote:
>>>
>>> Hi Gareth,
>>>>
>>>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support
>>>> it:
>>>>
>>>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html
>>>> https://sqlite.org/lang_select.html
>>>>
>>>> FOR UPDATE only makes sense for relational databases. Adding
>>>> assumptions it's available would make it harder to use a NoSQL database
>>>> in future.
>>>>
>>>> In the case of SQLite, in effect it's got serialized isolation
>>>> anyway, so the FOR UPDATE is not strictly necessary.
>>>>
>>>> I suggest OFBiz shouldn't have a method that generates syntactically
>>>> incorrect SQL for any database we want to support, and we claim to
>>>> support several. Where there are variations between SQL dialects, we
>>>> have attributes in the data source, such as "join-style".
>>>>
>>>> So think our choices are:
>>>>
>>>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a
>>>> data source. OFBiz would only add the "FOR UPDATE" if it is
>>>> supported. In the case of SQLite, I suspect we'd get away without it,
>>>> but there may be other databases where the absence of "FOR UPDATE"
>>>> might bring on unexpected behaviour.
>>>>
>>>> 2. Don't provide a method to expose FOR UPDATE.
>>>>
>>>> Could you achieve the results you want with a higher transaction
>>>> isolation level like repeatable read, or with optimistic locking
>>>> (enable-lock)?
>>>>
>>>> What do you think?
>>>>
>>>> Cheers
>>>>
>>>> Paul Foxworthy
>>>>
>>>>
>>>> On 26 October 2017 at 07:50, Gareth Carter
>>>> <Ga...@stannah.co.uk>
>>>> wrote:
>>>>
>>>> Ok I will create a Jira with patch for review.
>>>>>
>>>>>
>>>>>
>>>>> Sent from my Samsung Galaxy smartphone.
>>>>>
>>>>>
>>>>>
>>>>> Gareth Carter
>>>>>
>>>>>
>>>>> Software Development Analyst
>>>>>
>>>>>
>>>>> Stannah Management Services Ltd
>>>>>
>>>>>
>>>>> IT Department
>>>>>
>>>>>
>>>>> Ext:
>>>>>
>>>>>
>>>>> 7036
>>>>>
>>>>>
>>>>> DDI:
>>>>>
>>>>>
>>>>> 01264 364311
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [http://logos.stannah.co.uk/stan150.jpg]
>>>>>
>>>>>
>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>>>>> environment before printing this email.
>>>>>
>>>>>
>>>>> -------- Original message --------
>>>>> From: deepak nigam <de...@gmail.com>
>>>>> Date: 25/10/2017 14:06 (GMT+00:00)
>>>>> To: dev@ofbiz.apache.org
>>>>> Subject: Re: Updates to EntityQuery
>>>>>
>>>>> Looking good to me also. Let me know if I can be of any help.
>>>>>
>>>>>
>>>>> Thanks & Regards
>>>>> --
>>>>> Deepak Nigam
>>>>>
>>>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <
>>>>> yash.sharma@hotwaxsystems.com>
>>>>> wrote:
>>>>>
>>>>> I am all up for it, please let me know if I could help.
>>>>>>
>>>>>> Thanks & Regards,
>>>>>> --
>>>>>> *Pradhan Yash Sharma*
>>>>>> *HotWax Systems* |
>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>>>>>>
>>>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
>>>>>> yash.sharma@hotwaxsystems.com>
>>>>>> wrote:
>>>>>>
>>>>>> Yes, Usage of the stream will surely enhance performance to a
>>>>>>> certain extent and removes ceremony from the code base. I think
>>>>>>> Parallel
>>>>>>>
>>>>>> streams
>>>>>
>>>>>> will add enhancements in many folds as we are using a multicore
>>>>>>>
>>>>>> processor
>>>>>
>>>>>> (I have not tested yet), but the Functional approach seems promising.
>>>>>>>
>>>>>>> Thanks & Regards,
>>>>>>> --
>>>>>>> *Pradhan Yash Sharma*
>>>>>>> *HotWax Systems* | www.hotwaxsystems.com<http://
>>>>>>>
>>>>>> www.hotwaxsystems.com>
>>>>
>>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
>>>>>>> Gareth.Carter@stannah.co.uk> wrote:
>>>>>>>
>>>>>>> forUpdate
>>>>>>>>
>>>>>>>> We patched EntityFindOptions with a new field "forUpdate" with
>>>>>>>>
>>>>>>> shorthand
>>>>>
>>>>>> methods in EntityQuery to enable. We then made a change to
>>>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE"
>>>>>>>> on the
>>>>>>>>
>>>>>>> end
>>>>>
>>>>>> of the SQL select statement - this allows for DB row locks (we
>>>>>>>> use
>>>>>>>>
>>>>>>> postgres
>>>>>>
>>>>>>> and works but have not tested other databases). I believe
>>>>>>>> there may
>>>>>>>>
>>>>>>> have
>>>>>
>>>>>> been a discussion about this before
>>>>>>>>
>>>>>>>>
>>>>>>>> forEach on EntityQuery
>>>>>>>>
>>>>>>>> Use Consumer in java and groovy to iterate over a query. This
>>>>>>>> can
>>>>>>>>
>>>>>>> reduce
>>>>>
>>>>>> memory consumption (replacement for queryList()) and boiler
>>>>>>>> plate
>>>>>>>>
>>>>>>> code
>>>>
>>>>> (eg
>>>>>>
>>>>>>> queryIterator(), while loop and close)
>>>>>>>>
>>>>>>>> Example:
>>>>>>>>
>>>>>>>> EntityQuery.use(delegator).from("Foobar")
>>>>>>>> .forEach(item ->
>>>>>>>>      Debug.logInfo(item.toString(), module); );
>>>>>>>>
>>>>>>>> A further update could be to provide stream capabilities
>>>>>>>>
>>>>>>>> Hope this helps, I can provide a patch aswell
>>>>>>>>
>>>>>>>> Gareth Carter
>>>>>>>> Software Development Analyst
>>>>>>>> Stannah Management Services Ltd IT Department
>>>>>>>> Ext:
>>>>>>>> 7036
>>>>>>>> DDI:
>>>>>>>> 01264 364311
>>>>>>>>
>>>>>>>>
>>>>>>>> Please consider the environment before printing this email.
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
>>>>>>>> Sent: 25 October 2017 5:49 AM
>>>>>>>> To: dev@ofbiz.apache.org
>>>>>>>> Cc: arunpatidar@apache.org
>>>>>>>> Subject: Re: Updates to EntityQuery
>>>>>>>>
>>>>>>>> Hello Gareth,
>>>>>>>>
>>>>>>>> Please provide some more details or patch to understand  - forUpdate
>>>>>>>>
>>>>>>> and
>>>>>
>>>>>> forEach method utility.
>>>>>>>>
>>>>>>>> getFieldMap method looks good to me.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks & Regards
>>>>>>>> ---
>>>>>>>> Arun Patidar
>>>>>>>> Manager, Enterprise Software Development
>>>>>>>>
>>>>>>>> HotWax Systems Pvt Ltd.
>>>>>>>>
>>>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
>>>>>>>> Gareth.Carter@stannah.co.uk>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi all
>>>>>>>>>
>>>>>>>>> We have internally patched EntityQuery with some additional
>>>>>>>>> functionality and before I create a Jira was going to see what the
>>>>>>>>>
>>>>>>>> community thinks.
>>>>>>>>
>>>>>>>>> New functionality:
>>>>>>>>>
>>>>>>>>> -       New method getFieldMap which returns a Map object of
>>>>>>>>>
>>>>>>>> selected
>>>>>
>>>>>> fields from GenericValue objects, useful for creating cache map
>>>>>>>>> objects for lookup
>>>>>>>>>
>>>>>>>>> -       Support forUpdate
>>>>>>>>>
>>>>>>>>> -       forEach method to accept Consumer
>>>>>>>>>
>>>>>>>>> We have found these useful and believe the project can benefit,
>>>>>>>>>
>>>>>>>> let
>>>>
>>>>> me
>>>>>
>>>>>> know what you think
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>>
>>>>>>>>> Gareth Carter
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Software Development Analyst
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Stannah Management Services Ltd
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> IT Department
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ext:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 7036
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> DDI:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 01264 364311
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [http://logos.stannah.co.uk/stan150.jpg]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>>>>>>>>>
>>>>>>>> environment
>>>>>>
>>>>>>> before printing this email.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This email is intended only for the above addressee. It may
>>>>>>>>>
>>>>>>>> contain
>>>>
>>>>> privileged information. If you are not the addressee you must not
>>>>>>>>> copy, distribute, disclose or use any of the information in it. If
>>>>>>>>>
>>>>>>>> you
>>>>>
>>>>>> have received it in error, please delete it and notify the sender.
>>>>>>>>>
>>>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah
>>>>>>>>>
>>>>>>>> Management
>>>>
>>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd
>>>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No.
>>>>>>>>>
>>>>>>>> 964804,
>>>>>
>>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
>>>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd registered
>>>>>>>>>
>>>>>>>> No.
>>>>>
>>>>>> 02452728.
>>>>>>>>
>>>>>>>>> All registered offices at Watt Close, East Portway, Andover,
>>>>>>>>> Hampshire,
>>>>>>>>> SP10 3SD, England.
>>>>>>>>>
>>>>>>>>> All Registered in England and Wales.
>>>>>>>>>
>>>>>>>>> This message has been scanned for malware by Websense.
>>>>>>>>> www.websense.com<http://www.websense.com>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> To report this email as spam, please send the original message,
>>>>>>>>
>>>>>>> complete
>>>>>
>>>>>> with headers to spam@websense.com
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> Coherent Software Australia Pty Ltd
>>>> PO Box 2773
>>>> Cheltenham Vic 3192
>>>> Australia
>>>>
>>>> Phone: +61 3 9585 6788
>>>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxN
>>>> CsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A
>>>> -OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSA
>>>> Ux33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z
>>>> Email: info@coherentsoftware.com.au
>>>>
>>>>
>


-- 
Coherent Software Australia Pty Ltd
PO Box 2773
Cheltenham Vic 3192
Australia

Phone: +61 3 9585 6788
Web: http://www.coherentsoftware.com.au/
Email: info@coherentsoftware.com.au

Re: Updates to EntityQuery

Posted by Jacques Le Roux <ja...@les7arts.com>.
I agree with Scott and Taher,

About your question Gareth, we once had a hard coded  "for update" somewhere in OFBiz code (in SequenceUtil.SequenceBank.fillBank() ).
I put it in. Then Jacopo crossed an issue in a cluster and rightly changed it to something better
See https://issues.apache.org/jira/browse/OFBIZ-2353 for more where Adrian gave some directions also.

Since I did not read all in the thread and there is no patch yet I don't know if something alike applies to EntityQuery API. Putting a for update is 
certainly much simpler, so just saying for now.

Jacques


Le 26/10/2017 à 10:46, Gareth Carter a écrit :
> HI all
>
> Thank for the response
>
> It could be considered a hint, if the underlying db does not support it then don't use it. Adding an attribute on the datasource could be the way to go but I also believe we need methods to tell the system when to use it, you don't need it all the time only in selective cases.
>
> Until support is added I see ofbiz as purely RDMS. I don't believe ofbiz should restrict itself on the assumption it might support different databases such as nosql.
>
> There could be an alternative to forUpdate, anyone considered implementing a distributed locking mechanism? The whole point for using forUpdate is to prevent one or more code blocks changing the same database records concurrently, db row locks help but are not supported by all databases but an external locking mechanism may provide a generic way of dealing with this. OFBiz already provides a semaphore on services but I believe you need something at entity level
> Gareth Carter
> Software Development Analyst
> Stannah Management Services Ltd
> IT Department
> Ext:
> 7036
> DDI:
> 01264 364311
>
>
> Please consider the environment before printing this email.
>
> -----Original Message-----
> From: Taher Alkhateeb [mailto:slidingfilaments@gmail.com]
> Sent: 26 October 2017 6:13 AM
> To: OFBIZ Development Mailing List <de...@ofbiz.apache.org>
> Subject: Re: Updates to EntityQuery
>
> Hmmm usually my first gut reaction would be "keep things consistent", however Scott made a good argument for supporting non-common DB features. So my suggestion would be to use something like the widget's <platform-specific ...> tag (or equivalent in code) for any features not supported in all DBMSs. By labeling it as such the user is aware that the DB selection is restricted.
>
> Now with regards to your comment about NoSQL, such databases are fundamentally different, and act more like a key-value based entities without a schema or the resulting ACID properties. It is possible to make it work with the entity engine, but then you'd lose the flexibility it gives you because you want to restrict yourself to the relational model that we apply in OFBiz. The idea of NoSQL is to think at a different conceptual level of your domain model.
>
> On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <sc...@hotwaxsystems.com> wrote:
>> I think it's better to have for-update available for use for the
>> majority of use-cases than it is to exclude it altogether.  I'd rather
>> lose the databases that don't support it than constrain the ones that
>> do and are used most by pretty much everyone.  It would be nice to
>> hear from anyone using a database that doesn't support it since I'm
>> just guessing most of us use postgres or mysql.
>>
>> Has anyone actually looked at how NoSQL databases might work with the
>> OFBiz entity engine or are the concepts too far removed from SQL for
>> it to even be workable?
>>
>> If we did want to support for-update globally, we could always have
>> the delegator simulate it by issuing an UPDATE to lock the rows prior
>> to performing the SELECT.  Although it might not work perfectly in all
>> cases, it could be a good compromise. e.g.
>> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions --
>> lock the rows SELECT * FROM table WHERE ...conditions -- select the
>> rows
>>
>> Regards
>> Scott
>>
>>
>>
>> On 26 October 2017 at 13:45, Paul Foxworthy <pa...@cohsoft.com.au> wrote:
>>
>>> Hi Gareth,
>>>
>>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it:
>>>
>>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html
>>> https://sqlite.org/lang_select.html
>>>
>>> FOR UPDATE only makes sense for relational databases. Adding
>>> assumptions it's available would make it harder to use a NoSQL database in future.
>>>
>>> In the case of SQLite, in effect it's got serialized isolation
>>> anyway, so the FOR UPDATE is not strictly necessary.
>>>
>>> I suggest OFBiz shouldn't have a method that generates syntactically
>>> incorrect SQL for any database we want to support, and we claim to
>>> support several. Where there are variations between SQL dialects, we
>>> have attributes in the data source, such as "join-style".
>>>
>>> So think our choices are:
>>>
>>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a
>>> data source. OFBiz would only add the "FOR UPDATE" if it is
>>> supported. In the case of SQLite, I suspect we'd get away without it,
>>> but there may be other databases where the absence of "FOR UPDATE"
>>> might bring on unexpected behaviour.
>>>
>>> 2. Don't provide a method to expose FOR UPDATE.
>>>
>>> Could you achieve the results you want with a higher transaction
>>> isolation level like repeatable read, or with optimistic locking (enable-lock)?
>>>
>>> What do you think?
>>>
>>> Cheers
>>>
>>> Paul Foxworthy
>>>
>>>
>>> On 26 October 2017 at 07:50, Gareth Carter
>>> <Ga...@stannah.co.uk>
>>> wrote:
>>>
>>>> Ok I will create a Jira with patch for review.
>>>>
>>>>
>>>>
>>>> Sent from my Samsung Galaxy smartphone.
>>>>
>>>>
>>>>
>>>> Gareth Carter
>>>>
>>>>
>>>> Software Development Analyst
>>>>
>>>>
>>>> Stannah Management Services Ltd
>>>>
>>>>
>>>> IT Department
>>>>
>>>>
>>>> Ext:
>>>>
>>>>
>>>> 7036
>>>>
>>>>
>>>> DDI:
>>>>
>>>>
>>>> 01264 364311
>>>>
>>>>
>>>>
>>>>
>>>> [http://logos.stannah.co.uk/stan150.jpg]
>>>>
>>>>
>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>>>> environment before printing this email.
>>>>
>>>>
>>>> -------- Original message --------
>>>> From: deepak nigam <de...@gmail.com>
>>>> Date: 25/10/2017 14:06 (GMT+00:00)
>>>> To: dev@ofbiz.apache.org
>>>> Subject: Re: Updates to EntityQuery
>>>>
>>>> Looking good to me also. Let me know if I can be of any help.
>>>>
>>>>
>>>> Thanks & Regards
>>>> --
>>>> Deepak Nigam
>>>>
>>>> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <
>>>> yash.sharma@hotwaxsystems.com>
>>>> wrote:
>>>>
>>>>> I am all up for it, please let me know if I could help.
>>>>>
>>>>> Thanks & Regards,
>>>>> --
>>>>> *Pradhan Yash Sharma*
>>>>> *HotWax Systems* |
>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>>>>>
>>>>> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
>>>>> yash.sharma@hotwaxsystems.com>
>>>>> wrote:
>>>>>
>>>>>> Yes, Usage of the stream will surely enhance performance to a
>>>>>> certain extent and removes ceremony from the code base. I think
>>>>>> Parallel
>>>> streams
>>>>>> will add enhancements in many folds as we are using a multicore
>>>> processor
>>>>>> (I have not tested yet), but the Functional approach seems promising.
>>>>>>
>>>>>> Thanks & Regards,
>>>>>> --
>>>>>> *Pradhan Yash Sharma*
>>>>>> *HotWax Systems* | www.hotwaxsystems.com<http://
>>> www.hotwaxsystems.com>
>>>>>> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
>>>>>> Gareth.Carter@stannah.co.uk> wrote:
>>>>>>
>>>>>>> forUpdate
>>>>>>>
>>>>>>> We patched EntityFindOptions with a new field "forUpdate" with
>>>> shorthand
>>>>>>> methods in EntityQuery to enable. We then made a change to
>>>>>>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE"
>>>>>>> on the
>>>> end
>>>>>>> of the SQL select statement - this allows for DB row locks (we
>>>>>>> use
>>>>> postgres
>>>>>>> and works but have not tested other databases). I believe
>>>>>>> there may
>>>> have
>>>>>>> been a discussion about this before
>>>>>>>
>>>>>>>
>>>>>>> forEach on EntityQuery
>>>>>>>
>>>>>>> Use Consumer in java and groovy to iterate over a query. This
>>>>>>> can
>>>> reduce
>>>>>>> memory consumption (replacement for queryList()) and boiler
>>>>>>> plate
>>> code
>>>>> (eg
>>>>>>> queryIterator(), while loop and close)
>>>>>>>
>>>>>>> Example:
>>>>>>>
>>>>>>> EntityQuery.use(delegator).from("Foobar")
>>>>>>> .forEach(item ->
>>>>>>>      Debug.logInfo(item.toString(), module); );
>>>>>>>
>>>>>>> A further update could be to provide stream capabilities
>>>>>>>
>>>>>>> Hope this helps, I can provide a patch aswell
>>>>>>>
>>>>>>> Gareth Carter
>>>>>>> Software Development Analyst
>>>>>>> Stannah Management Services Ltd IT Department
>>>>>>> Ext:
>>>>>>> 7036
>>>>>>> DDI:
>>>>>>> 01264 364311
>>>>>>>
>>>>>>>
>>>>>>> Please consider the environment before printing this email.
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
>>>>>>> Sent: 25 October 2017 5:49 AM
>>>>>>> To: dev@ofbiz.apache.org
>>>>>>> Cc: arunpatidar@apache.org
>>>>>>> Subject: Re: Updates to EntityQuery
>>>>>>>
>>>>>>> Hello Gareth,
>>>>>>>
>>>>>>> Please provide some more details or patch to understand  - forUpdate
>>>> and
>>>>>>> forEach method utility.
>>>>>>>
>>>>>>> getFieldMap method looks good to me.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thanks & Regards
>>>>>>> ---
>>>>>>> Arun Patidar
>>>>>>> Manager, Enterprise Software Development
>>>>>>>
>>>>>>> HotWax Systems Pvt Ltd.
>>>>>>>
>>>>>>> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
>>>>>>> Gareth.Carter@stannah.co.uk>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all
>>>>>>>>
>>>>>>>> We have internally patched EntityQuery with some additional
>>>>>>>> functionality and before I create a Jira was going to see what the
>>>>>>> community thinks.
>>>>>>>> New functionality:
>>>>>>>>
>>>>>>>> -       New method getFieldMap which returns a Map object of
>>>> selected
>>>>>>>> fields from GenericValue objects, useful for creating cache map
>>>>>>>> objects for lookup
>>>>>>>>
>>>>>>>> -       Support forUpdate
>>>>>>>>
>>>>>>>> -       forEach method to accept Consumer
>>>>>>>>
>>>>>>>> We have found these useful and believe the project can benefit,
>>> let
>>>> me
>>>>>>>> know what you think
>>>>>>>>
>>>>>>>> Regards
>>>>>>>>
>>>>>>>> Gareth Carter
>>>>>>>>
>>>>>>>>
>>>>>>>> Software Development Analyst
>>>>>>>>
>>>>>>>>
>>>>>>>> Stannah Management Services Ltd
>>>>>>>>
>>>>>>>>
>>>>>>>> IT Department
>>>>>>>>
>>>>>>>>
>>>>>>>> Ext:
>>>>>>>>
>>>>>>>>
>>>>>>>> 7036
>>>>>>>>
>>>>>>>>
>>>>>>>> DDI:
>>>>>>>>
>>>>>>>>
>>>>>>>> 01264 364311
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [http://logos.stannah.co.uk/stan150.jpg]
>>>>>>>>
>>>>>>>>
>>>>>>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>>>>> environment
>>>>>>>> before printing this email.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> This email is intended only for the above addressee. It may
>>> contain
>>>>>>>> privileged information. If you are not the addressee you must not
>>>>>>>> copy, distribute, disclose or use any of the information in it. If
>>>> you
>>>>>>>> have received it in error, please delete it and notify the sender.
>>>>>>>>
>>>>>>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah
>>> Management
>>>>>>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd
>>>>>>>> registered No. 1189799, Stannah Microlifts Ltd registered No.
>>>> 964804,
>>>>>>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
>>>>>>>> registered No. 1401451, Global Upholstery Solutions Ltd registered
>>>> No.
>>>>>>> 02452728.
>>>>>>>> All registered offices at Watt Close, East Portway, Andover,
>>>>>>>> Hampshire,
>>>>>>>> SP10 3SD, England.
>>>>>>>>
>>>>>>>> All Registered in England and Wales.
>>>>>>>>
>>>>>>>> This message has been scanned for malware by Websense.
>>>>>>>> www.websense.com<http://www.websense.com>
>>>>>>>>
>>>>>>>
>>>>>>> To report this email as spam, please send the original message,
>>>> complete
>>>>>>> with headers to spam@websense.com
>>>>>>>
>>>>>>
>>>
>>>
>>> --
>>> Coherent Software Australia Pty Ltd
>>> PO Box 2773
>>> Cheltenham Vic 3192
>>> Australia
>>>
>>> Phone: +61 3 9585 6788
>>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxNCsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A-OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSAUx33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z
>>> Email: info@coherentsoftware.com.au
>>>

RE: Updates to EntityQuery

Posted by Gareth Carter <Ga...@stannah.co.uk>.
HI all

Thank for the response

It could be considered a hint, if the underlying db does not support it then don't use it. Adding an attribute on the datasource could be the way to go but I also believe we need methods to tell the system when to use it, you don't need it all the time only in selective cases.

Until support is added I see ofbiz as purely RDMS. I don't believe ofbiz should restrict itself on the assumption it might support different databases such as nosql.

There could be an alternative to forUpdate, anyone considered implementing a distributed locking mechanism? The whole point for using forUpdate is to prevent one or more code blocks changing the same database records concurrently, db row locks help but are not supported by all databases but an external locking mechanism may provide a generic way of dealing with this. OFBiz already provides a semaphore on services but I believe you need something at entity level
Gareth Carter
Software Development Analyst
Stannah Management Services Ltd
IT Department
Ext:
7036
DDI:
01264 364311


Please consider the environment before printing this email.

-----Original Message-----
From: Taher Alkhateeb [mailto:slidingfilaments@gmail.com]
Sent: 26 October 2017 6:13 AM
To: OFBIZ Development Mailing List <de...@ofbiz.apache.org>
Subject: Re: Updates to EntityQuery

Hmmm usually my first gut reaction would be "keep things consistent", however Scott made a good argument for supporting non-common DB features. So my suggestion would be to use something like the widget's <platform-specific ...> tag (or equivalent in code) for any features not supported in all DBMSs. By labeling it as such the user is aware that the DB selection is restricted.

Now with regards to your comment about NoSQL, such databases are fundamentally different, and act more like a key-value based entities without a schema or the resulting ACID properties. It is possible to make it work with the entity engine, but then you'd lose the flexibility it gives you because you want to restrict yourself to the relational model that we apply in OFBiz. The idea of NoSQL is to think at a different conceptual level of your domain model.

On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray <sc...@hotwaxsystems.com> wrote:
> I think it's better to have for-update available for use for the
> majority of use-cases than it is to exclude it altogether.  I'd rather
> lose the databases that don't support it than constrain the ones that
> do and are used most by pretty much everyone.  It would be nice to
> hear from anyone using a database that doesn't support it since I'm
> just guessing most of us use postgres or mysql.
>
> Has anyone actually looked at how NoSQL databases might work with the
> OFBiz entity engine or are the concepts too far removed from SQL for
> it to even be workable?
>
> If we did want to support for-update globally, we could always have
> the delegator simulate it by issuing an UPDATE to lock the rows prior
> to performing the SELECT.  Although it might not work perfectly in all
> cases, it could be a good compromise. e.g.
> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions --
> lock the rows SELECT * FROM table WHERE ...conditions -- select the
> rows
>
> Regards
> Scott
>
>
>
> On 26 October 2017 at 13:45, Paul Foxworthy <pa...@cohsoft.com.au> wrote:
>
>> Hi Gareth,
>>
>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it:
>>
>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html
>> https://sqlite.org/lang_select.html
>>
>> FOR UPDATE only makes sense for relational databases. Adding
>> assumptions it's available would make it harder to use a NoSQL database in future.
>>
>> In the case of SQLite, in effect it's got serialized isolation
>> anyway, so the FOR UPDATE is not strictly necessary.
>>
>> I suggest OFBiz shouldn't have a method that generates syntactically
>> incorrect SQL for any database we want to support, and we claim to
>> support several. Where there are variations between SQL dialects, we
>> have attributes in the data source, such as "join-style".
>>
>> So think our choices are:
>>
>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a
>> data source. OFBiz would only add the "FOR UPDATE" if it is
>> supported. In the case of SQLite, I suspect we'd get away without it,
>> but there may be other databases where the absence of "FOR UPDATE"
>> might bring on unexpected behaviour.
>>
>> 2. Don't provide a method to expose FOR UPDATE.
>>
>> Could you achieve the results you want with a higher transaction
>> isolation level like repeatable read, or with optimistic locking (enable-lock)?
>>
>> What do you think?
>>
>> Cheers
>>
>> Paul Foxworthy
>>
>>
>> On 26 October 2017 at 07:50, Gareth Carter
>> <Ga...@stannah.co.uk>
>> wrote:
>>
>> > Ok I will create a Jira with patch for review.
>> >
>> >
>> >
>> > Sent from my Samsung Galaxy smartphone.
>> >
>> >
>> >
>> > Gareth Carter
>> >
>> >
>> > Software Development Analyst
>> >
>> >
>> > Stannah Management Services Ltd
>> >
>> >
>> > IT Department
>> >
>> >
>> > Ext:
>> >
>> >
>> > 7036
>> >
>> >
>> > DDI:
>> >
>> >
>> > 01264 364311
>> >
>> >
>> >
>> >
>> > [http://logos.stannah.co.uk/stan150.jpg]
>> >
>> >
>> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>> > environment before printing this email.
>> >
>> >
>> > -------- Original message --------
>> > From: deepak nigam <de...@gmail.com>
>> > Date: 25/10/2017 14:06 (GMT+00:00)
>> > To: dev@ofbiz.apache.org
>> > Subject: Re: Updates to EntityQuery
>> >
>> > Looking good to me also. Let me know if I can be of any help.
>> >
>> >
>> > Thanks & Regards
>> > --
>> > Deepak Nigam
>> >
>> > On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <
>> > yash.sharma@hotwaxsystems.com>
>> > wrote:
>> >
>> > > I am all up for it, please let me know if I could help.
>> > >
>> > > Thanks & Regards,
>> > > --
>> > > *Pradhan Yash Sharma*
>> > > *HotWax Systems* |
>> > > www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>> > >
>> > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
>> > > yash.sharma@hotwaxsystems.com>
>> > > wrote:
>> > >
>> > > > Yes, Usage of the stream will surely enhance performance to a
>> > > > certain extent and removes ceremony from the code base. I think
>> > > > Parallel
>> > streams
>> > > > will add enhancements in many folds as we are using a multicore
>> > processor
>> > > > (I have not tested yet), but the Functional approach seems promising.
>> > > >
>> > > > Thanks & Regards,
>> > > > --
>> > > > *Pradhan Yash Sharma*
>> > > > *HotWax Systems* | www.hotwaxsystems.com<http://
>> www.hotwaxsystems.com>
>> > > >
>> > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
>> > > > Gareth.Carter@stannah.co.uk> wrote:
>> > > >
>> > > >> forUpdate
>> > > >>
>> > > >> We patched EntityFindOptions with a new field "forUpdate" with
>> > shorthand
>> > > >> methods in EntityQuery to enable. We then made a change to
>> > > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE"
>> > > >> on the
>> > end
>> > > >> of the SQL select statement - this allows for DB row locks (we
>> > > >> use
>> > > postgres
>> > > >> and works but have not tested other databases). I believe
>> > > >> there may
>> > have
>> > > >> been a discussion about this before
>> > > >>
>> > > >>
>> > > >> forEach on EntityQuery
>> > > >>
>> > > >> Use Consumer in java and groovy to iterate over a query. This
>> > > >> can
>> > reduce
>> > > >> memory consumption (replacement for queryList()) and boiler
>> > > >> plate
>> code
>> > > (eg
>> > > >> queryIterator(), while loop and close)
>> > > >>
>> > > >> Example:
>> > > >>
>> > > >> EntityQuery.use(delegator).from("Foobar")
>> > > >> .forEach(item ->
>> > > >>     Debug.logInfo(item.toString(), module); );
>> > > >>
>> > > >> A further update could be to provide stream capabilities
>> > > >>
>> > > >> Hope this helps, I can provide a patch aswell
>> > > >>
>> > > >> Gareth Carter
>> > > >> Software Development Analyst
>> > > >> Stannah Management Services Ltd IT Department
>> > > >> Ext:
>> > > >> 7036
>> > > >> DDI:
>> > > >> 01264 364311
>> > > >>
>> > > >>
>> > > >> Please consider the environment before printing this email.
>> > > >>
>> > > >> -----Original Message-----
>> > > >> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
>> > > >> Sent: 25 October 2017 5:49 AM
>> > > >> To: dev@ofbiz.apache.org
>> > > >> Cc: arunpatidar@apache.org
>> > > >> Subject: Re: Updates to EntityQuery
>> > > >>
>> > > >> Hello Gareth,
>> > > >>
>> > > >> Please provide some more details or patch to understand  - forUpdate
>> > and
>> > > >> forEach method utility.
>> > > >>
>> > > >> getFieldMap method looks good to me.
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> Thanks & Regards
>> > > >> ---
>> > > >> Arun Patidar
>> > > >> Manager, Enterprise Software Development
>> > > >>
>> > > >> HotWax Systems Pvt Ltd.
>> > > >>
>> > > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>> > > >>
>> > > >>
>> > > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
>> > > >> Gareth.Carter@stannah.co.uk>
>> > > >> wrote:
>> > > >>
>> > > >> > Hi all
>> > > >> >
>> > > >> > We have internally patched EntityQuery with some additional
>> > > >> > functionality and before I create a Jira was going to see what the
>> > > >> community thinks.
>> > > >> >
>> > > >> > New functionality:
>> > > >> >
>> > > >> > -       New method getFieldMap which returns a Map object of
>> > selected
>> > > >> > fields from GenericValue objects, useful for creating cache map
>> > > >> > objects for lookup
>> > > >> >
>> > > >> > -       Support forUpdate
>> > > >> >
>> > > >> > -       forEach method to accept Consumer
>> > > >> >
>> > > >> > We have found these useful and believe the project can benefit,
>> let
>> > me
>> > > >> > know what you think
>> > > >> >
>> > > >> > Regards
>> > > >> >
>> > > >> > Gareth Carter
>> > > >> >
>> > > >> >
>> > > >> > Software Development Analyst
>> > > >> >
>> > > >> >
>> > > >> > Stannah Management Services Ltd
>> > > >> >
>> > > >> >
>> > > >> > IT Department
>> > > >> >
>> > > >> >
>> > > >> > Ext:
>> > > >> >
>> > > >> >
>> > > >> > 7036
>> > > >> >
>> > > >> >
>> > > >> > DDI:
>> > > >> >
>> > > >> >
>> > > >> > 01264 364311
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > [http://logos.stannah.co.uk/stan150.jpg]
>> > > >> >
>> > > >> >
>> > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>> > > environment
>> > > >> > before printing this email.
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > This email is intended only for the above addressee. It may
>> contain
>> > > >> > privileged information. If you are not the addressee you must not
>> > > >> > copy, distribute, disclose or use any of the information in it. If
>> > you
>> > > >> > have received it in error, please delete it and notify the sender.
>> > > >> >
>> > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah
>> Management
>> > > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
>> > > >> > registered No. 1189799, Stannah Microlifts Ltd registered No.
>> > 964804,
>> > > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
>> > > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered
>> > No.
>> > > >> 02452728.
>> > > >> >
>> > > >> > All registered offices at Watt Close, East Portway, Andover,
>> > > >> > Hampshire,
>> > > >> > SP10 3SD, England.
>> > > >> >
>> > > >> > All Registered in England and Wales.
>> > > >> >
>> > > >> > This message has been scanned for malware by Websense.
>> > > >> > www.websense.com<http://www.websense.com>
>> > > >> >
>> > > >>
>> > > >>
>> > > >> To report this email as spam, please send the original message,
>> > complete
>> > > >> with headers to spam@websense.com
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> Coherent Software Australia Pty Ltd
>> PO Box 2773
>> Cheltenham Vic 3192
>> Australia
>>
>> Phone: +61 3 9585 6788
>> Web: http://webdefence.global.blackspider.com/urlwrap/?q=AXicFYxNCsIwEEa_E02K-NPqRlDs2t5gKJGIaVIm0wYPVnDr0rt4COP28d5Di8MHWN6A-OeqZkoy08B338egEj31ccDcXDfn7tJV63q3bdCyWHV0YlErx6QcArsi0vSAUx33xuScC3BWbNAUb5pL8l8RTwbA9wX8ANMdKWg&Z
>> Email: info@coherentsoftware.com.au
>>

Re: Updates to EntityQuery

Posted by Jacques Le Roux <ja...@les7arts.com>.
Le 27/10/2017 à 09:50, Jacopo Cappellato a écrit :
> On Thu, Oct 26, 2017 at 7:13 AM, Taher Alkhateeb <slidingfilaments@gmail.com
>> wrote:
>> [...] The idea of NoSQL is to think at a different conceptual level of
>> your domain model.
>>
> I agree! In order to leverage the power of NoSQL databases you have to
> write your code accordingly and apply patterns that are fundamentally
> different from the ones adopted with relational databases; also, NoSQL is a
> rather large group that contains different families of products that
> require different handling.
> For all of these reasons I don't think that porting the entity engine to a
> NoSQL database, while technically possible, would be actually useful.
>
> Jacopo
>
Hi,

There are several TLPs which are *more or less* leveraging NoSQL databases with SQL, at least: Apache Drill, Apache Hive, Apache Impala
They have JDBC drivers, but I don't know how much of SQL they support.
Of course there are also other not Apache solutions: Presto (Facebook), Dremel  (Google), ...

One of them, Trafodion (initially from HP), is in incubation for 2+ years: http://trafodion.apache.org/
Here is a simple, clear answer about its possible usage
https://mail-archives.apache.org/mod_mbox/incubator-trafodion-dev/201711.mbox/browser

There are JDBC and ODBC drivers for Trafodion which is based on HBase
http://trafodion.apache.org/architecture-overview.html
http://trafodion.apache.org/faq.html

Jacques

Re: Updates to EntityQuery

Posted by Jacopo Cappellato <ja...@hotwaxsystems.com>.
On Thu, Oct 26, 2017 at 7:13 AM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:

> [...] The idea of NoSQL is to think at a different conceptual level of
> your domain model.
>

I agree! In order to leverage the power of NoSQL databases you have to
write your code accordingly and apply patterns that are fundamentally
different from the ones adopted with relational databases; also, NoSQL is a
rather large group that contains different families of products that
require different handling.
For all of these reasons I don't think that porting the entity engine to a
NoSQL database, while technically possible, would be actually useful.

Jacopo

Re: Updates to EntityQuery

Posted by Taher Alkhateeb <sl...@gmail.com>.
Hmmm usually my first gut reaction would be "keep things consistent",
however Scott made a good argument for supporting non-common DB
features. So my suggestion would be to use something like the widget's
<platform-specific ...> tag (or equivalent in code) for any features
not supported in all DBMSs. By labeling it as such the user is aware
that the DB selection is restricted.

Now with regards to your comment about NoSQL, such databases are
fundamentally different, and act more like a key-value based entities
without a schema or the resulting ACID properties. It is possible to
make it work with the entity engine, but then you'd lose the
flexibility it gives you because you want to restrict yourself to the
relational model that we apply in OFBiz. The idea of NoSQL is to think
at a different conceptual level of your domain model.

On Thu, Oct 26, 2017 at 4:33 AM, Scott Gray
<sc...@hotwaxsystems.com> wrote:
> I think it's better to have for-update available for use for the majority
> of use-cases than it is to exclude it altogether.  I'd rather lose the
> databases that don't support it than constrain the ones that do and are
> used most by pretty much everyone.  It would be nice to hear from anyone
> using a database that doesn't support it since I'm just guessing most of us
> use postgres or mysql.
>
> Has anyone actually looked at how NoSQL databases might work with the OFBiz
> entity engine or are the concepts too far removed from SQL for it to even
> be workable?
>
> If we did want to support for-update globally, we could always have the
> delegator simulate it by issuing an UPDATE to lock the rows prior to
> performing the SELECT.  Although it might not work perfectly in all cases,
> it could be a good compromise. e.g.
> UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- lock the
> rows
> SELECT * FROM table WHERE ...conditions -- select the rows
>
> Regards
> Scott
>
>
>
> On 26 October 2017 at 13:45, Paul Foxworthy <pa...@cohsoft.com.au> wrote:
>
>> Hi Gareth,
>>
>> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it:
>>
>> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html
>> https://sqlite.org/lang_select.html
>>
>> FOR UPDATE only makes sense for relational databases. Adding assumptions
>> it's available would make it harder to use a NoSQL database in future.
>>
>> In the case of SQLite, in effect it's got serialized isolation anyway, so
>> the FOR UPDATE is not strictly necessary.
>>
>> I suggest OFBiz shouldn't have a method that generates syntactically
>> incorrect SQL for any database we want to support, and we claim to support
>> several. Where there are variations between SQL dialects, we have
>> attributes in the data source, such as "join-style".
>>
>> So think our choices are:
>>
>> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a data
>> source. OFBiz would only add the "FOR UPDATE" if it is supported. In the
>> case of SQLite, I suspect we'd get away without it, but there may be other
>> databases where the absence of "FOR UPDATE" might bring on unexpected
>> behaviour.
>>
>> 2. Don't provide a method to expose FOR UPDATE.
>>
>> Could you achieve the results you want with a higher transaction isolation
>> level like repeatable read, or with optimistic locking (enable-lock)?
>>
>> What do you think?
>>
>> Cheers
>>
>> Paul Foxworthy
>>
>>
>> On 26 October 2017 at 07:50, Gareth Carter <Ga...@stannah.co.uk>
>> wrote:
>>
>> > Ok I will create a Jira with patch for review.
>> >
>> >
>> >
>> > Sent from my Samsung Galaxy smartphone.
>> >
>> >
>> >
>> > Gareth Carter
>> >
>> >
>> > Software Development Analyst
>> >
>> >
>> > Stannah Management Services Ltd
>> >
>> >
>> > IT Department
>> >
>> >
>> > Ext:
>> >
>> >
>> > 7036
>> >
>> >
>> > DDI:
>> >
>> >
>> > 01264 364311
>> >
>> >
>> >
>> >
>> > [http://logos.stannah.co.uk/stan150.jpg]
>> >
>> >
>> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
>> > before printing this email.
>> >
>> >
>> > -------- Original message --------
>> > From: deepak nigam <de...@gmail.com>
>> > Date: 25/10/2017 14:06 (GMT+00:00)
>> > To: dev@ofbiz.apache.org
>> > Subject: Re: Updates to EntityQuery
>> >
>> > Looking good to me also. Let me know if I can be of any help.
>> >
>> >
>> > Thanks & Regards
>> > --
>> > Deepak Nigam
>> >
>> > On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <
>> > yash.sharma@hotwaxsystems.com>
>> > wrote:
>> >
>> > > I am all up for it, please let me know if I could help.
>> > >
>> > > Thanks & Regards,
>> > > --
>> > > *Pradhan Yash Sharma*
>> > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>> > >
>> > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
>> > > yash.sharma@hotwaxsystems.com>
>> > > wrote:
>> > >
>> > > > Yes, Usage of the stream will surely enhance performance to a certain
>> > > > extent and removes ceremony from the code base. I think Parallel
>> > streams
>> > > > will add enhancements in many folds as we are using a multicore
>> > processor
>> > > > (I have not tested yet), but the Functional approach seems promising.
>> > > >
>> > > > Thanks & Regards,
>> > > > --
>> > > > *Pradhan Yash Sharma*
>> > > > *HotWax Systems* | www.hotwaxsystems.com<http://
>> www.hotwaxsystems.com>
>> > > >
>> > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
>> > > > Gareth.Carter@stannah.co.uk> wrote:
>> > > >
>> > > >> forUpdate
>> > > >>
>> > > >> We patched EntityFindOptions with a new field "forUpdate" with
>> > shorthand
>> > > >> methods in EntityQuery to enable. We then made a change to
>> > > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the
>> > end
>> > > >> of the SQL select statement - this allows for DB row locks (we use
>> > > postgres
>> > > >> and works but have not tested other databases). I believe there may
>> > have
>> > > >> been a discussion about this before
>> > > >>
>> > > >>
>> > > >> forEach on EntityQuery
>> > > >>
>> > > >> Use Consumer in java and groovy to iterate over a query. This can
>> > reduce
>> > > >> memory consumption (replacement for queryList()) and boiler plate
>> code
>> > > (eg
>> > > >> queryIterator(), while loop and close)
>> > > >>
>> > > >> Example:
>> > > >>
>> > > >> EntityQuery.use(delegator).from("Foobar")
>> > > >> .forEach(item ->
>> > > >>     Debug.logInfo(item.toString(), module);
>> > > >> );
>> > > >>
>> > > >> A further update could be to provide stream capabilities
>> > > >>
>> > > >> Hope this helps, I can provide a patch aswell
>> > > >>
>> > > >> Gareth Carter
>> > > >> Software Development Analyst
>> > > >> Stannah Management Services Ltd
>> > > >> IT Department
>> > > >> Ext:
>> > > >> 7036
>> > > >> DDI:
>> > > >> 01264 364311
>> > > >>
>> > > >>
>> > > >> Please consider the environment before printing this email.
>> > > >>
>> > > >> -----Original Message-----
>> > > >> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
>> > > >> Sent: 25 October 2017 5:49 AM
>> > > >> To: dev@ofbiz.apache.org
>> > > >> Cc: arunpatidar@apache.org
>> > > >> Subject: Re: Updates to EntityQuery
>> > > >>
>> > > >> Hello Gareth,
>> > > >>
>> > > >> Please provide some more details or patch to understand  - forUpdate
>> > and
>> > > >> forEach method utility.
>> > > >>
>> > > >> getFieldMap method looks good to me.
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> Thanks & Regards
>> > > >> ---
>> > > >> Arun Patidar
>> > > >> Manager, Enterprise Software Development
>> > > >>
>> > > >> HotWax Systems Pvt Ltd.
>> > > >>
>> > > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>> > > >>
>> > > >>
>> > > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
>> > > >> Gareth.Carter@stannah.co.uk>
>> > > >> wrote:
>> > > >>
>> > > >> > Hi all
>> > > >> >
>> > > >> > We have internally patched EntityQuery with some additional
>> > > >> > functionality and before I create a Jira was going to see what the
>> > > >> community thinks.
>> > > >> >
>> > > >> > New functionality:
>> > > >> >
>> > > >> > -       New method getFieldMap which returns a Map object of
>> > selected
>> > > >> > fields from GenericValue objects, useful for creating cache map
>> > > >> > objects for lookup
>> > > >> >
>> > > >> > -       Support forUpdate
>> > > >> >
>> > > >> > -       forEach method to accept Consumer
>> > > >> >
>> > > >> > We have found these useful and believe the project can benefit,
>> let
>> > me
>> > > >> > know what you think
>> > > >> >
>> > > >> > Regards
>> > > >> >
>> > > >> > Gareth Carter
>> > > >> >
>> > > >> >
>> > > >> > Software Development Analyst
>> > > >> >
>> > > >> >
>> > > >> > Stannah Management Services Ltd
>> > > >> >
>> > > >> >
>> > > >> > IT Department
>> > > >> >
>> > > >> >
>> > > >> > Ext:
>> > > >> >
>> > > >> >
>> > > >> > 7036
>> > > >> >
>> > > >> >
>> > > >> > DDI:
>> > > >> >
>> > > >> >
>> > > >> > 01264 364311
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > [http://logos.stannah.co.uk/stan150.jpg]
>> > > >> >
>> > > >> >
>> > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the
>> > > environment
>> > > >> > before printing this email.
>> > > >> >
>> > > >> >
>> > > >> >
>> > > >> > This email is intended only for the above addressee. It may
>> contain
>> > > >> > privileged information. If you are not the addressee you must not
>> > > >> > copy, distribute, disclose or use any of the information in it. If
>> > you
>> > > >> > have received it in error, please delete it and notify the sender.
>> > > >> >
>> > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah
>> Management
>> > > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
>> > > >> > registered No. 1189799, Stannah Microlifts Ltd registered No.
>> > 964804,
>> > > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
>> > > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered
>> > No.
>> > > >> 02452728.
>> > > >> >
>> > > >> > All registered offices at Watt Close, East Portway, Andover,
>> > > >> > Hampshire,
>> > > >> > SP10 3SD, England.
>> > > >> >
>> > > >> > All Registered in England and Wales.
>> > > >> >
>> > > >> > This message has been scanned for malware by Websense.
>> > > >> > www.websense.com<http://www.websense.com>
>> > > >> >
>> > > >>
>> > > >>
>> > > >> To report this email as spam, please send the original message,
>> > complete
>> > > >> with headers to spam@websense.com
>> > > >>
>> > > >
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> Coherent Software Australia Pty Ltd
>> PO Box 2773
>> Cheltenham Vic 3192
>> Australia
>>
>> Phone: +61 3 9585 6788
>> Web: http://www.coherentsoftware.com.au/
>> Email: info@coherentsoftware.com.au
>>

Re: Updates to EntityQuery

Posted by Scott Gray <sc...@hotwaxsystems.com>.
I think it's better to have for-update available for use for the majority
of use-cases than it is to exclude it altogether.  I'd rather lose the
databases that don't support it than constrain the ones that do and are
used most by pretty much everyone.  It would be nice to hear from anyone
using a database that doesn't support it since I'm just guessing most of us
use postgres or mysql.

Has anyone actually looked at how NoSQL databases might work with the OFBiz
entity engine or are the concepts too far removed from SQL for it to even
be workable?

If we did want to support for-update globally, we could always have the
delegator simulate it by issuing an UPDATE to lock the rows prior to
performing the SELECT.  Although it might not work perfectly in all cases,
it could be a good compromise. e.g.
UPDATE table SET lastUpdatedTxStamp = NOW() WHERE ...conditions -- lock the
rows
SELECT * FROM table WHERE ...conditions -- select the rows

Regards
Scott



On 26 October 2017 at 13:45, Paul Foxworthy <pa...@cohsoft.com.au> wrote:

> Hi Gareth,
>
> FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it:
>
> http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html
> https://sqlite.org/lang_select.html
>
> FOR UPDATE only makes sense for relational databases. Adding assumptions
> it's available would make it harder to use a NoSQL database in future.
>
> In the case of SQLite, in effect it's got serialized isolation anyway, so
> the FOR UPDATE is not strictly necessary.
>
> I suggest OFBiz shouldn't have a method that generates syntactically
> incorrect SQL for any database we want to support, and we claim to support
> several. Where there are variations between SQL dialects, we have
> attributes in the data source, such as "join-style".
>
> So think our choices are:
>
> 1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a data
> source. OFBiz would only add the "FOR UPDATE" if it is supported. In the
> case of SQLite, I suspect we'd get away without it, but there may be other
> databases where the absence of "FOR UPDATE" might bring on unexpected
> behaviour.
>
> 2. Don't provide a method to expose FOR UPDATE.
>
> Could you achieve the results you want with a higher transaction isolation
> level like repeatable read, or with optimistic locking (enable-lock)?
>
> What do you think?
>
> Cheers
>
> Paul Foxworthy
>
>
> On 26 October 2017 at 07:50, Gareth Carter <Ga...@stannah.co.uk>
> wrote:
>
> > Ok I will create a Jira with patch for review.
> >
> >
> >
> > Sent from my Samsung Galaxy smartphone.
> >
> >
> >
> > Gareth Carter
> >
> >
> > Software Development Analyst
> >
> >
> > Stannah Management Services Ltd
> >
> >
> > IT Department
> >
> >
> > Ext:
> >
> >
> > 7036
> >
> >
> > DDI:
> >
> >
> > 01264 364311
> >
> >
> >
> >
> > [http://logos.stannah.co.uk/stan150.jpg]
> >
> >
> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
> > before printing this email.
> >
> >
> > -------- Original message --------
> > From: deepak nigam <de...@gmail.com>
> > Date: 25/10/2017 14:06 (GMT+00:00)
> > To: dev@ofbiz.apache.org
> > Subject: Re: Updates to EntityQuery
> >
> > Looking good to me also. Let me know if I can be of any help.
> >
> >
> > Thanks & Regards
> > --
> > Deepak Nigam
> >
> > On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <
> > yash.sharma@hotwaxsystems.com>
> > wrote:
> >
> > > I am all up for it, please let me know if I could help.
> > >
> > > Thanks & Regards,
> > > --
> > > *Pradhan Yash Sharma*
> > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com>
> > >
> > > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
> > > yash.sharma@hotwaxsystems.com>
> > > wrote:
> > >
> > > > Yes, Usage of the stream will surely enhance performance to a certain
> > > > extent and removes ceremony from the code base. I think Parallel
> > streams
> > > > will add enhancements in many folds as we are using a multicore
> > processor
> > > > (I have not tested yet), but the Functional approach seems promising.
> > > >
> > > > Thanks & Regards,
> > > > --
> > > > *Pradhan Yash Sharma*
> > > > *HotWax Systems* | www.hotwaxsystems.com<http://
> www.hotwaxsystems.com>
> > > >
> > > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
> > > > Gareth.Carter@stannah.co.uk> wrote:
> > > >
> > > >> forUpdate
> > > >>
> > > >> We patched EntityFindOptions with a new field "forUpdate" with
> > shorthand
> > > >> methods in EntityQuery to enable. We then made a change to
> > > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the
> > end
> > > >> of the SQL select statement - this allows for DB row locks (we use
> > > postgres
> > > >> and works but have not tested other databases). I believe there may
> > have
> > > >> been a discussion about this before
> > > >>
> > > >>
> > > >> forEach on EntityQuery
> > > >>
> > > >> Use Consumer in java and groovy to iterate over a query. This can
> > reduce
> > > >> memory consumption (replacement for queryList()) and boiler plate
> code
> > > (eg
> > > >> queryIterator(), while loop and close)
> > > >>
> > > >> Example:
> > > >>
> > > >> EntityQuery.use(delegator).from("Foobar")
> > > >> .forEach(item ->
> > > >>     Debug.logInfo(item.toString(), module);
> > > >> );
> > > >>
> > > >> A further update could be to provide stream capabilities
> > > >>
> > > >> Hope this helps, I can provide a patch aswell
> > > >>
> > > >> Gareth Carter
> > > >> Software Development Analyst
> > > >> Stannah Management Services Ltd
> > > >> IT Department
> > > >> Ext:
> > > >> 7036
> > > >> DDI:
> > > >> 01264 364311
> > > >>
> > > >>
> > > >> Please consider the environment before printing this email.
> > > >>
> > > >> -----Original Message-----
> > > >> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
> > > >> Sent: 25 October 2017 5:49 AM
> > > >> To: dev@ofbiz.apache.org
> > > >> Cc: arunpatidar@apache.org
> > > >> Subject: Re: Updates to EntityQuery
> > > >>
> > > >> Hello Gareth,
> > > >>
> > > >> Please provide some more details or patch to understand  - forUpdate
> > and
> > > >> forEach method utility.
> > > >>
> > > >> getFieldMap method looks good to me.
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Thanks & Regards
> > > >> ---
> > > >> Arun Patidar
> > > >> Manager, Enterprise Software Development
> > > >>
> > > >> HotWax Systems Pvt Ltd.
> > > >>
> > > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
> > > >>
> > > >>
> > > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
> > > >> Gareth.Carter@stannah.co.uk>
> > > >> wrote:
> > > >>
> > > >> > Hi all
> > > >> >
> > > >> > We have internally patched EntityQuery with some additional
> > > >> > functionality and before I create a Jira was going to see what the
> > > >> community thinks.
> > > >> >
> > > >> > New functionality:
> > > >> >
> > > >> > -       New method getFieldMap which returns a Map object of
> > selected
> > > >> > fields from GenericValue objects, useful for creating cache map
> > > >> > objects for lookup
> > > >> >
> > > >> > -       Support forUpdate
> > > >> >
> > > >> > -       forEach method to accept Consumer
> > > >> >
> > > >> > We have found these useful and believe the project can benefit,
> let
> > me
> > > >> > know what you think
> > > >> >
> > > >> > Regards
> > > >> >
> > > >> > Gareth Carter
> > > >> >
> > > >> >
> > > >> > Software Development Analyst
> > > >> >
> > > >> >
> > > >> > Stannah Management Services Ltd
> > > >> >
> > > >> >
> > > >> > IT Department
> > > >> >
> > > >> >
> > > >> > Ext:
> > > >> >
> > > >> >
> > > >> > 7036
> > > >> >
> > > >> >
> > > >> > DDI:
> > > >> >
> > > >> >
> > > >> > 01264 364311
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > [http://logos.stannah.co.uk/stan150.jpg]
> > > >> >
> > > >> >
> > > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the
> > > environment
> > > >> > before printing this email.
> > > >> >
> > > >> >
> > > >> >
> > > >> > This email is intended only for the above addressee. It may
> contain
> > > >> > privileged information. If you are not the addressee you must not
> > > >> > copy, distribute, disclose or use any of the information in it. If
> > you
> > > >> > have received it in error, please delete it and notify the sender.
> > > >> >
> > > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah
> Management
> > > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
> > > >> > registered No. 1189799, Stannah Microlifts Ltd registered No.
> > 964804,
> > > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
> > > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered
> > No.
> > > >> 02452728.
> > > >> >
> > > >> > All registered offices at Watt Close, East Portway, Andover,
> > > >> > Hampshire,
> > > >> > SP10 3SD, England.
> > > >> >
> > > >> > All Registered in England and Wales.
> > > >> >
> > > >> > This message has been scanned for malware by Websense.
> > > >> > www.websense.com<http://www.websense.com>
> > > >> >
> > > >>
> > > >>
> > > >> To report this email as spam, please send the original message,
> > complete
> > > >> with headers to spam@websense.com
> > > >>
> > > >
> > > >
> > >
> >
>
>
>
> --
> Coherent Software Australia Pty Ltd
> PO Box 2773
> Cheltenham Vic 3192
> Australia
>
> Phone: +61 3 9585 6788
> Web: http://www.coherentsoftware.com.au/
> Email: info@coherentsoftware.com.au
>

Re: Updates to EntityQuery

Posted by Paul Foxworthy <pa...@cohsoft.com.au>.
Hi Gareth,

FOR UPDATE is standard SQL, but even so SQLite for one doesn't support it:

http://sqlite.1065341.n5.nabble.com/SELECT-FOR-UPDATE-td89630.html
https://sqlite.org/lang_select.html

FOR UPDATE only makes sense for relational databases. Adding assumptions
it's available would make it harder to use a NoSQL database in future.

In the case of SQLite, in effect it's got serialized isolation anyway, so
the FOR UPDATE is not strictly necessary.

I suggest OFBiz shouldn't have a method that generates syntactically
incorrect SQL for any database we want to support, and we claim to support
several. Where there are variations between SQL dialects, we have
attributes in the data source, such as "join-style".

So think our choices are:

1. Add an attribute to say "SELECT ... FOR UPDATE" is supported by a data
source. OFBiz would only add the "FOR UPDATE" if it is supported. In the
case of SQLite, I suspect we'd get away without it, but there may be other
databases where the absence of "FOR UPDATE" might bring on unexpected
behaviour.

2. Don't provide a method to expose FOR UPDATE.

Could you achieve the results you want with a higher transaction isolation
level like repeatable read, or with optimistic locking (enable-lock)?

What do you think?

Cheers

Paul Foxworthy


On 26 October 2017 at 07:50, Gareth Carter <Ga...@stannah.co.uk>
wrote:

> Ok I will create a Jira with patch for review.
>
>
>
> Sent from my Samsung Galaxy smartphone.
>
>
>
> Gareth Carter
>
>
> Software Development Analyst
>
>
> Stannah Management Services Ltd
>
>
> IT Department
>
>
> Ext:
>
>
> 7036
>
>
> DDI:
>
>
> 01264 364311
>
>
>
>
> [http://logos.stannah.co.uk/stan150.jpg]
>
>
> [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
> before printing this email.
>
>
> -------- Original message --------
> From: deepak nigam <de...@gmail.com>
> Date: 25/10/2017 14:06 (GMT+00:00)
> To: dev@ofbiz.apache.org
> Subject: Re: Updates to EntityQuery
>
> Looking good to me also. Let me know if I can be of any help.
>
>
> Thanks & Regards
> --
> Deepak Nigam
>
> On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <
> yash.sharma@hotwaxsystems.com>
> wrote:
>
> > I am all up for it, please let me know if I could help.
> >
> > Thanks & Regards,
> > --
> > *Pradhan Yash Sharma*
> > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com>
> >
> > On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
> > yash.sharma@hotwaxsystems.com>
> > wrote:
> >
> > > Yes, Usage of the stream will surely enhance performance to a certain
> > > extent and removes ceremony from the code base. I think Parallel
> streams
> > > will add enhancements in many folds as we are using a multicore
> processor
> > > (I have not tested yet), but the Functional approach seems promising.
> > >
> > > Thanks & Regards,
> > > --
> > > *Pradhan Yash Sharma*
> > > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com>
> > >
> > > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
> > > Gareth.Carter@stannah.co.uk> wrote:
> > >
> > >> forUpdate
> > >>
> > >> We patched EntityFindOptions with a new field "forUpdate" with
> shorthand
> > >> methods in EntityQuery to enable. We then made a change to
> > >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the
> end
> > >> of the SQL select statement - this allows for DB row locks (we use
> > postgres
> > >> and works but have not tested other databases). I believe there may
> have
> > >> been a discussion about this before
> > >>
> > >>
> > >> forEach on EntityQuery
> > >>
> > >> Use Consumer in java and groovy to iterate over a query. This can
> reduce
> > >> memory consumption (replacement for queryList()) and boiler plate code
> > (eg
> > >> queryIterator(), while loop and close)
> > >>
> > >> Example:
> > >>
> > >> EntityQuery.use(delegator).from("Foobar")
> > >> .forEach(item ->
> > >>     Debug.logInfo(item.toString(), module);
> > >> );
> > >>
> > >> A further update could be to provide stream capabilities
> > >>
> > >> Hope this helps, I can provide a patch aswell
> > >>
> > >> Gareth Carter
> > >> Software Development Analyst
> > >> Stannah Management Services Ltd
> > >> IT Department
> > >> Ext:
> > >> 7036
> > >> DDI:
> > >> 01264 364311
> > >>
> > >>
> > >> Please consider the environment before printing this email.
> > >>
> > >> -----Original Message-----
> > >> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
> > >> Sent: 25 October 2017 5:49 AM
> > >> To: dev@ofbiz.apache.org
> > >> Cc: arunpatidar@apache.org
> > >> Subject: Re: Updates to EntityQuery
> > >>
> > >> Hello Gareth,
> > >>
> > >> Please provide some more details or patch to understand  - forUpdate
> and
> > >> forEach method utility.
> > >>
> > >> getFieldMap method looks good to me.
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks & Regards
> > >> ---
> > >> Arun Patidar
> > >> Manager, Enterprise Software Development
> > >>
> > >> HotWax Systems Pvt Ltd.
> > >>
> > >> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
> > >>
> > >>
> > >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
> > >> Gareth.Carter@stannah.co.uk>
> > >> wrote:
> > >>
> > >> > Hi all
> > >> >
> > >> > We have internally patched EntityQuery with some additional
> > >> > functionality and before I create a Jira was going to see what the
> > >> community thinks.
> > >> >
> > >> > New functionality:
> > >> >
> > >> > -       New method getFieldMap which returns a Map object of
> selected
> > >> > fields from GenericValue objects, useful for creating cache map
> > >> > objects for lookup
> > >> >
> > >> > -       Support forUpdate
> > >> >
> > >> > -       forEach method to accept Consumer
> > >> >
> > >> > We have found these useful and believe the project can benefit, let
> me
> > >> > know what you think
> > >> >
> > >> > Regards
> > >> >
> > >> > Gareth Carter
> > >> >
> > >> >
> > >> > Software Development Analyst
> > >> >
> > >> >
> > >> > Stannah Management Services Ltd
> > >> >
> > >> >
> > >> > IT Department
> > >> >
> > >> >
> > >> > Ext:
> > >> >
> > >> >
> > >> > 7036
> > >> >
> > >> >
> > >> > DDI:
> > >> >
> > >> >
> > >> > 01264 364311
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > [http://logos.stannah.co.uk/stan150.jpg]
> > >> >
> > >> >
> > >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the
> > environment
> > >> > before printing this email.
> > >> >
> > >> >
> > >> >
> > >> > This email is intended only for the above addressee. It may contain
> > >> > privileged information. If you are not the addressee you must not
> > >> > copy, distribute, disclose or use any of the information in it. If
> you
> > >> > have received it in error, please delete it and notify the sender.
> > >> >
> > >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
> > >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
> > >> > registered No. 1189799, Stannah Microlifts Ltd registered No.
> 964804,
> > >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
> > >> > registered No. 1401451, Global Upholstery Solutions Ltd registered
> No.
> > >> 02452728.
> > >> >
> > >> > All registered offices at Watt Close, East Portway, Andover,
> > >> > Hampshire,
> > >> > SP10 3SD, England.
> > >> >
> > >> > All Registered in England and Wales.
> > >> >
> > >> > This message has been scanned for malware by Websense.
> > >> > www.websense.com<http://www.websense.com>
> > >> >
> > >>
> > >>
> > >> To report this email as spam, please send the original message,
> complete
> > >> with headers to spam@websense.com
> > >>
> > >
> > >
> >
>



-- 
Coherent Software Australia Pty Ltd
PO Box 2773
Cheltenham Vic 3192
Australia

Phone: +61 3 9585 6788
Web: http://www.coherentsoftware.com.au/
Email: info@coherentsoftware.com.au

Re: Updates to EntityQuery

Posted by Gareth Carter <Ga...@stannah.co.uk>.
Ok I will create a Jira with patch for review.



Sent from my Samsung Galaxy smartphone.



Gareth Carter


Software Development Analyst


Stannah Management Services Ltd


IT Department


Ext:


7036


DDI:


01264 364311




[http://logos.stannah.co.uk/stan150.jpg]


[http://logos.stannah.co.uk/enviro.jpg]Please consider the environment before printing this email.


-------- Original message --------
From: deepak nigam <de...@gmail.com>
Date: 25/10/2017 14:06 (GMT+00:00)
To: dev@ofbiz.apache.org
Subject: Re: Updates to EntityQuery

Looking good to me also. Let me know if I can be of any help.


Thanks & Regards
--
Deepak Nigam

On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <ya...@hotwaxsystems.com>
wrote:

> I am all up for it, please let me know if I could help.
>
> Thanks & Regards,
> --
> *Pradhan Yash Sharma*
> *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com>
>
> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
> yash.sharma@hotwaxsystems.com>
> wrote:
>
> > Yes, Usage of the stream will surely enhance performance to a certain
> > extent and removes ceremony from the code base. I think Parallel streams
> > will add enhancements in many folds as we are using a multicore processor
> > (I have not tested yet), but the Functional approach seems promising.
> >
> > Thanks & Regards,
> > --
> > *Pradhan Yash Sharma*
> > *HotWax Systems* | www.hotwaxsystems.com<http://www.hotwaxsystems.com>
> >
> > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
> > Gareth.Carter@stannah.co.uk> wrote:
> >
> >> forUpdate
> >>
> >> We patched EntityFindOptions with a new field "forUpdate" with shorthand
> >> methods in EntityQuery to enable. We then made a change to
> >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end
> >> of the SQL select statement - this allows for DB row locks (we use
> postgres
> >> and works but have not tested other databases). I believe there may have
> >> been a discussion about this before
> >>
> >>
> >> forEach on EntityQuery
> >>
> >> Use Consumer in java and groovy to iterate over a query. This can reduce
> >> memory consumption (replacement for queryList()) and boiler plate code
> (eg
> >> queryIterator(), while loop and close)
> >>
> >> Example:
> >>
> >> EntityQuery.use(delegator).from("Foobar")
> >> .forEach(item ->
> >>     Debug.logInfo(item.toString(), module);
> >> );
> >>
> >> A further update could be to provide stream capabilities
> >>
> >> Hope this helps, I can provide a patch aswell
> >>
> >> Gareth Carter
> >> Software Development Analyst
> >> Stannah Management Services Ltd
> >> IT Department
> >> Ext:
> >> 7036
> >> DDI:
> >> 01264 364311
> >>
> >>
> >> Please consider the environment before printing this email.
> >>
> >> -----Original Message-----
> >> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
> >> Sent: 25 October 2017 5:49 AM
> >> To: dev@ofbiz.apache.org
> >> Cc: arunpatidar@apache.org
> >> Subject: Re: Updates to EntityQuery
> >>
> >> Hello Gareth,
> >>
> >> Please provide some more details or patch to understand  - forUpdate and
> >> forEach method utility.
> >>
> >> getFieldMap method looks good to me.
> >>
> >>
> >>
> >> --
> >> Thanks & Regards
> >> ---
> >> Arun Patidar
> >> Manager, Enterprise Software Development
> >>
> >> HotWax Systems Pvt Ltd.
> >>
> >> www.hotwaxsystems.com<http://www.hotwaxsystems.com>
> >>
> >>
> >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
> >> Gareth.Carter@stannah.co.uk>
> >> wrote:
> >>
> >> > Hi all
> >> >
> >> > We have internally patched EntityQuery with some additional
> >> > functionality and before I create a Jira was going to see what the
> >> community thinks.
> >> >
> >> > New functionality:
> >> >
> >> > -       New method getFieldMap which returns a Map object of selected
> >> > fields from GenericValue objects, useful for creating cache map
> >> > objects for lookup
> >> >
> >> > -       Support forUpdate
> >> >
> >> > -       forEach method to accept Consumer
> >> >
> >> > We have found these useful and believe the project can benefit, let me
> >> > know what you think
> >> >
> >> > Regards
> >> >
> >> > Gareth Carter
> >> >
> >> >
> >> > Software Development Analyst
> >> >
> >> >
> >> > Stannah Management Services Ltd
> >> >
> >> >
> >> > IT Department
> >> >
> >> >
> >> > Ext:
> >> >
> >> >
> >> > 7036
> >> >
> >> >
> >> > DDI:
> >> >
> >> >
> >> > 01264 364311
> >> >
> >> >
> >> >
> >> >
> >> > [http://logos.stannah.co.uk/stan150.jpg]
> >> >
> >> >
> >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the
> environment
> >> > before printing this email.
> >> >
> >> >
> >> >
> >> > This email is intended only for the above addressee. It may contain
> >> > privileged information. If you are not the addressee you must not
> >> > copy, distribute, disclose or use any of the information in it. If you
> >> > have received it in error, please delete it and notify the sender.
> >> >
> >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
> >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
> >> > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804,
> >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
> >> > registered No. 1401451, Global Upholstery Solutions Ltd registered No.
> >> 02452728.
> >> >
> >> > All registered offices at Watt Close, East Portway, Andover,
> >> > Hampshire,
> >> > SP10 3SD, England.
> >> >
> >> > All Registered in England and Wales.
> >> >
> >> > This message has been scanned for malware by Websense.
> >> > www.websense.com<http://www.websense.com>
> >> >
> >>
> >>
> >> To report this email as spam, please send the original message, complete
> >> with headers to spam@websense.com
> >>
> >
> >
>

Re: Updates to EntityQuery

Posted by deepak nigam <de...@gmail.com>.
Looking good to me also. Let me know if I can be of any help.


Thanks & Regards
--
Deepak Nigam

On Wed, Oct 25, 2017 at 6:26 PM, Yash Sharma <ya...@hotwaxsystems.com>
wrote:

> I am all up for it, please let me know if I could help.
>
> Thanks & Regards,
> --
> *Pradhan Yash Sharma*
> *HotWax Systems* | www.hotwaxsystems.com
>
> On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <
> yash.sharma@hotwaxsystems.com>
> wrote:
>
> > Yes, Usage of the stream will surely enhance performance to a certain
> > extent and removes ceremony from the code base. I think Parallel streams
> > will add enhancements in many folds as we are using a multicore processor
> > (I have not tested yet), but the Functional approach seems promising.
> >
> > Thanks & Regards,
> > --
> > *Pradhan Yash Sharma*
> > *HotWax Systems* | www.hotwaxsystems.com
> >
> > On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
> > Gareth.Carter@stannah.co.uk> wrote:
> >
> >> forUpdate
> >>
> >> We patched EntityFindOptions with a new field "forUpdate" with shorthand
> >> methods in EntityQuery to enable. We then made a change to
> >> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end
> >> of the SQL select statement - this allows for DB row locks (we use
> postgres
> >> and works but have not tested other databases). I believe there may have
> >> been a discussion about this before
> >>
> >>
> >> forEach on EntityQuery
> >>
> >> Use Consumer in java and groovy to iterate over a query. This can reduce
> >> memory consumption (replacement for queryList()) and boiler plate code
> (eg
> >> queryIterator(), while loop and close)
> >>
> >> Example:
> >>
> >> EntityQuery.use(delegator).from("Foobar")
> >> .forEach(item ->
> >>     Debug.logInfo(item.toString(), module);
> >> );
> >>
> >> A further update could be to provide stream capabilities
> >>
> >> Hope this helps, I can provide a patch aswell
> >>
> >> Gareth Carter
> >> Software Development Analyst
> >> Stannah Management Services Ltd
> >> IT Department
> >> Ext:
> >> 7036
> >> DDI:
> >> 01264 364311
> >>
> >>
> >> Please consider the environment before printing this email.
> >>
> >> -----Original Message-----
> >> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
> >> Sent: 25 October 2017 5:49 AM
> >> To: dev@ofbiz.apache.org
> >> Cc: arunpatidar@apache.org
> >> Subject: Re: Updates to EntityQuery
> >>
> >> Hello Gareth,
> >>
> >> Please provide some more details or patch to understand  - forUpdate and
> >> forEach method utility.
> >>
> >> getFieldMap method looks good to me.
> >>
> >>
> >>
> >> --
> >> Thanks & Regards
> >> ---
> >> Arun Patidar
> >> Manager, Enterprise Software Development
> >>
> >> HotWax Systems Pvt Ltd.
> >>
> >> www.hotwaxsystems.com
> >>
> >>
> >> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
> >> Gareth.Carter@stannah.co.uk>
> >> wrote:
> >>
> >> > Hi all
> >> >
> >> > We have internally patched EntityQuery with some additional
> >> > functionality and before I create a Jira was going to see what the
> >> community thinks.
> >> >
> >> > New functionality:
> >> >
> >> > -       New method getFieldMap which returns a Map object of selected
> >> > fields from GenericValue objects, useful for creating cache map
> >> > objects for lookup
> >> >
> >> > -       Support forUpdate
> >> >
> >> > -       forEach method to accept Consumer
> >> >
> >> > We have found these useful and believe the project can benefit, let me
> >> > know what you think
> >> >
> >> > Regards
> >> >
> >> > Gareth Carter
> >> >
> >> >
> >> > Software Development Analyst
> >> >
> >> >
> >> > Stannah Management Services Ltd
> >> >
> >> >
> >> > IT Department
> >> >
> >> >
> >> > Ext:
> >> >
> >> >
> >> > 7036
> >> >
> >> >
> >> > DDI:
> >> >
> >> >
> >> > 01264 364311
> >> >
> >> >
> >> >
> >> >
> >> > [http://logos.stannah.co.uk/stan150.jpg]
> >> >
> >> >
> >> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the
> environment
> >> > before printing this email.
> >> >
> >> >
> >> >
> >> > This email is intended only for the above addressee. It may contain
> >> > privileged information. If you are not the addressee you must not
> >> > copy, distribute, disclose or use any of the information in it. If you
> >> > have received it in error, please delete it and notify the sender.
> >> >
> >> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
> >> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
> >> > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804,
> >> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
> >> > registered No. 1401451, Global Upholstery Solutions Ltd registered No.
> >> 02452728.
> >> >
> >> > All registered offices at Watt Close, East Portway, Andover,
> >> > Hampshire,
> >> > SP10 3SD, England.
> >> >
> >> > All Registered in England and Wales.
> >> >
> >> > This message has been scanned for malware by Websense.
> >> > www.websense.com
> >> >
> >>
> >>
> >> To report this email as spam, please send the original message, complete
> >> with headers to spam@websense.com
> >>
> >
> >
>

Re: Updates to EntityQuery

Posted by Yash Sharma <ya...@hotwaxsystems.com>.
I am all up for it, please let me know if I could help.

Thanks & Regards,
--
*Pradhan Yash Sharma*
*HotWax Systems* | www.hotwaxsystems.com

On Wed, Oct 25, 2017 at 6:23 PM, Yash Sharma <ya...@hotwaxsystems.com>
wrote:

> Yes, Usage of the stream will surely enhance performance to a certain
> extent and removes ceremony from the code base. I think Parallel streams
> will add enhancements in many folds as we are using a multicore processor
> (I have not tested yet), but the Functional approach seems promising.
>
> Thanks & Regards,
> --
> *Pradhan Yash Sharma*
> *HotWax Systems* | www.hotwaxsystems.com
>
> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <
> Gareth.Carter@stannah.co.uk> wrote:
>
>> forUpdate
>>
>> We patched EntityFindOptions with a new field "forUpdate" with shorthand
>> methods in EntityQuery to enable. We then made a change to
>> GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end
>> of the SQL select statement - this allows for DB row locks (we use postgres
>> and works but have not tested other databases). I believe there may have
>> been a discussion about this before
>>
>>
>> forEach on EntityQuery
>>
>> Use Consumer in java and groovy to iterate over a query. This can reduce
>> memory consumption (replacement for queryList()) and boiler plate code (eg
>> queryIterator(), while loop and close)
>>
>> Example:
>>
>> EntityQuery.use(delegator).from("Foobar")
>> .forEach(item ->
>>     Debug.logInfo(item.toString(), module);
>> );
>>
>> A further update could be to provide stream capabilities
>>
>> Hope this helps, I can provide a patch aswell
>>
>> Gareth Carter
>> Software Development Analyst
>> Stannah Management Services Ltd
>> IT Department
>> Ext:
>> 7036
>> DDI:
>> 01264 364311
>>
>>
>> Please consider the environment before printing this email.
>>
>> -----Original Message-----
>> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
>> Sent: 25 October 2017 5:49 AM
>> To: dev@ofbiz.apache.org
>> Cc: arunpatidar@apache.org
>> Subject: Re: Updates to EntityQuery
>>
>> Hello Gareth,
>>
>> Please provide some more details or patch to understand  - forUpdate and
>> forEach method utility.
>>
>> getFieldMap method looks good to me.
>>
>>
>>
>> --
>> Thanks & Regards
>> ---
>> Arun Patidar
>> Manager, Enterprise Software Development
>>
>> HotWax Systems Pvt Ltd.
>>
>> www.hotwaxsystems.com
>>
>>
>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
>> Gareth.Carter@stannah.co.uk>
>> wrote:
>>
>> > Hi all
>> >
>> > We have internally patched EntityQuery with some additional
>> > functionality and before I create a Jira was going to see what the
>> community thinks.
>> >
>> > New functionality:
>> >
>> > -       New method getFieldMap which returns a Map object of selected
>> > fields from GenericValue objects, useful for creating cache map
>> > objects for lookup
>> >
>> > -       Support forUpdate
>> >
>> > -       forEach method to accept Consumer
>> >
>> > We have found these useful and believe the project can benefit, let me
>> > know what you think
>> >
>> > Regards
>> >
>> > Gareth Carter
>> >
>> >
>> > Software Development Analyst
>> >
>> >
>> > Stannah Management Services Ltd
>> >
>> >
>> > IT Department
>> >
>> >
>> > Ext:
>> >
>> >
>> > 7036
>> >
>> >
>> > DDI:
>> >
>> >
>> > 01264 364311
>> >
>> >
>> >
>> >
>> > [http://logos.stannah.co.uk/stan150.jpg]
>> >
>> >
>> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
>> > before printing this email.
>> >
>> >
>> >
>> > This email is intended only for the above addressee. It may contain
>> > privileged information. If you are not the addressee you must not
>> > copy, distribute, disclose or use any of the information in it. If you
>> > have received it in error, please delete it and notify the sender.
>> >
>> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
>> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
>> > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804,
>> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
>> > registered No. 1401451, Global Upholstery Solutions Ltd registered No.
>> 02452728.
>> >
>> > All registered offices at Watt Close, East Portway, Andover,
>> > Hampshire,
>> > SP10 3SD, England.
>> >
>> > All Registered in England and Wales.
>> >
>> > This message has been scanned for malware by Websense.
>> > www.websense.com
>> >
>>
>>
>> To report this email as spam, please send the original message, complete
>> with headers to spam@websense.com
>>
>
>

Re: Updates to EntityQuery

Posted by Jacques Le Roux <ja...@les7arts.com>.
Hi Yash, Gareth,

Not something obvious, but as explained in https://dzone.com/articles/whats-wrong-java-8-part-iii

hoping to *blindly *use Parallel Streams for intensive computation tasks is not a good idea :)

Jacques


Le 25/10/2017 à 14:53, Yash Sharma a écrit :
> Yes, Usage of the stream will surely enhance performance to a certain
> extent and removes ceremony from the code base. I think Parallel streams
> will add enhancements in many folds as we are using a multicore processor
> (I have not tested yet), but the Functional approach seems promising.
>
> Thanks & Regards,
> --
> *Pradhan Yash Sharma*
> *HotWax Systems* | www.hotwaxsystems.com
>
> On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <Ga...@stannah.co.uk>
> wrote:
>
>> forUpdate
>>
>> We patched EntityFindOptions with a new field "forUpdate" with shorthand
>> methods in EntityQuery to enable. We then made a change to GenericDAO.selectListIteratorByCondition
>> to add "FOR UPDATE" on the end of the SQL select statement - this allows
>> for DB row locks (we use postgres and works but have not tested other
>> databases). I believe there may have been a discussion about this before
>>
>>
>> forEach on EntityQuery
>>
>> Use Consumer in java and groovy to iterate over a query. This can reduce
>> memory consumption (replacement for queryList()) and boiler plate code (eg
>> queryIterator(), while loop and close)
>>
>> Example:
>>
>> EntityQuery.use(delegator).from("Foobar")
>> .forEach(item ->
>>      Debug.logInfo(item.toString(), module);
>> );
>>
>> A further update could be to provide stream capabilities
>>
>> Hope this helps, I can provide a patch aswell
>>
>> Gareth Carter
>> Software Development Analyst
>> Stannah Management Services Ltd
>> IT Department
>> Ext:
>> 7036
>> DDI:
>> 01264 364311
>>
>>
>> Please consider the environment before printing this email.
>>
>> -----Original Message-----
>> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
>> Sent: 25 October 2017 5:49 AM
>> To: dev@ofbiz.apache.org
>> Cc: arunpatidar@apache.org
>> Subject: Re: Updates to EntityQuery
>>
>> Hello Gareth,
>>
>> Please provide some more details or patch to understand  - forUpdate and
>> forEach method utility.
>>
>> getFieldMap method looks good to me.
>>
>>
>>
>> --
>> Thanks & Regards
>> ---
>> Arun Patidar
>> Manager, Enterprise Software Development
>>
>> HotWax Systems Pvt Ltd.
>>
>> www.hotwaxsystems.com
>>
>>
>> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
>> Gareth.Carter@stannah.co.uk>
>> wrote:
>>
>>> Hi all
>>>
>>> We have internally patched EntityQuery with some additional
>>> functionality and before I create a Jira was going to see what the
>> community thinks.
>>> New functionality:
>>>
>>> -       New method getFieldMap which returns a Map object of selected
>>> fields from GenericValue objects, useful for creating cache map
>>> objects for lookup
>>>
>>> -       Support forUpdate
>>>
>>> -       forEach method to accept Consumer
>>>
>>> We have found these useful and believe the project can benefit, let me
>>> know what you think
>>>
>>> Regards
>>>
>>> Gareth Carter
>>>
>>>
>>> Software Development Analyst
>>>
>>>
>>> Stannah Management Services Ltd
>>>
>>>
>>> IT Department
>>>
>>>
>>> Ext:
>>>
>>>
>>> 7036
>>>
>>>
>>> DDI:
>>>
>>>
>>> 01264 364311
>>>
>>>
>>>
>>>
>>> [http://logos.stannah.co.uk/stan150.jpg]
>>>
>>>
>>> [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
>>> before printing this email.
>>>
>>>
>>>
>>> This email is intended only for the above addressee. It may contain
>>> privileged information. If you are not the addressee you must not
>>> copy, distribute, disclose or use any of the information in it. If you
>>> have received it in error, please delete it and notify the sender.
>>>
>>> Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
>>> Services Ltd registered No. 2483693, Stannah Lift Services Ltd
>>> registered No. 1189799, Stannah Microlifts Ltd registered No. 964804,
>>> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
>>> registered No. 1401451, Global Upholstery Solutions Ltd registered No.
>> 02452728.
>>> All registered offices at Watt Close, East Portway, Andover,
>>> Hampshire,
>>> SP10 3SD, England.
>>>
>>> All Registered in England and Wales.
>>>
>>> This message has been scanned for malware by Websense.
>>> www.websense.com
>>>
>>
>> To report this email as spam, please send the original message, complete
>> with headers to spam@websense.com
>>

Re: Updates to EntityQuery

Posted by Yash Sharma <ya...@hotwaxsystems.com>.
Yes, Usage of the stream will surely enhance performance to a certain
extent and removes ceremony from the code base. I think Parallel streams
will add enhancements in many folds as we are using a multicore processor
(I have not tested yet), but the Functional approach seems promising.

Thanks & Regards,
--
*Pradhan Yash Sharma*
*HotWax Systems* | www.hotwaxsystems.com

On Wed, Oct 25, 2017 at 5:59 PM, Gareth Carter <Ga...@stannah.co.uk>
wrote:

> forUpdate
>
> We patched EntityFindOptions with a new field "forUpdate" with shorthand
> methods in EntityQuery to enable. We then made a change to GenericDAO.selectListIteratorByCondition
> to add "FOR UPDATE" on the end of the SQL select statement - this allows
> for DB row locks (we use postgres and works but have not tested other
> databases). I believe there may have been a discussion about this before
>
>
> forEach on EntityQuery
>
> Use Consumer in java and groovy to iterate over a query. This can reduce
> memory consumption (replacement for queryList()) and boiler plate code (eg
> queryIterator(), while loop and close)
>
> Example:
>
> EntityQuery.use(delegator).from("Foobar")
> .forEach(item ->
>     Debug.logInfo(item.toString(), module);
> );
>
> A further update could be to provide stream capabilities
>
> Hope this helps, I can provide a patch aswell
>
> Gareth Carter
> Software Development Analyst
> Stannah Management Services Ltd
> IT Department
> Ext:
> 7036
> DDI:
> 01264 364311
>
>
> Please consider the environment before printing this email.
>
> -----Original Message-----
> From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
> Sent: 25 October 2017 5:49 AM
> To: dev@ofbiz.apache.org
> Cc: arunpatidar@apache.org
> Subject: Re: Updates to EntityQuery
>
> Hello Gareth,
>
> Please provide some more details or patch to understand  - forUpdate and
> forEach method utility.
>
> getFieldMap method looks good to me.
>
>
>
> --
> Thanks & Regards
> ---
> Arun Patidar
> Manager, Enterprise Software Development
>
> HotWax Systems Pvt Ltd.
>
> www.hotwaxsystems.com
>
>
> On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <
> Gareth.Carter@stannah.co.uk>
> wrote:
>
> > Hi all
> >
> > We have internally patched EntityQuery with some additional
> > functionality and before I create a Jira was going to see what the
> community thinks.
> >
> > New functionality:
> >
> > -       New method getFieldMap which returns a Map object of selected
> > fields from GenericValue objects, useful for creating cache map
> > objects for lookup
> >
> > -       Support forUpdate
> >
> > -       forEach method to accept Consumer
> >
> > We have found these useful and believe the project can benefit, let me
> > know what you think
> >
> > Regards
> >
> > Gareth Carter
> >
> >
> > Software Development Analyst
> >
> >
> > Stannah Management Services Ltd
> >
> >
> > IT Department
> >
> >
> > Ext:
> >
> >
> > 7036
> >
> >
> > DDI:
> >
> >
> > 01264 364311
> >
> >
> >
> >
> > [http://logos.stannah.co.uk/stan150.jpg]
> >
> >
> > [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
> > before printing this email.
> >
> >
> >
> > This email is intended only for the above addressee. It may contain
> > privileged information. If you are not the addressee you must not
> > copy, distribute, disclose or use any of the information in it. If you
> > have received it in error, please delete it and notify the sender.
> >
> > Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
> > Services Ltd registered No. 2483693, Stannah Lift Services Ltd
> > registered No. 1189799, Stannah Microlifts Ltd registered No. 964804,
> > Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
> > registered No. 1401451, Global Upholstery Solutions Ltd registered No.
> 02452728.
> >
> > All registered offices at Watt Close, East Portway, Andover,
> > Hampshire,
> > SP10 3SD, England.
> >
> > All Registered in England and Wales.
> >
> > This message has been scanned for malware by Websense.
> > www.websense.com
> >
>
>
> To report this email as spam, please send the original message, complete
> with headers to spam@websense.com
>

RE: Updates to EntityQuery

Posted by Gareth Carter <Ga...@stannah.co.uk>.
forUpdate

We patched EntityFindOptions with a new field "forUpdate" with shorthand methods in EntityQuery to enable. We then made a change to GenericDAO.selectListIteratorByCondition to add "FOR UPDATE" on the end of the SQL select statement - this allows for DB row locks (we use postgres and works but have not tested other databases). I believe there may have been a discussion about this before


forEach on EntityQuery

Use Consumer in java and groovy to iterate over a query. This can reduce memory consumption (replacement for queryList()) and boiler plate code (eg queryIterator(), while loop and close)

Example:

EntityQuery.use(delegator).from("Foobar")
.forEach(item ->
    Debug.logInfo(item.toString(), module);
);

A further update could be to provide stream capabilities

Hope this helps, I can provide a patch aswell

Gareth Carter
Software Development Analyst
Stannah Management Services Ltd
IT Department
Ext:
7036
DDI:
01264 364311


Please consider the environment before printing this email.

-----Original Message-----
From: Arun Patidar [mailto:arun.patidar@hotwaxsystems.com]
Sent: 25 October 2017 5:49 AM
To: dev@ofbiz.apache.org
Cc: arunpatidar@apache.org
Subject: Re: Updates to EntityQuery

Hello Gareth,

Please provide some more details or patch to understand  - forUpdate and forEach method utility.

getFieldMap method looks good to me.



--
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development

HotWax Systems Pvt Ltd.

www.hotwaxsystems.com


On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <Ga...@stannah.co.uk>
wrote:

> Hi all
>
> We have internally patched EntityQuery with some additional
> functionality and before I create a Jira was going to see what the community thinks.
>
> New functionality:
>
> -       New method getFieldMap which returns a Map object of selected
> fields from GenericValue objects, useful for creating cache map
> objects for lookup
>
> -       Support forUpdate
>
> -       forEach method to accept Consumer
>
> We have found these useful and believe the project can benefit, let me
> know what you think
>
> Regards
>
> Gareth Carter
>
>
> Software Development Analyst
>
>
> Stannah Management Services Ltd
>
>
> IT Department
>
>
> Ext:
>
>
> 7036
>
>
> DDI:
>
>
> 01264 364311
>
>
>
>
> [http://logos.stannah.co.uk/stan150.jpg]
>
>
> [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
> before printing this email.
>
>
>
> This email is intended only for the above addressee. It may contain
> privileged information. If you are not the addressee you must not
> copy, distribute, disclose or use any of the information in it. If you
> have received it in error, please delete it and notify the sender.
>
> Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
> Services Ltd registered No. 2483693, Stannah Lift Services Ltd
> registered No. 1189799, Stannah Microlifts Ltd registered No. 964804,
> Stannah Lifts Ltd registered No. 1189836, Stannah Stairlifts Ltd
> registered No. 1401451, Global Upholstery Solutions Ltd registered No. 02452728.
>
> All registered offices at Watt Close, East Portway, Andover,
> Hampshire,
> SP10 3SD, England.
>
> All Registered in England and Wales.
>
> This message has been scanned for malware by Websense.
> www.websense.com
>


To report this email as spam, please send the original message, complete with headers to spam@websense.com

Re: Updates to EntityQuery

Posted by Arun Patidar <ar...@hotwaxsystems.com>.
Hello Gareth,

Please provide some more details or patch to understand  - forUpdate
and forEach
method utility.

getFieldMap method looks good to me.



-- 
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development

HotWax Systems Pvt Ltd.

www.hotwaxsystems.com


On Tue, Oct 24, 2017 at 9:06 PM, Gareth Carter <Ga...@stannah.co.uk>
wrote:

> Hi all
>
> We have internally patched EntityQuery with some additional functionality
> and before I create a Jira was going to see what the community thinks.
>
> New functionality:
>
> -       New method getFieldMap which returns a Map object of selected
> fields from GenericValue objects, useful for creating cache map objects for
> lookup
>
> -       Support forUpdate
>
> -       forEach method to accept Consumer
>
> We have found these useful and believe the project can benefit, let me
> know what you think
>
> Regards
>
> Gareth Carter
>
>
> Software Development Analyst
>
>
> Stannah Management Services Ltd
>
>
> IT Department
>
>
> Ext:
>
>
> 7036
>
>
> DDI:
>
>
> 01264 364311
>
>
>
>
> [http://logos.stannah.co.uk/stan150.jpg]
>
>
> [http://logos.stannah.co.uk/enviro.jpg]Please consider the environment
> before printing this email.
>
>
>
> This email is intended only for the above addressee. It may contain
> privileged information. If you are not the addressee you must not copy,
> distribute, disclose or use any of the information in it. If you have
> received it in error, please delete it and notify the sender.
>
> Stannah Lift Holdings Ltd registered No. 686996, Stannah Management
> Services Ltd registered No. 2483693, Stannah Lift Services Ltd registered
> No. 1189799, Stannah Microlifts Ltd registered No. 964804, Stannah Lifts
> Ltd registered No. 1189836, Stannah Stairlifts Ltd registered No. 1401451,
> Global Upholstery Solutions Ltd registered No. 02452728.
>
> All registered offices at Watt Close, East Portway, Andover, Hampshire,
> SP10 3SD, England.
>
> All Registered in England and Wales.
>
> This message has been scanned for malware by Websense. www.websense.com
>