You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ofbiz.apache.org by "Robert G." <ga...@mercon24.de> on 2013/05/21 18:19:29 UTC
Groovy call a service within a while loop (wait, notify?)...
hey guys, I have the following question (maybe groovy basics)...:
I do read a csv file with opencsv, so I get it line by line within a while
loop. Within this loop I call a simple method service to store selected data
from each line in database. With that I thought not to overload my memory,
because only one line of Csv should be in memory. But now I realized, that I
get a transaction timeout error from my service. It also does not store
anything in database (with just 100 entries it works fine, but not wiht
10.000 entries and more). The transaction is rolled back. So I believe that
it tries to use just one transaction for all data? Could that maybe happen
because the while loop does not wait for finishing the transaction and just
push's more data to it? So if is so, how could I wait for the end of the
service before taking the next line from my csv (next while loop).
my loop is like that:
/...some code....
while ((nextLine = reader.readNext()) != null) {
dispatcher.runSync("exampleService", [userLogin: userLogin, var1:var1,
var2:var2]);
}
...some code..../
any help I appreciate!
greetings, robert
--
View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435.html
Sent from the OFBiz - User mailing list archive at Nabble.com.
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Robert G." <ga...@mercon24.de>
> yes exactly what I thought, the timout needs to be suspended too. So it works
> good. What I am wondering why its not possible to have multiple transactions
> paralell. If my hardware recources and maybe my internet connection are good
> enough I could handle thousands of transactions, so the time all together
> would shrink to a minimum (with high ressource ussage).
>
> With the require-new-transaction="true" I get the results at the end, but
> all transactions will be handled in a row, what in my opinion is ressource
> friendly, but takes long.
Then, if loop steps don't depend on each other, call the service asynchronously rather than synchronously.
Depending of the security you want for each step to be completed, you would want to make these async calls persistent (jobs) or not.
To follow jobs use /webtools/control/FindJob
HTH
Jacques
> btw. also thanks Deepak, your hints helped me a lot to use Groovy things as
> jobs, great!
>
>
>
> --
> View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435p4641451.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by "Robert G." <ga...@mercon24.de>.
yes exactly what I thought, the timout needs to be suspended too. So it works
good. What I am wondering why its not possible to have multiple transactions
paralell. If my hardware recources and maybe my internet connection are good
enough I could handle thousands of transactions, so the time all together
would shrink to a minimum (with high ressource ussage).
With the require-new-transaction="true" I get the results at the end, but
all transactions will be handled in a row, what in my opinion is ressource
friendly, but takes long.
btw. also thanks Deepak, your hints helped me a lot to use Groovy things as
jobs, great!
--
View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435p4641451.html
Sent from the OFBiz - User mailing list archive at Nabble.com.
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Robert G." <ga...@mercon24.de>
> ok now I used require-new-transaction="true" within the service declaration
> in services.xml. This seems to work. It imports all data (all rows are
> proceeded).
>
> "Attribute : require-new-transaction
> If set to true and there is a transaction already in place the Service
> Engine will suspend that transaction,
> begin a new one just for this service, commit or rollback the local
> transaction when the service is
> complete, and will resume the original transaction. If set to true and
> there is no transaction already in
> place it will just begin a transaction and manage it as would be done for a
> normal user-transaction=true.
> If use-transaction=false this setting is ignored."
>
> I understand it that it pauses an active transaction, when a new one
> arrives, and resume it after the new one is ready, right?
Yes, by using require-new-transaction="true" you suspend the (surrouding) transaction in which your Groovy file is (I guess you have one "automatically", normally you should), and create a new one for your called service. Else the current (surrouding) transaction, in which your groovy file was called, will be used by the called service. By using require-new-transaction="true", the suspended (surrouding) transaction is then resumed when the called service finishes. So it seems from your result (I never tested that) the suspended transaction see also its timeout "suspended". Which is why it would work for you. Interesting if true (I did not check the code), I see only this explanation
Note that by default the time out of a transaction is 60 seconds (defined in the Transaction Manager). By using require-new-transaction="true" you can set a higher (or different at large) time out for a called service if necesssary.
require-new-transaction="true" is only necessary for a service timeout to work if you are inside a previous transaction.
Another solution would have been to set a very high timeout on the action which is the origin (a request event I guess). But AFAIK this would work only if the surrounding call is a service. Because events (w/o service call) in requests use a transaction but there are no ways to set a timeout. So the default 60 sec would apply. If your event use a service, then another solution would have been to set a high timeout without using require-new-transaction="true". Because this is only needed for services called inside a service. This near what suggested Adrian, to set the surrounding call in which you groovy file is, to not use a transaction, hence no timeout. But again this would only work if the event which eventually calls the service in the loop is itself a service. And, by and large,it's always better to set a timeout purposedly thantnot using a transaction.
Last but not least there is a specific trap, see https://issues.apache.org/jira/browse/OFBIZ-5183, which remembers me to do the point 1!
HTH
Jacques
> I think this is fine so, what do you think?
>
>
>
> --
> View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435p4641441.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by "Robert G." <ga...@mercon24.de>.
ok now I used require-new-transaction="true" within the service declaration
in services.xml. This seems to work. It imports all data (all rows are
proceeded).
"Attribute : require-new-transaction
If set to true and there is a transaction already in place the Service
Engine will suspend that transaction,
begin a new one just for this service, commit or rollback the local
transaction when the service is
complete, and will resume the original transaction. If set to true and
there is no transaction already in
place it will just begin a transaction and manage it as would be done for a
normal user-transaction=true.
If use-transaction=false this setting is ignored."
I understand it that it pauses an active transaction, when a new one
arrives, and resume it after the new one is ready, right?
I think this is fine so, what do you think?
--
View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435p4641441.html
Sent from the OFBiz - User mailing list archive at Nabble.com.
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by "Robert G." <ga...@mercon24.de>.
Hey david, also thank you for the fast answer.
How can I handle that with my situation?
I heard from you about --> use-transaction="false"
and from another thread sth. like: --> require-new-transaction=true.
so I understand it like my situation uses one single transaction what in my
eyes is not that good concerning memory.
my situation is:
a screen includes a submit button to call a ftl with groovy logic. The
groovy file calls the opencsv classes in java and gives back an iterator for
the array (each line is one array, ittereated with the while loop). within
this loop (groovy) I call the service to store the data in database
(dispatcher.runSync("createPlSummary", [someData:someData])), service is
written in xml. so I think here i sould say sth like
require-new-transaction, but where do i put that? And is this the right way?
btw. the transaction stops after 180seconds (measured again :) )…
--
View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435p4641440.html
Sent from the OFBiz - User mailing list archive at Nabble.com.
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by Deepak Agarwal <da...@gmail.com>.
Yes, you are right, scheduled service is a job. Considering Adrian's
suggestion you should keep a low transaction timeout for exampleService and
use-transaction="false" for the groovy running this service.
Groovy files can be used as ofbiz service. You need to prepare a map and
put your OUT param in it to return, that's it (you can also use
ServiceUtil.returnSuccess/error methods).
On Tue, May 21, 2013 at 10:09 PM, Robert G. <ga...@mercon24.de> wrote:
> ha, that was fast :)
>
> I see and this sounds reasonable. But on the otherhand I can change the
> timeout for the browser, and 10.000 lines is not that much in my opinion.
> The transaction stops after 2.5 minutes.
>
> Or do you have a hint, how I can change the groovy code to a job? btw. what
> is the difference between a job and a service, is the job a scheduled
> service?
>
> I integrated that import into the webtools and start it from there, so I
> did
> it with groovy and minilang for the database service (Create...).
>
>
>
>
>
> --
> View this message in context:
> http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435p4641438.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.
>
--
Thanks,
Deepak Agarwal,
Mobile: +91 9501190044
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by "Robert G." <ga...@mercon24.de>.
ha, that was fast :)
I see and this sounds reasonable. But on the otherhand I can change the
timeout for the browser, and 10.000 lines is not that much in my opinion.
The transaction stops after 2.5 minutes.
Or do you have a hint, how I can change the groovy code to a job? btw. what
is the difference between a job and a service, is the job a scheduled
service?
I integrated that import into the webtools and start it from there, so I did
it with groovy and minilang for the database service (Create...).
--
View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435p4641438.html
Sent from the OFBiz - User mailing list archive at Nabble.com.
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by Deepak Agarwal <da...@gmail.com>.
In my experience there is a reason why most of the erp provide such long
tasks as jobs. Because, if it is a long task then it is better to do in
separate async jobs (else you might get a browser timeout ).
My suggestion is to make this as a service with a long timeout and run it
as a job.
On Tue, May 21, 2013 at 9:49 PM, Robert G. <ga...@mercon24.de> wrote:
> hey guys, I have the following question (maybe groovy basics)...:
>
> I do read a csv file with opencsv, so I get it line by line within a while
> loop. Within this loop I call a simple method service to store selected
> data
> from each line in database. With that I thought not to overload my memory,
> because only one line of Csv should be in memory. But now I realized, that
> I
> get a transaction timeout error from my service. It also does not store
> anything in database (with just 100 entries it works fine, but not wiht
> 10.000 entries and more). The transaction is rolled back. So I believe that
> it tries to use just one transaction for all data? Could that maybe happen
> because the while loop does not wait for finishing the transaction and just
> push's more data to it? So if is so, how could I wait for the end of the
> service before taking the next line from my csv (next while loop).
>
> my loop is like that:
>
>
> /...some code....
>
> while ((nextLine = reader.readNext()) != null) {
> dispatcher.runSync("exampleService", [userLogin:
> userLogin, var1:var1,
> var2:var2]);
> }
> ...some code..../
>
> any help I appreciate!
>
> greetings, robert
>
>
>
> --
> View this message in context:
> http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.
>
--
Thanks,
Deepak Agarwal,
Mobile: +91 9501190044
Re: Groovy call a service within a while loop (wait, notify?)...
Posted by Adrian Crum <ad...@sandglass-software.com>.
Set use-transaction="false" in the service that contains the loop, and
make sure the service to store the CSV record uses a transaction.
-Adrian
On 5/21/2013 5:19 PM, Robert G. wrote:
> hey guys, I have the following question (maybe groovy basics)...:
>
> I do read a csv file with opencsv, so I get it line by line within a while
> loop. Within this loop I call a simple method service to store selected data
> from each line in database. With that I thought not to overload my memory,
> because only one line of Csv should be in memory. But now I realized, that I
> get a transaction timeout error from my service. It also does not store
> anything in database (with just 100 entries it works fine, but not wiht
> 10.000 entries and more). The transaction is rolled back. So I believe that
> it tries to use just one transaction for all data? Could that maybe happen
> because the while loop does not wait for finishing the transaction and just
> push's more data to it? So if is so, how could I wait for the end of the
> service before taking the next line from my csv (next while loop).
>
> my loop is like that:
>
>
> /...some code....
>
> while ((nextLine = reader.readNext()) != null) {
> dispatcher.runSync("exampleService", [userLogin: userLogin, var1:var1,
> var2:var2]);
> }
> ...some code..../
>
> any help I appreciate!
>
> greetings, robert
>
>
>
> --
> View this message in context: http://ofbiz.135035.n4.nabble.com/Groovy-call-a-service-within-a-while-loop-wait-notify-tp4641435.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.