GPARS.withPool, xml.MarkupBuilder, XmlParser and HTTPBuilder

[Paolo <qo...@gmx.de>]

Hi all,

 

I am pretty new to Groovy and after playing around for a little while I decided to use Groovy for my next little challenge … Basically I want to process a huge list of files (in parallel) using  GPARS.withPool. Each file needs to be  parsed and converted into a XML file using MarkupBuilder. After the XML file is created I would like to reread it with the XmlParser and upload the original file and the corresponding XML file to a webservice using HTTPBuilder.

 

So basically I have a closure called fileworker that will process a file

 

def fileworker = {

  open file

  parse file, adjust some values and create XML file using MarkupBuilder

  parse the created XML file using XmlParser

  upload file and XML file using HTTPBuilder

}

 

and this closure will be used for processing the list in parallel (collectParallel)

 

groovyx.gpars.GParsPool.withPool(threads) {

                result = list.collectParallel fileworker

}

 

Due to my missing experiences I am a little concerned about thread safety when GPARS comes into place …. I have a working implemententation of all steps described above to run sequencially … but now when deploying GPARS, is there anything special that needs to be considered concerning the usage of MarkupBuilder, XmlParser, HTTPBuilder in that closure? I would appreciate if someone could share his experience and opinion …

 

Thanks in advance

Paolo

Re: GPARS.withPool, xml.MarkupBuilder, XmlParser and HTTPBuilder

[Jochen Theodorou <bl...@gmx.org>]

On 16.06.2016 09:40, Paolo wrote:
[...]
> So basically I have a closure called fileworker that will process a file
>
> def fileworker = {
>    open file
>    parse file, adjust some values and create XML file using MarkupBuilder
>    parse the created XML file using XmlParser
>    upload file and XML file using HTTPBuilder
> }
>
> and this closure will be used for processing the list in parallel
> (collectParallel)
>
> groovyx.gpars.GParsPool.withPool(threads) {
>                 result = list.collectParallel fileworker
> }
>
> Due to my missing experiences I am a little concerned about thread
> safety when GPARS comes into place \u2026.

fileworker is self-contained from what I see, so you do not have to 
worry about concurrency issues in this case. You have to worry about 
those once you have data shared by multiple threads.

bye Jochen