You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Denis DEBARBIEUX <dd...@norsys.fr> on 2015/07/27 14:02:33 UTC
How to synchronize worker (prepare method)
Dear all,
I am writting a bolt. It is in charge of saving information in some
database (cassandra). In its prepare method, I am creating the table
(CQL query like CREATE TABLE IF NOT EXISTS ...).
When I run my topology with many workers, I got a concurrent access
exception. Indeed, all my workers are creating the same table at the
same time. How to avoid this?
My first idea is to elect one worker (e.g. base on the taskId) that will
create the table. Other worker will wait until the job is done. Is it a
good design?
Thanks for your help.
Denis
---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus
Re: How to synchronize worker (prepare method)
Posted by Denis DEBARBIEUX <dd...@norsys.fr>.
That is a great idea.
Thanks.
Denis
Le 27/07/2015 16:54, John Reilly a écrit :
> Another approach you could take is to create the table at the time
> that you create the topology rather than have the topology create it.
> On Mon, Jul 27, 2015 at 05:41 Nathan Leung <ncleung@gmail.com
> <ma...@gmail.com>> wrote:
>
> Using task id is a good idea. I don't recall if 0 or 1 is the
> lowest, but it's probably a good to use the lowest possible one
> for intialization, just in case you ever run a topology with only
> 1 bolt.
>
> On Mon, Jul 27, 2015 at 8:02 AM, Denis DEBARBIEUX
> <ddebarbieux@norsys.fr <ma...@norsys.fr>> wrote:
>
> Dear all,
>
> I am writting a bolt. It is in charge of saving information in
> some database (cassandra). In its prepare method, I am
> creating the table (CQL query like CREATE TABLE IF NOT EXISTS
> ...).
>
> When I run my topology with many workers, I got a concurrent
> access exception. Indeed, all my workers are creating the same
> table at the same time. How to avoid this?
>
> My first idea is to elect one worker (e.g. base on the taskId)
> that will create the table. Other worker will wait until the
> job is done. Is it a good design?
>
> Thanks for your help.
>
> Denis
>
> ---
> L'absence de virus dans ce courrier électronique a été
> vérifiée par le logiciel antivirus Avast.
> https://www.avast.com/antivirus
>
>
---
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus
Re: How to synchronize worker (prepare method)
Posted by John Reilly <jr...@inconspicuous.org>.
Another approach you could take is to create the table at the time that you
create the topology rather than have the topology create it.
On Mon, Jul 27, 2015 at 05:41 Nathan Leung <nc...@gmail.com> wrote:
> Using task id is a good idea. I don't recall if 0 or 1 is the lowest, but
> it's probably a good to use the lowest possible one for intialization, just
> in case you ever run a topology with only 1 bolt.
>
> On Mon, Jul 27, 2015 at 8:02 AM, Denis DEBARBIEUX <dd...@norsys.fr>
> wrote:
>
>> Dear all,
>>
>> I am writting a bolt. It is in charge of saving information in some
>> database (cassandra). In its prepare method, I am creating the table (CQL
>> query like CREATE TABLE IF NOT EXISTS ...).
>>
>> When I run my topology with many workers, I got a concurrent access
>> exception. Indeed, all my workers are creating the same table at the same
>> time. How to avoid this?
>>
>> My first idea is to elect one worker (e.g. base on the taskId) that will
>> create the table. Other worker will wait until the job is done. Is it a
>> good design?
>>
>> Thanks for your help.
>>
>> Denis
>>
>> ---
>> L'absence de virus dans ce courrier électronique a été vérifiée par le
>> logiciel antivirus Avast.
>> https://www.avast.com/antivirus
>>
>>
>
Re: How to synchronize worker (prepare method)
Posted by Nathan Leung <nc...@gmail.com>.
Using task id is a good idea. I don't recall if 0 or 1 is the lowest, but
it's probably a good to use the lowest possible one for intialization, just
in case you ever run a topology with only 1 bolt.
On Mon, Jul 27, 2015 at 8:02 AM, Denis DEBARBIEUX <dd...@norsys.fr>
wrote:
> Dear all,
>
> I am writting a bolt. It is in charge of saving information in some
> database (cassandra). In its prepare method, I am creating the table (CQL
> query like CREATE TABLE IF NOT EXISTS ...).
>
> When I run my topology with many workers, I got a concurrent access
> exception. Indeed, all my workers are creating the same table at the same
> time. How to avoid this?
>
> My first idea is to elect one worker (e.g. base on the taskId) that will
> create the table. Other worker will wait until the job is done. Is it a
> good design?
>
> Thanks for your help.
>
> Denis
>
> ---
> L'absence de virus dans ce courrier électronique a été vérifiée par le
> logiciel antivirus Avast.
> https://www.avast.com/antivirus
>
>