DataTorrent application Parllel processing

["Bandaru, Srinivas" <sr...@optum.com>]

Hi, Need some help. How we can achieve parallelism for data torrent application (running with maprstresms) for benchmarking?

Thanks,
Srinivas Bandaru


This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.

Re: DataTorrent application Parllel processing

[Thomas Weise <th...@apache.org>]

There is some draft documentation for 0.9 here:

https://github.com/chaithu14/incubator-apex-malhar/blob/APEXMALHAR-2242-kafka0.9doc/docs/operators/kafkaInputOperator.md#09-version-of-kafka-input-operator


On Tue, Oct 11, 2016 at 3:21 PM, Thomas Weise <th...@gmail.com>
wrote:

> Hi,
>
> There is general information on partitioning here:
>
> http://apex.apache.org/docs/apex/application_development/#partitioning
>
> and a more recent presentation here:
>
> http://www.slideshare.net/ApacheApex/smart-partitioning-
> with-apache-apex-webinar
>
> For MapR streams you are using the 0.9 operator, and you can set the
> partitioning strategy on it:
>
> https://github.com/apache/apex-malhar/blob/master/kafka/
> src/main/java/org/apache/apex/malhar/kafka/PartitionStrategy.java
>
> Thanks,
> Thomas
>
>
>
> On Tue, Oct 11, 2016 at 11:30 AM, Bandaru, Srinivas <
> srinivas.bandaru@optum.com> wrote:
>
>> Hi, Need some help. How we can achieve parallelism for data torrent
>> application (running with maprstresms) for benchmarking?
>>
>>
>>
>> Thanks,
>>
>> *Srinivas Bandaru*
>>
>>
>>
>>
>> This e-mail, including attachments, may include confidential and/or
>> proprietary information, and may be used only by the person or entity
>> to which it is addressed. If the reader of this e-mail is not the intended
>> recipient or his or her authorized agent, the reader is hereby notified
>> that any dissemination, distribution or copying of this e-mail is
>> prohibited. If you have received this e-mail in error, please notify the
>> sender by replying to this message and delete this e-mail immediately.
>>
>
>

Re: DataTorrent application Parllel processing

[Thomas Weise <th...@gmail.com>]

Hi,

There is general information on partitioning here:

http://apex.apache.org/docs/apex/application_development/#partitioning

and a more recent presentation here:

http://www.slideshare.net/ApacheApex/smart-partitioning-with-apache-apex-webinar

For MapR streams you are using the 0.9 operator, and you can set the
partitioning strategy on it:

https://github.com/apache/apex-malhar/blob/master/kafka/src/main/java/org/apache/apex/malhar/kafka/PartitionStrategy.java

Thanks,
Thomas



On Tue, Oct 11, 2016 at 11:30 AM, Bandaru, Srinivas <
srinivas.bandaru@optum.com> wrote:

> Hi, Need some help. How we can achieve parallelism for data torrent
> application (running with maprstresms) for benchmarking?
>
>
>
> Thanks,
>
> *Srinivas Bandaru*
>
>
>
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity
> to which it is addressed. If the reader of this e-mail is not the intended
> recipient or his or her authorized agent, the reader is hereby notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify the
> sender by replying to this message and delete this e-mail immediately.
>