You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by Naresh Guntupalli <na...@gmail.com> on 2016/09/13 00:13:56 UTC
HDHT operator
The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
Apex-Malhar 3.x releases, but the user docs still references it. Was the
operator removed or moved somewhere else?
Re: HDHT operator
Posted by Bhupesh Chawda <bh...@datatorrent.com>.
Hi Naresh,
You can also have a look at the managed state which is also a persistent
bucketing system supporting asynchronous reads.
https://github.com/apache/apex-malhar/tree/master/library/src/main/java/org/apache/apex/malhar/lib/state/managed
See
https://github.com/apache/apex-malhar/blob/master/library/src/main/java/org/apache/apex/malhar/lib/dedup/AbstractDeduper.java
to see how to use it.
~ Bhupesh
On Tue, Sep 13, 2016 at 7:41 AM, Naresh Guntupalli <
naresh.guntupalli@gmail.com> wrote:
> Ok, got it. No, the events are enriched based on the stored state
> information and emitted (per tuple).
>
> On Mon, Sep 12, 2016 at 6:47 PM, Thomas Weise <th...@gmail.com>
> wrote:
>
>> That documentation is not for Malhar. You find the user documentation for
>> Apex here:
>>
>> http://apex.apache.org/docs.html
>>
>> See links on top.
>>
>> So you want to enrich the data after it was emitted from the window?
>>
>>
>>
>>
>> On Mon, Sep 12, 2016 at 6:41 PM, Naresh Guntupalli <
>> naresh.guntupalli@gmail.com> wrote:
>>
>>> Thanks Ashwin and Thomas for your quick help.
>>>
>>> @Thomas: http://docs.datatorrent.com/operators/hdht/ - This is the doc
>>> I was referring to earlier.
>>>
>>> The use case is store and enrich certain user events in a (bigger)
>>> rolling window.
>>>
>>> -Naresh
>>>
>>> On Mon, Sep 12, 2016 at 5:43 PM, Thomas Weise <th...@gmail.com>
>>> wrote:
>>>
>>>> Here is an example for the equivalent functionality in Malhar:
>>>>
>>>> https://github.com/apache/apex-malhar/blob/master/benchmark/
>>>> src/main/java/com/datatorrent/benchmark/state/StoreOperator.java
>>>>
>>>> https://issues.apache.org/jira/browse/APEXMALHAR-2205
>>>>
>>>> It will still be useful to understand the use case as there are also a
>>>> couple ready to use operators that use this state management facility.
>>>>
>>>> Thomas
>>>>
>>>>
>>>> On Mon, Sep 12, 2016 at 5:34 PM, Thomas Weise <th...@gmail.com>
>>>> wrote:
>>>>
>>>>> Naresh,
>>>>>
>>>>> Which doc references this operator? Malhar has a replacement for this.
>>>>> Can you share some more info about your use case so that we can point you
>>>>> to the appropriate operator to start from?
>>>>>
>>>>> Thanks,
>>>>> Thomas
>>>>>
>>>>>
>>>>> On Mon, Sep 12, 2016 at 5:13 PM, Naresh Guntupalli <
>>>>> naresh.guntupalli@gmail.com> wrote:
>>>>>
>>>>>> The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
>>>>>> Apex-Malhar 3.x releases, but the user docs still references it. Was the
>>>>>> operator removed or moved somewhere else?
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: HDHT operator
Posted by Naresh Guntupalli <na...@gmail.com>.
Ok, got it. No, the events are enriched based on the stored state
information and emitted (per tuple).
On Mon, Sep 12, 2016 at 6:47 PM, Thomas Weise <th...@gmail.com>
wrote:
> That documentation is not for Malhar. You find the user documentation for
> Apex here:
>
> http://apex.apache.org/docs.html
>
> See links on top.
>
> So you want to enrich the data after it was emitted from the window?
>
>
>
>
> On Mon, Sep 12, 2016 at 6:41 PM, Naresh Guntupalli <
> naresh.guntupalli@gmail.com> wrote:
>
>> Thanks Ashwin and Thomas for your quick help.
>>
>> @Thomas: http://docs.datatorrent.com/operators/hdht/ - This is the doc
>> I was referring to earlier.
>>
>> The use case is store and enrich certain user events in a (bigger)
>> rolling window.
>>
>> -Naresh
>>
>> On Mon, Sep 12, 2016 at 5:43 PM, Thomas Weise <th...@gmail.com>
>> wrote:
>>
>>> Here is an example for the equivalent functionality in Malhar:
>>>
>>> https://github.com/apache/apex-malhar/blob/master/benchmark/
>>> src/main/java/com/datatorrent/benchmark/state/StoreOperator.java
>>>
>>> https://issues.apache.org/jira/browse/APEXMALHAR-2205
>>>
>>> It will still be useful to understand the use case as there are also a
>>> couple ready to use operators that use this state management facility.
>>>
>>> Thomas
>>>
>>>
>>> On Mon, Sep 12, 2016 at 5:34 PM, Thomas Weise <th...@gmail.com>
>>> wrote:
>>>
>>>> Naresh,
>>>>
>>>> Which doc references this operator? Malhar has a replacement for this.
>>>> Can you share some more info about your use case so that we can point you
>>>> to the appropriate operator to start from?
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>>
>>>> On Mon, Sep 12, 2016 at 5:13 PM, Naresh Guntupalli <
>>>> naresh.guntupalli@gmail.com> wrote:
>>>>
>>>>> The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
>>>>> Apex-Malhar 3.x releases, but the user docs still references it. Was the
>>>>> operator removed or moved somewhere else?
>>>>>
>>>>
>>>>
>>>
>>
>
Re: HDHT operator
Posted by Thomas Weise <th...@gmail.com>.
That documentation is not for Malhar. You find the user documentation for
Apex here:
http://apex.apache.org/docs.html
See links on top.
So you want to enrich the data after it was emitted from the window?
On Mon, Sep 12, 2016 at 6:41 PM, Naresh Guntupalli <
naresh.guntupalli@gmail.com> wrote:
> Thanks Ashwin and Thomas for your quick help.
>
> @Thomas: http://docs.datatorrent.com/operators/hdht/ - This is the doc I
> was referring to earlier.
>
> The use case is store and enrich certain user events in a (bigger) rolling
> window.
>
> -Naresh
>
> On Mon, Sep 12, 2016 at 5:43 PM, Thomas Weise <th...@gmail.com>
> wrote:
>
>> Here is an example for the equivalent functionality in Malhar:
>>
>> https://github.com/apache/apex-malhar/blob/master/benchmark/
>> src/main/java/com/datatorrent/benchmark/state/StoreOperator.java
>>
>> https://issues.apache.org/jira/browse/APEXMALHAR-2205
>>
>> It will still be useful to understand the use case as there are also a
>> couple ready to use operators that use this state management facility.
>>
>> Thomas
>>
>>
>> On Mon, Sep 12, 2016 at 5:34 PM, Thomas Weise <th...@gmail.com>
>> wrote:
>>
>>> Naresh,
>>>
>>> Which doc references this operator? Malhar has a replacement for this.
>>> Can you share some more info about your use case so that we can point you
>>> to the appropriate operator to start from?
>>>
>>> Thanks,
>>> Thomas
>>>
>>>
>>> On Mon, Sep 12, 2016 at 5:13 PM, Naresh Guntupalli <
>>> naresh.guntupalli@gmail.com> wrote:
>>>
>>>> The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
>>>> Apex-Malhar 3.x releases, but the user docs still references it. Was the
>>>> operator removed or moved somewhere else?
>>>>
>>>
>>>
>>
>
Re: HDHT operator
Posted by Naresh Guntupalli <na...@gmail.com>.
Thanks Ashwin and Thomas for your quick help.
@Thomas: http://docs.datatorrent.com/operators/hdht/ - This is the doc I
was referring to earlier.
The use case is store and enrich certain user events in a (bigger) rolling
window.
-Naresh
On Mon, Sep 12, 2016 at 5:43 PM, Thomas Weise <th...@gmail.com>
wrote:
> Here is an example for the equivalent functionality in Malhar:
>
> https://github.com/apache/apex-malhar/blob/master/
> benchmark/src/main/java/com/datatorrent/benchmark/state/StoreOperator.java
>
> https://issues.apache.org/jira/browse/APEXMALHAR-2205
>
> It will still be useful to understand the use case as there are also a
> couple ready to use operators that use this state management facility.
>
> Thomas
>
>
> On Mon, Sep 12, 2016 at 5:34 PM, Thomas Weise <th...@gmail.com>
> wrote:
>
>> Naresh,
>>
>> Which doc references this operator? Malhar has a replacement for this.
>> Can you share some more info about your use case so that we can point you
>> to the appropriate operator to start from?
>>
>> Thanks,
>> Thomas
>>
>>
>> On Mon, Sep 12, 2016 at 5:13 PM, Naresh Guntupalli <
>> naresh.guntupalli@gmail.com> wrote:
>>
>>> The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
>>> Apex-Malhar 3.x releases, but the user docs still references it. Was the
>>> operator removed or moved somewhere else?
>>>
>>
>>
>
Re: HDHT operator
Posted by Thomas Weise <th...@gmail.com>.
Here is an example for the equivalent functionality in Malhar:
https://github.com/apache/apex-malhar/blob/master/benchmark/src/main/java/com/datatorrent/benchmark/state/StoreOperator.java
https://issues.apache.org/jira/browse/APEXMALHAR-2205
It will still be useful to understand the use case as there are also a
couple ready to use operators that use this state management facility.
Thomas
On Mon, Sep 12, 2016 at 5:34 PM, Thomas Weise <th...@gmail.com>
wrote:
> Naresh,
>
> Which doc references this operator? Malhar has a replacement for this. Can
> you share some more info about your use case so that we can point you to
> the appropriate operator to start from?
>
> Thanks,
> Thomas
>
>
> On Mon, Sep 12, 2016 at 5:13 PM, Naresh Guntupalli <
> naresh.guntupalli@gmail.com> wrote:
>
>> The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
>> Apex-Malhar 3.x releases, but the user docs still references it. Was the
>> operator removed or moved somewhere else?
>>
>
>
Re: HDHT operator
Posted by Thomas Weise <th...@gmail.com>.
Naresh,
Which doc references this operator? Malhar has a replacement for this. Can
you share some more info about your use case so that we can point you to
the appropriate operator to start from?
Thanks,
Thomas
On Mon, Sep 12, 2016 at 5:13 PM, Naresh Guntupalli <
naresh.guntupalli@gmail.com> wrote:
> The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
> Apex-Malhar 3.x releases, but the user docs still references it. Was the
> operator removed or moved somewhere else?
>
Re: HDHT operator
Posted by Ashwin Chandra Putta <as...@gmail.com>.
Naresh,
Use it from the following repository for now. The Apex community is working
on moving classes from this repository to apex malhar soon.
https://github.com/DataTorrent/Megh/blob/master/contrib/src/main/java/com/datatorrent/contrib/hdht/AbstractSinglePortHDHTWriter.java
Regards,
Ashwin.
On Mon, Sep 12, 2016 at 5:13 PM, Naresh Guntupalli <
naresh.guntupalli@gmail.com> wrote:
> The AbstractSinglePortHDHTWriter (HDHT) operator is missing from
> Apex-Malhar 3.x releases, but the user docs still references it. Was the
> operator removed or moved somewhere else?
>
--
Regards,
Ashwin.