You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Meghashyam Sandeep V <vr...@gmail.com> on 2017/03/13 17:25:01 UTC
Suggestion for top 'k' products
Hi All,
I'm trying to use Flink for a use case where I would want to see my top
selling products in time windows in near real time (windows of size 1-2
mins if fine). I guess this is the most common use case to use streaming
apis in e-commerce. I see that I can iterate over records in a windowed
stream and do the sorting myself. I'm wondering if thats the best way. Is
there any in built sort functionality that I missed anywhere in Flink docs?
Thanks,
Sandeep
Re: Suggestion for top 'k' products
Posted by Meghashyam Sandeep V <vr...@gmail.com>.
Is there an equivalent of spark function like 'takeOrdered' in Flink? If I
implement a function to order messages in the stream, I'm not sure if thats
executed in a distributed mode by splitting data into available task
manager nodes and then evaluate the function.
Thanks,
Sandeep
On Mon, Mar 13, 2017 at 11:22 AM, Meghashyam Sandeep V <
vr1meghashyam@gmail.com> wrote:
> Thanks Suneel. Exactly what I was looking for.
>
> On Mon, Mar 13, 2017 at 10:31 AM, Suneel Marthi <sm...@apache.org>
> wrote:
>
>> For an example implementation using Flink, check out
>> https://github.com/bigpetstore/bigpetstore-flink/blob/
>> master/src/main/java/org/apache/bigtop/bigpetstore/flink/
>> java/FlinkStreamingRecommender.java
>>
>> On Mon, Mar 13, 2017 at 1:29 PM, Suneel Marthi <sm...@apache.org>
>> wrote:
>>
>>> A simple way is to populate a Priority Queue of max size 'k' and
>>> implement a comparator on ur records. That would ensure that u always have
>>> Top k records at any instant in time.
>>>
>>> On Mon, Mar 13, 2017 at 1:25 PM, Meghashyam Sandeep V <
>>> vr1meghashyam@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I'm trying to use Flink for a use case where I would want to see my top
>>>> selling products in time windows in near real time (windows of size 1-2
>>>> mins if fine). I guess this is the most common use case to use streaming
>>>> apis in e-commerce. I see that I can iterate over records in a windowed
>>>> stream and do the sorting myself. I'm wondering if thats the best way. Is
>>>> there any in built sort functionality that I missed anywhere in Flink docs?
>>>>
>>>> Thanks,
>>>> Sandeep
>>>>
>>>
>>>
>>
>
Re: Suggestion for top 'k' products
Posted by Meghashyam Sandeep V <vr...@gmail.com>.
Thanks Suneel. Exactly what I was looking for.
On Mon, Mar 13, 2017 at 10:31 AM, Suneel Marthi <sm...@apache.org> wrote:
> For an example implementation using Flink, check out https://github.com/
> bigpetstore/bigpetstore-flink/blob/master/src/main/java/org/
> apache/bigtop/bigpetstore/flink/java/FlinkStreamingRecommender.java
>
> On Mon, Mar 13, 2017 at 1:29 PM, Suneel Marthi <sm...@apache.org> wrote:
>
>> A simple way is to populate a Priority Queue of max size 'k' and
>> implement a comparator on ur records. That would ensure that u always have
>> Top k records at any instant in time.
>>
>> On Mon, Mar 13, 2017 at 1:25 PM, Meghashyam Sandeep V <
>> vr1meghashyam@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I'm trying to use Flink for a use case where I would want to see my top
>>> selling products in time windows in near real time (windows of size 1-2
>>> mins if fine). I guess this is the most common use case to use streaming
>>> apis in e-commerce. I see that I can iterate over records in a windowed
>>> stream and do the sorting myself. I'm wondering if thats the best way. Is
>>> there any in built sort functionality that I missed anywhere in Flink docs?
>>>
>>> Thanks,
>>> Sandeep
>>>
>>
>>
>
Re: Suggestion for top 'k' products
Posted by Suneel Marthi <sm...@apache.org>.
For an example implementation using Flink, check out
https://github.com/bigpetstore/bigpetstore-flink/blob/master/src/main/java/org/apache/bigtop/bigpetstore/flink/java/FlinkStreamingRecommender.java
On Mon, Mar 13, 2017 at 1:29 PM, Suneel Marthi <sm...@apache.org> wrote:
> A simple way is to populate a Priority Queue of max size 'k' and
> implement a comparator on ur records. That would ensure that u always have
> Top k records at any instant in time.
>
> On Mon, Mar 13, 2017 at 1:25 PM, Meghashyam Sandeep V <
> vr1meghashyam@gmail.com> wrote:
>
>> Hi All,
>>
>> I'm trying to use Flink for a use case where I would want to see my top
>> selling products in time windows in near real time (windows of size 1-2
>> mins if fine). I guess this is the most common use case to use streaming
>> apis in e-commerce. I see that I can iterate over records in a windowed
>> stream and do the sorting myself. I'm wondering if thats the best way. Is
>> there any in built sort functionality that I missed anywhere in Flink docs?
>>
>> Thanks,
>> Sandeep
>>
>
>
Re: Suggestion for top 'k' products
Posted by Suneel Marthi <sm...@apache.org>.
A simple way is to populate a Priority Queue of max size 'k' and implement
a comparator on ur records. That would ensure that u always have Top k
records at any instant in time.
On Mon, Mar 13, 2017 at 1:25 PM, Meghashyam Sandeep V <
vr1meghashyam@gmail.com> wrote:
> Hi All,
>
> I'm trying to use Flink for a use case where I would want to see my top
> selling products in time windows in near real time (windows of size 1-2
> mins if fine). I guess this is the most common use case to use streaming
> apis in e-commerce. I see that I can iterate over records in a windowed
> stream and do the sorting myself. I'm wondering if thats the best way. Is
> there any in built sort functionality that I missed anywhere in Flink docs?
>
> Thanks,
> Sandeep
>