You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Xinyu Zhang <ws...@163.com> on 2016/11/21 07:51:28 UTC
Re:Re: Multiple streaming aggregations in structured streaming
MapWithState is also very useful.
I want to calculate UV in real time, but "distinct count" and "multiple streaming aggregations" are not supported.
Is there any method to calculate real-time UV in the current version?
At 2016-11-19 06:01:45, "Michael Armbrust" <mi...@databricks.com> wrote:
Doing this generally is pretty hard. We will likely support algebraic aggregate eventually, but this is not currently slotted for 2.2. Instead I think we will add something like mapWithState that lets users compute arbitrary stateful things. What is your use case?
On Wed, Nov 16, 2016 at 6:58 PM, wszxyh <ws...@163.com> wrote:
Hi
Multiple streaming aggregations are not yet supported. When will it be supported? Is it in the plan?
Thanks
Re: Re: Re: Multiple streaming aggregations in structured streaming
Posted by Reynold Xin <rx...@databricks.com>.
It's just the "approx_count_distinct" aggregate function.
On Tue, Nov 22, 2016 at 6:51 PM, Xinyu Zhang <ws...@163.com> wrote:
> Could you please tell me how to use the approximate count distinct? Is
> there any docs?
>
> Thanks
>
>
> At 2016-11-21 15:56:21, "Reynold Xin" <rx...@databricks.com> wrote:
>
> Can you use the approximate count distinct?
>
>
> On Sun, Nov 20, 2016 at 11:51 PM, Xinyu Zhang <ws...@163.com> wrote:
>
>>
>> MapWithState is also very useful.
>> I want to calculate UV in real time, but "distinct count" and "multiple
>> streaming aggregations" are not supported.
>> Is there any method to calculate real-time UV in the current version?
>>
>>
>>
>> At 2016-11-19 06:01:45, "Michael Armbrust" <mi...@databricks.com>
>> wrote:
>>
>> Doing this generally is pretty hard. We will likely support algebraic
>> aggregate eventually, but this is not currently slotted for 2.2. Instead I
>> think we will add something like mapWithState that lets users compute
>> arbitrary stateful things. What is your use case?
>>
>>
>> On Wed, Nov 16, 2016 at 6:58 PM, wszxyh <ws...@163.com> wrote:
>>
>>> Hi
>>>
>>> Multiple streaming aggregations are not yet supported. When will it be
>>> supported? Is it in the plan?
>>>
>>> Thanks
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
>
>
Re:Re: Re: Multiple streaming aggregations in structured streaming
Posted by Xinyu Zhang <ws...@163.com>.
Could you please tell me how to use the approximate count distinct? Is there any docs?
Thanks
At 2016-11-21 15:56:21, "Reynold Xin" <rx...@databricks.com> wrote:
Can you use the approximate count distinct?
On Sun, Nov 20, 2016 at 11:51 PM, Xinyu Zhang <ws...@163.com> wrote:
MapWithState is also very useful.
I want to calculate UV in real time, but "distinct count" and "multiple streaming aggregations" are not supported.
Is there any method to calculate real-time UV in the current version?
At 2016-11-19 06:01:45, "Michael Armbrust" <mi...@databricks.com> wrote:
Doing this generally is pretty hard. We will likely support algebraic aggregate eventually, but this is not currently slotted for 2.2. Instead I think we will add something like mapWithState that lets users compute arbitrary stateful things. What is your use case?
On Wed, Nov 16, 2016 at 6:58 PM, wszxyh <ws...@163.com> wrote:
Hi
Multiple streaming aggregations are not yet supported. When will it be supported? Is it in the plan?
Thanks
Re: Re: Multiple streaming aggregations in structured streaming
Posted by Reynold Xin <rx...@databricks.com>.
Can you use the approximate count distinct?
On Sun, Nov 20, 2016 at 11:51 PM, Xinyu Zhang <ws...@163.com> wrote:
>
> MapWithState is also very useful.
> I want to calculate UV in real time, but "distinct count" and "multiple
> streaming aggregations" are not supported.
> Is there any method to calculate real-time UV in the current version?
>
>
>
> At 2016-11-19 06:01:45, "Michael Armbrust" <mi...@databricks.com> wrote:
>
> Doing this generally is pretty hard. We will likely support algebraic
> aggregate eventually, but this is not currently slotted for 2.2. Instead I
> think we will add something like mapWithState that lets users compute
> arbitrary stateful things. What is your use case?
>
>
> On Wed, Nov 16, 2016 at 6:58 PM, wszxyh <ws...@163.com> wrote:
>
>> Hi
>>
>> Multiple streaming aggregations are not yet supported. When will it be
>> supported? Is it in the plan?
>>
>> Thanks
>>
>>
>>
>>
>
>
>
>
>
Re: Re: Multiple streaming aggregations in structured streaming
Posted by Reynold Xin <rx...@databricks.com>.
Can you use the approximate count distinct?
On Sun, Nov 20, 2016 at 11:51 PM, Xinyu Zhang <ws...@163.com> wrote:
>
> MapWithState is also very useful.
> I want to calculate UV in real time, but "distinct count" and "multiple
> streaming aggregations" are not supported.
> Is there any method to calculate real-time UV in the current version?
>
>
>
> At 2016-11-19 06:01:45, "Michael Armbrust" <mi...@databricks.com> wrote:
>
> Doing this generally is pretty hard. We will likely support algebraic
> aggregate eventually, but this is not currently slotted for 2.2. Instead I
> think we will add something like mapWithState that lets users compute
> arbitrary stateful things. What is your use case?
>
>
> On Wed, Nov 16, 2016 at 6:58 PM, wszxyh <ws...@163.com> wrote:
>
>> Hi
>>
>> Multiple streaming aggregations are not yet supported. When will it be
>> supported? Is it in the plan?
>>
>> Thanks
>>
>>
>>
>>
>
>
>
>
>