You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Bryan Cutler <cu...@gmail.com> on 2016/08/02 20:46:31 UTC
AccumulatorV2 += operator
It seems like the += operator is missing from the new accumulator API,
although the docs still make reference to it. Anyone know if it was
intentionally not put in? I'm happy to do a PR for it or update the docs
to just use the add() method, just want to check if there was some reason
first.
Bryan
Re: AccumulatorV2 += operator
Posted by Holden Karau <ho...@pigscanfly.ca>.
Ah in that case the programming guides text is still talking about the
deprecated accumulator API despite having an updated code sample (the way
it suggests making an accumulator is also deprecated). I think the fix is
updating the programming guide rather than adding += to the API.
On Wednesday, August 3, 2016, Bryan Cutler <cu...@gmail.com> wrote:
> No, I was referring to the programming guide section on accumulators, it
> says " Tasks running on a cluster can then add to it using the add method
> or the += operator (in Scala and Python)."
>
> On Aug 2, 2016 2:52 PM, "Holden Karau" <holden@pigscanfly.ca
> <javascript:_e(%7B%7D,'cvml','holden@pigscanfly.ca');>> wrote:
>
>> I believe it was intentional with the idea that it would be more unified
>> between Java and Scala APIs. If your talking about the javadoc mention in
>> https://github.com/apache/spark/pull/14466/files - I believe the += is
>> meant to refer to what the internal implementation of the add function can
>> be for someone extending the accumulator (but it certainly could cause
>> confusion).
>>
>> Reynold can provide a more definitive answer in this case.
>>
>> On Tue, Aug 2, 2016 at 1:46 PM, Bryan Cutler <cutlerb@gmail.com
>> <javascript:_e(%7B%7D,'cvml','cutlerb@gmail.com');>> wrote:
>>
>>> It seems like the += operator is missing from the new accumulator API,
>>> although the docs still make reference to it. Anyone know if it was
>>> intentionally not put in? I'm happy to do a PR for it or update the docs
>>> to just use the add() method, just want to check if there was some reason
>>> first.
>>>
>>> Bryan
>>>
>>
>>
>>
>> --
>> Cell : 425-233-8271
>> Twitter: https://twitter.com/holdenkarau
>>
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Re: AccumulatorV2 += operator
Posted by Bryan Cutler <cu...@gmail.com>.
No, I was referring to the programming guide section on accumulators, it
says " Tasks running on a cluster can then add to it using the add method
or the += operator (in Scala and Python)."
On Aug 2, 2016 2:52 PM, "Holden Karau" <ho...@pigscanfly.ca> wrote:
> I believe it was intentional with the idea that it would be more unified
> between Java and Scala APIs. If your talking about the javadoc mention in
> https://github.com/apache/spark/pull/14466/files - I believe the += is
> meant to refer to what the internal implementation of the add function can
> be for someone extending the accumulator (but it certainly could cause
> confusion).
>
> Reynold can provide a more definitive answer in this case.
>
> On Tue, Aug 2, 2016 at 1:46 PM, Bryan Cutler <cu...@gmail.com> wrote:
>
>> It seems like the += operator is missing from the new accumulator API,
>> although the docs still make reference to it. Anyone know if it was
>> intentionally not put in? I'm happy to do a PR for it or update the docs
>> to just use the add() method, just want to check if there was some reason
>> first.
>>
>> Bryan
>>
>
>
>
> --
> Cell : 425-233-8271
> Twitter: https://twitter.com/holdenkarau
>
Re: AccumulatorV2 += operator
Posted by Holden Karau <ho...@pigscanfly.ca>.
I believe it was intentional with the idea that it would be more unified
between Java and Scala APIs. If your talking about the javadoc mention in
https://github.com/apache/spark/pull/14466/files - I believe the += is
meant to refer to what the internal implementation of the add function can
be for someone extending the accumulator (but it certainly could cause
confusion).
Reynold can provide a more definitive answer in this case.
On Tue, Aug 2, 2016 at 1:46 PM, Bryan Cutler <cu...@gmail.com> wrote:
> It seems like the += operator is missing from the new accumulator API,
> although the docs still make reference to it. Anyone know if it was
> intentionally not put in? I'm happy to do a PR for it or update the docs
> to just use the add() method, just want to check if there was some reason
> first.
>
> Bryan
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau