[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

[squito <gi...@git.apache.org>]

Github user squito commented on the issue:

    https://github.com/apache/spark/pull/11105
  
    I'm thinking about the binary compatability issue specifically with `DoubleAccumulator` etc.  Whether we intended it or not, `DoubleAccumulator` is not `final`, so a user could have subclassed it.  Here's an example which isn't totally insane -- suppose a user created a subclass in which they overrode `add` and `merge` be more numerically stable.
    
    We're avoiding binary incompatibility by not declaring your `add` and `merge` to be final methods, even though really they should be.  The net effect is that the user's code still works as non-data-property accumulators, right?  But if they tried to use it as a data property accumulator, it would silently still behave as a non-data-property accumulator?
    
    I'm wondering if we can live with that for now, and just put big warning signs in the doc.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org