You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2015/04/25 14:42:45 UTC
spark git commit: update the deprecated CountMinSketchMonoid function
to TopPctCMS function
Repository: spark
Updated Branches:
refs/heads/master 59b7cfc41 -> cca9905b9
update the deprecated CountMinSketchMonoid function to TopPctCMS function
http://twitter.github.io/algebird/index.html#com.twitter.algebird.legacy.CountMinSketchMonoid$
The CountMinSketchMonoid has been deprecated since 0.8.1. Newer code should use TopPctCMS.monoid().
Author: KeheCAI <ca...@gmail.com>
Closes #5629 from caikehe/master and squashes the following commits:
e8aa06f [KeheCAI] update algebird-core to version 0.9.0 from 0.8.1
5653351 [KeheCAI] change scala code style
4c0dfd1 [KeheCAI] update the deprecated CountMinSketchMonoid function to TopPctCMS function
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/cca9905b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/cca9905b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/cca9905b
Branch: refs/heads/master
Commit: cca9905b93483614b330b09b36c6526b551e17dc
Parents: 59b7cfc
Author: KeheCAI <ca...@gmail.com>
Authored: Sat Apr 25 08:42:38 2015 -0400
Committer: Sean Owen <so...@cloudera.com>
Committed: Sat Apr 25 08:42:38 2015 -0400
----------------------------------------------------------------------
examples/pom.xml | 2 +-
.../org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala | 4 +++-
2 files changed, 4 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/cca9905b/examples/pom.xml
----------------------------------------------------------------------
diff --git a/examples/pom.xml b/examples/pom.xml
index df17174..5b04b4f 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -245,7 +245,7 @@
<dependency>
<groupId>com.twitter</groupId>
<artifactId>algebird-core_${scala.binary.version}</artifactId>
- <version>0.8.1</version>
+ <version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.scalacheck</groupId>
http://git-wip-us.apache.org/repos/asf/spark/blob/cca9905b/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala
----------------------------------------------------------------------
diff --git a/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala b/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala
index 62f4953..c10de84 100644
--- a/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala
+++ b/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala
@@ -18,6 +18,7 @@
package org.apache.spark.examples.streaming
import com.twitter.algebird._
+import com.twitter.algebird.CMSHasherImplicits._
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext._
@@ -67,7 +68,8 @@ object TwitterAlgebirdCMS {
val users = stream.map(status => status.getUser.getId)
- val cms = new CountMinSketchMonoid(EPS, DELTA, SEED, PERC)
+ // val cms = new CountMinSketchMonoid(EPS, DELTA, SEED, PERC)
+ val cms = TopPctCMS.monoid[Long](EPS, DELTA, SEED, PERC)
var globalCMS = cms.zero
val mm = new MapMonoid[Long, Int]()
var globalExact = Map[Long, Int]()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org