You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2016/04/11 10:42:27 UTC
[jira] [Commented] (MAHOUT-1800) Pare down Classtag overuse
[ https://issues.apache.org/jira/browse/MAHOUT-1800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234719#comment-15234719 ]
Hudson commented on MAHOUT-1800:
--------------------------------
FAILURE: Integrated in Mahout-Quality #3324 (See [https://builds.apache.org/job/Mahout-Quality/3324/])
MAHOUT-1800: Pare down Casstag overuse closes apache/mahout#183 (smarthi: rev 97a3f2e022bd2c5fbefb4b53d670b3f313a39cc9)
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/Par.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/MapBlock.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/DrmRddOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpABAnyKey.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/DrmLike.scala
* spark/src/main/scala/org/apache/mahout/classifier/naivebayes/SparkNaiveBayes.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/RbindAB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAewScalar.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/ABt.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AinCoreB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/package.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtx.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/AewB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpRbind.scala
* math-scala/src/main/scala/org/apache/mahout/math/decompositions/DQR.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpABt.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/Ax.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/AbstractBinaryOp.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAx.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/CheckpointAction.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAewUnaryFuncFusion.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/CbindAB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpTimesLeftMatrix.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAewB.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpCbindScalar.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/RLikeDrmOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/DistributedEngine.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtAnyKey.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAtA.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/package.scala
* math-scala/src/test/scala/org/apache/mahout/math/drm/DrmLikeOpsSuiteBase.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpRowRange.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/CheckpointedDrmSparkOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSSVD.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAB.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/blas/package.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/drm/package.scala
* spark/src/main/scala/org/apache/mahout/sparkbindings/SparkEngine.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpTimesRightMatrix.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpMapBlock.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/CheckpointedOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/decompositions/ALS.scala
* math-scala/src/main/scala/org/apache/mahout/classifier/naivebayes/NaiveBayes.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAewUnaryFunc.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpAt.scala
* spark/src/main/scala/org/apache/mahout/drivers/TrainNBDriver.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpCbind.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/DrmLikeOps.scala
* math-scala/src/main/scala/org/apache/mahout/math/decompositions/DSPCA.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/OpPar.scala
* math-scala/src/main/scala/org/apache/mahout/math/drm/logical/AbstractUnaryOp.scala
* h2o/src/main/scala/org/apache/mahout/h2obindings/H2OEngine.scala
> Pare down Classtag overuse
> --------------------------
>
> Key: MAHOUT-1800
> URL: https://issues.apache.org/jira/browse/MAHOUT-1800
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.11.1
> Reporter: Andrew Palumbo
> Fix For: 0.11.2
>
>
> currently, almost every operator requires implicit parameter for the classtag context bound of drm rowset key type, even for things like drmA + drmB.
> in reality though DAG can already infer that similarly to e.g. it infers product geometry because classtags are already embedded in the logical plan.
> for example, {{classtag(drmA+drmB) == classtag(drmA) == classtag(drmB)}}.
> Not only does the DAG already contain this information, but also it opens doors to a loss of inference, since the optimizer doesn't verify that the new context bound is actually valid by retracing the inference. So any operation may introduce an invalid row key type, and as a consequence, invalid optimization information, without any further checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)