You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by xubo245 <gi...@git.apache.org> on 2016/07/30 15:43:47 UTC
[GitHub] spark pull request #14422: Add rand(numRows: Int, numCols: Int) functions
GitHub user xubo245 opened a pull request:
https://github.com/apache/spark/pull/14422
Add rand(numRows: Int, numCols: Int) functions
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
add rand(numRows: Int, numCols: Int) functions to DenseMatrix object,like breeze.linalg.DenseMatrix.rand()
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xubo245/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14422.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14422
----
commit a7a1261f52112a3bca375dd0bed1c1bc0a2e0ed8
Author: \u5f90\u6ce2 <60...@qq.com>
Date: 2016-07-30T15:43:36Z
Add rand(numRows: Int, numCols: Int) functions
add rand(numRows: Int, numCols: Int) functions to DenseMatrix object,like breeze.linalg.DenseMatrix.rand()
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
GitHub user xubo245 reopened a pull request:
https://github.com/apache/spark/pull/14422
Add rand(numRows: Int, numCols: Int) functions
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
add rand(numRows: Int, numCols: Int) functions to DenseMatrix object,like breeze.linalg.DenseMatrix.rand()
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xubo245/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14422.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14422
----
commit a7a1261f52112a3bca375dd0bed1c1bc0a2e0ed8
Author: \u5f90\u6ce2 <60...@qq.com>
Date: 2016-07-30T15:43:36Z
Add rand(numRows: Int, numCols: Int) functions
add rand(numRows: Int, numCols: Int) functions to DenseMatrix object,like breeze.linalg.DenseMatrix.rand()
commit 054b70ccce73c02cce04caf9f7958cfc555df829
Author: \u5f90\u6ce2 <60...@qq.com>
Date: 2016-07-30T16:36:30Z
fix RNG
fix RNG , his makes a new RNG for All element
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14422
I guess it might be ncer if the contributions you make follow https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark.just like other contributions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/spark/pull/14422
we can use it to replacebreeze.linalg.DenseMatrix.rand(numRows: Int, numCols: Int)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/spark/pull/14422
@srowen sorry, please close the issue. I will learning more before next PR. The PR is only because breeze have the function. In spark ,there is no use for them.
Could you tell me some issue for starter? Please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/spark/pull/14422
@HyukjinKwon Thank you.
This is my first time to push request to spark, Sorrry, I will follow the https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/14422#discussion_r72889871
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -497,6 +497,20 @@ object DenseMatrix {
}
/**
+ * Generate a `DenseMatrix` consisting of `i.i.d.` uniform random numbers.
+ *
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ @Since("2.0.0")
+ def rand(numRows: Int, numCols: Int): DenseMatrix = {
+ require(numRows.toLong * numCols <= Int.MaxValue,
+ s"$numRows x $numCols dense matrix is too large to allocate")
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)((new Random).nextDouble()))
--- End diff --
This makes a new RNG for every element, which isn't great.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14422
@xubo245 you need to close the issues, we can't do so directly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14422
Please don't open other PRs for related changes. Usually, you also need a JIRA. Although I understand the use for these methods, is there any use for them in Spark? we probably wouldn't add these things unless Spark itself needed them a few places, or else it was clearly a common task. I am not clear these are.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 closed the pull request at:
https://github.com/apache/spark/pull/14422
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14422
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 closed the pull request at:
https://github.com/apache/spark/pull/14422
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/spark/pull/14422
I add test : https://github.com/apache/spark/pull/14424
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14422
Doesn't that need a JIRA, test and PR descriptio maybe?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/spark/pull/14422
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14422: Add rand(numRows: Int, numCols: Int) functions
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14422#discussion_r72890434
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -497,6 +497,20 @@ object DenseMatrix {
}
/**
+ * Generate a `DenseMatrix` consisting of `i.i.d.` uniform random numbers.
+ *
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ @Since("2.0.0")
+ def rand(numRows: Int, numCols: Int): DenseMatrix = {
+ require(numRows.toLong * numCols <= Int.MaxValue,
+ s"$numRows x $numCols dense matrix is too large to allocate")
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)((new Random).nextDouble()))
--- End diff --
Can fix RNG, This makes a new RNG for all element :
val rng = new Random()
new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rng.nextDouble()))
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org