You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by kayousterhout <gi...@git.apache.org> on 2017/02/21 22:14:42 UTC
[GitHub] spark pull request #17018: [SPARK-19684] [DOCS] Remove developer info from d...
GitHub user kayousterhout opened a pull request:
https://github.com/apache/spark/pull/17018
[SPARK-19684] [DOCS] Remove developer info from docs.
This commit moves developer-specific information from the release-
specific documentation in this repo to the developer tools page on
the main Spark website. This commit relies on this PR on the
Spark website: https://github.com/apache/spark-website/pull/33.
@srowen
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kayousterhout/spark-1 SPARK-19684
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17018.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17018
----
commit ee666c612a6f979f4a3d355c83e66d7ec6b48684
Author: Kay Ousterhout <ka...@gmail.com>
Date: 2017-02-21T22:12:30Z
[SPARK-19684] [DOCS] Remove developer info from docs.
This commit moves developer-specific information from the release-
specific documentation in this repo to the developer tools page on
the main Spark website. This commit relies on this PR on the
Spark website: https://github.com/apache/spark-website/pull/33.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17018
**[Test build #73356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73356/testReport)** for PR 17018 at commit [`c8a094a`](https://github.com/apache/spark/commit/c8a094a6db56a5e99f1d77652b099e06f5a3a6bc).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `public class TransportChannelHandler extends ChannelInboundHandlerAdapter `
* `trait Logging `
* `public class JavaLinearSVCExample `
* ` class LinearSVCWrapperWriter(instance: LinearSVCWrapper) extends MLWriter `
* ` class LinearSVCWrapperReader extends MLReader[LinearSVCWrapper] `
* `class LSHParams(Params):`
* `class LSHModel(JavaModel):`
* `class BucketedRandomProjectionLSH(JavaEstimator, LSHParams, HasInputCol, HasOutputCol, HasSeed,`
* `class BucketedRandomProjectionLSHModel(LSHModel, JavaMLReadable, JavaMLWritable):`
* `class MinHashLSH(JavaEstimator, LSHParams, HasInputCol, HasOutputCol, HasSeed,`
* `class MinHashLSHModel(LSHModel, JavaMLReadable, JavaMLWritable):`
* `class YarnProxyRedirectFilter extends Filter with Logging `
* `class NoSuchDatabaseException(val db: String) extends AnalysisException(s\"Database '$db' not found\")`
* ` class ResolveBroadcastHints(conf: CatalystConf) extends Rule[LogicalPlan] `
* `case class UnresolvedRelation(tableIdentifier: TableIdentifier) extends LeafNode `
* `case class JsonToStruct(`
* `case class StructToJson(`
* `case class Hint(name: String, parameters: Seq[String], child: LogicalPlan) extends UnaryNode `
* `case class InnerOuterEstimation(conf: CatalystConf, join: Join) extends Logging `
* `case class LeftSemiAntiEstimation(conf: CatalystConf, join: Join) `
* `case class NumericRange(min: JDecimal, max: JDecimal) extends Range`
* `sealed abstract class HiveStringType extends AtomicType `
* `case class CharType(length: Int) extends HiveStringType `
* `case class VarcharType(length: Int) extends HiveStringType `
* `case class StreamingExplainCommand(`
* `case class SaveIntoDataSourceCommand(`
* `abstract class JsonDataSource[T] extends Serializable `
* `class FileStreamOptions(parameters: CaseInsensitiveMap[String]) extends Logging `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17018
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17018
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73237/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17018
**[Test build #73237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73237/testReport)** for PR 17018 at commit [`ee666c6`](https://github.com/apache/spark/commit/ee666c612a6f979f4a3d355c83e66d7ec6b48684).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/17018
Just to clarify what I meant is that there was a similar error fixed due to Hadoop / SQL issues by @HyukjinKwon in https://github.com/apache/spark/pull/16927 - I'll see if this happens again and file a issue if it does
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17018
Let me run a build in my account to double check and create a JIRA with cc'ing you if it is broken.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17018
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17018
**[Test build #73237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73237/testReport)** for PR 17018 at commit [`ee666c6`](https://github.com/apache/spark/commit/ee666c612a6f979f4a3d355c83e66d7ec6b48684).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17018: [SPARK-19684] [DOCS] Remove developer info from d...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/17018
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17018
I have a confirmation that it is increased to 90 mins now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17018
Ah, this one seems timed out because it exceeds the 1 hour limitation in AppVeyor. It seems now R tests could take longer than an hour including the build rarely.
This limit could be increased a bit by asking a bit. My account has 1 and half hours (I asked when I tried to run some Scala/Java tests). Let me maybe try to ask increasing it first if you all think it is fine.
If it can be a problem with the increased time in the future, let me try to split the tests as I did for Scala/Java tests in my account.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/17018
Thanks - will keep an eye out to see if the time out errors are gone.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17018
**[Test build #73356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73356/testReport)** for PR 17018 at commit [`c8a094a`](https://github.com/apache/spark/commit/c8a094a6db56a5e99f1d77652b099e06f5a3a6bc).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17018
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73356/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17018
I see. Sure. Just as a side note, if you download the log, it prints the times.
In this case, it was so close assuming from the last log :).
```
[00:59:47] Windows-specific tests: .
[00:59:47]
[00:59:47] Warnings -----------------------------------------------------------------------
[00:59:47] 1. infer types and check types (@test_sparkSQL.R#109) - unable to identify current timezone 'C':
[00:59:47] please set environment variable 'TZ'
[00:59:47]
[00:59:47] DONE ===========================================================================
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/17018
Yeah asking them for 1.5hr seems like a good first step. We can also see what fraction of time it takes for build vs. tests. I guess we can't make builds faster without some effort, but we could look at making the tests faster or running a subset of them (for example the RDD tests in SparkR are probably not high priority as the API is private etc.)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the issue:
https://github.com/apache/spark/pull/17018
Hmm I think this specific one timed out, so it might not be worth investigating https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/build/824-master/messages
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by kayousterhout <gi...@git.apache.org>.
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17018
Ignored the appveyor failure based on advice from @shivaram that this was broken by the SQL folks (and it does seem clearly unrelated to this PR), and merged this into master. Thanks for the review @srowen.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17018: [SPARK-19684] [DOCS] Remove developer info from docs.
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17018
Thanks for cc'ing me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org