You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by igorcosta <gi...@git.apache.org> on 2015/03/19 19:41:08 UTC
[GitHub] spark pull request: Aditional information for users building from ...
GitHub user igorcosta opened a pull request:
https://github.com/apache/spark/pull/5092
Aditional information for users building from source code
Pullrequest to the correct branch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/igorcosta/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5092.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5092
----
commit eeb77a9c76a8891b0489ca55dcf6bd30efac6a43
Author: Igor Costa <ig...@b2wdigital.com>
Date: 2015-03-19T18:31:10Z
Aditional information for users building from source code
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86213066
@igorcosta sorry a few more trivial things -- what about the build command is needed to use PySpark? you could say "Hadoop 2.4" rather than "Hadoop" since that's specifically what the build is configured for, and likewise Hive 0.13. Finally I think you can show this command as a line of code with the triple-backticks. Might make it look nicer on the home page.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-83720159
Nah, I think this is purposely covered in more detail, separately, at http://spark.apache.org/docs/latest/building-spark.html which this generic overview page links to. I don't think we want to push duplicate info back into this wiki.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by igorcosta <gi...@git.apache.org>.
Github user igorcosta commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86203326
@srowen Just -Phive . Ive just added exceeded.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-83712268
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by igorcosta <gi...@git.apache.org>.
Github user igorcosta commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-83730033
Sort of I agree, but if you're just getting started you need other options specially with hive support right out of the box. I took myself as example.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by igorcosta <gi...@git.apache.org>.
Github user igorcosta commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86964751
Yep will create a new PR
On Mar 27, 2015 9:21 AM, "Sean Owen" <no...@github.com> wrote:
> @igorcosta <https://github.com/igorcosta> are you going to update this PR
> accordingly or do you want me to edit on merge? it would be almost like
> making a new PR so I wanted to check first.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/5092#issuecomment-86925265>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86925265
@igorcosta are you going to update this PR accordingly or do you want me to edit on merge? it would be almost like making a new PR so I wanted to check first.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by igorcosta <gi...@git.apache.org>.
Github user igorcosta commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86219236
@srowen I agree.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5092
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-83789238
OK I am convinced, merge it. I think both hive profiles are needed in this example?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86208096
@igorcosta are you saying you need to make an update or that this should be ready as is?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86965030
You can just push to this one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by igorcosta <gi...@git.apache.org>.
Github user igorcosta commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-86209285
@srowen ignore it. ready as is.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-88066311
@igorcosta would you mind updating this PR accordingly, or else would you mind closing this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/5092#discussion_r26800671
--- Diff: README.md ---
@@ -24,6 +24,10 @@ To build Spark and its example programs, run:
mvn -DskipTests clean package
+To build Spark with support for Hadoop, Hive and Python shell for example, you could use
+
+ mvn -Pyarn -Phive -Phive-0.13.1 -Phive-thriftserver -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package
--- End diff --
do we need both -Phive and -Phive-0.13.1?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-83731500
Rather than duplicate the info, I'd suggest just adding to the following sentence to highlight that you can find info about how to build with, say Hive support at the following link ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Aditional information for users building from ...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/5092#issuecomment-83769930
@srowen I agree if this adds a lot of lines, but since this is only adding one line, and a lot of people will probably want the Hive support, seems no brainer to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org