You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Jakob Homan (Created) (JIRA)" <ji...@apache.org> on 2011/12/10 05:56:39 UTC
[jira] [Created] (HIVE-2643) HIVE-2362 preventing Hadoop confs from
loading in 0.20.204
HIVE-2362 preventing Hadoop confs from loading in 0.20.204
----------------------------------------------------------
Key: HIVE-2643
URL: https://issues.apache.org/jira/browse/HIVE-2643
Project: Hive
Issue Type: Bug
Reporter: Jakob Homan
Priority: Blocker
After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
{noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2643:
---------------------------------
Status: Open (was: Patch Available)
Investigating some test failures.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167820#comment-13167820 ]
Carl Steinbach commented on HIVE-2643:
--------------------------------------
@Jakob: Can you please try testing HIVE-2643.D759.2.patch? This version has some changes that were not included in HIVE-2643.1.patch.txt. Thanks!
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) HIVE-2362 preventing Hadoop confs from
loading in 0.20.204
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2643:
------------------------------
Attachment: HIVE-2643.D759.1.patch
cwsteinbach requested code review of "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
Reviewers: JIRA
HIVE-2643
After the application of <a href="https://issues.apache.org/jira/browse/HIVE-2362" title="HiveConf properties not appearing in the output of 'set' or 'set -v'"><del>HIVE-2362</del></a>, we're unable to run queries against the Hive cluster (RC2 and 3): <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)</pre> </div>
</div>
this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D759
AFFECTED FILES
common/build.xml
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
common/src/test/resources/core-site.xml
common/src/test/resources/hive-site.xml
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1605/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> HIVE-2362 preventing Hadoop confs from loading in 0.20.204
> ----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "John Sichi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Sichi updated HIVE-2643:
-----------------------------
Resolution: Fixed
Fix Version/s: 0.8.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and 0.8-r2. Thanks Carl!
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Fix For: 0.8.0
>
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) HIVE-2362 preventing Hadoop confs from
loading in 0.20.204
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2643:
------------------------------
Attachment: HIVE-2643.D747.1.patch
cwsteinbach requested code review of "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
Reviewers: JIRA
HIVE-2643
After the application of <a href="https://issues.apache.org/jira/browse/HIVE-2362" title="HiveConf properties not appearing in the output of 'set' or 'set -v'"><del>HIVE-2362</del></a>, we're unable to run queries against the Hive cluster (RC2 and 3): <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)</pre> </div>
</div>
this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D747
AFFECTED FILES
common/build.xml
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
common/src/test/resources/core-site.xml
common/src/test/resources/hive-site.xml
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1593/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> HIVE-2362 preventing Hadoop confs from loading in 0.20.204
> ----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2643:
------------------------------
Attachment: HIVE-2643.D765.1.patch
cwsteinbach requested code review of "HIVE-2643 [jira] Recent patch prevents Hadoop confs from loading in 0.20.204".
Reviewers: JIRA
HIVE-2643
After the application of <a href="https://issues.apache.org/jira/browse/HIVE-2362" title="HiveConf properties not appearing in the output of 'set' or 'set -v'"><del>HIVE-2362</del></a>, we're unable to run queries against the Hive cluster (RC2 and 3): <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)</pre> </div>
</div>
this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D765
AFFECTED FILES
common/build.xml
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
common/src/test/resources/core-site.xml
common/src/test/resources/hive-site.xml
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1617/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167806#comment-13167806 ]
Phabricator commented on HIVE-2643:
-----------------------------------
mareksapotafb has abandoned the revision "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
REVISION DETAIL
https://reviews.facebook.net/D753
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167811#comment-13167811 ]
Phabricator commented on HIVE-2643:
-----------------------------------
mareksapotafb has abandoned the revision "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
REVISION DETAIL
https://reviews.facebook.net/D759
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2643:
------------------------------
Attachment: HIVE-2643.D789.1.patch
cwsteinbach requested code review of "HIVE-2643 [jira] Recent patch prevents Hadoop confs from loading in 0.20.204".
Reviewers: JIRA
cp
After the application of <a href="https://issues.apache.org/jira/browse/HIVE-2362" title="HiveConf properties not appearing in the output of 'set' or 'set -v'"><del>HIVE-2362</del></a>, we're unable to run queries against the Hive cluster (RC2 and 3): <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)</pre> </div>
</div>
this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D789
AFFECTED FILES
build.xml
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1671/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2643:
---------------------------------
Summary: Recent patch prevents Hadoop confs from loading in 0.20.204 (was: HIVE-2362 preventing Hadoop confs from loading in 0.20.204)
Changing the title of this ticket since referencing another JIRA seems to confuse arc.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2643) HIVE-2362 preventing Hadoop confs
from loading in 0.20.204
Posted by "Carl Steinbach (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach reassigned HIVE-2643:
------------------------------------
Assignee: Carl Steinbach
> HIVE-2362 preventing Hadoop confs from loading in 0.20.204
> ----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "John Sichi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167701#comment-13167701 ]
John Sichi commented on HIVE-2643:
----------------------------------
Marek and I will take a look into the arc confusion with the JIRA reference.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2643:
------------------------------
Attachment: HIVE-2643.D765.2.patch
cwsteinbach updated the revision "HIVE-2643 [jira] Recent patch prevents Hadoop confs from loading in 0.20.204".
Reviewers: JIRA
Fixed some test diffs.
REVISION DETAIL
https://reviews.facebook.net/D765
AFFECTED FILES
build.xml
common/build.xml
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
common/src/test/resources/core-site.xml
common/src/test/resources/hive-site.xml
conf/hive-default.xml.template
conf/hive-default.xml
conf/hive-site.xml
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) HIVE-2362 preventing Hadoop confs from
loading in 0.20.204
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2643:
---------------------------------
Status: Patch Available (was: Open)
Running tests now.
> HIVE-2362 preventing Hadoop confs from loading in 0.20.204
> ----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168188#comment-13168188 ]
Hudson commented on HIVE-2643:
------------------------------
Integrated in Hive-trunk-h0.21 #1142 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1142/])
HIVE-2643. Recent patch prevents Hadoop confs from loading in 0.20.204
(Carl Steinbach via jvs)
jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213514
Files :
* /hive/trunk/build.xml
* /hive/trunk/common/build.xml
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
* /hive/trunk/common/src/test/resources/core-site.xml
* /hive/trunk/common/src/test/resources/hive-site.xml
* /hive/trunk/conf/hive-default.xml
* /hive/trunk/conf/hive-default.xml.template
* /hive/trunk/conf/hive-site.xml
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Fix For: 0.8.0
>
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2643:
---------------------------------
Attachment: HIVE-2643.D759.2.patch
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Jakob Homan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167873#comment-13167873 ]
Jakob Homan commented on HIVE-2643:
-----------------------------------
Applied against RC3 and tested again. Everything works. Also reviewed the patch. +1, with the minor nit that having the test conf return "core-site.xml" for the default filesystem makes the test a bit wonky to read ("... wait, that's not right. oh, ok"), but that's not worth spinning another patch for if nothing else is blocking this version.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) HIVE-2362 preventing Hadoop confs from
loading in 0.20.204
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2643:
---------------------------------
Attachment: HIVE-2643.1.patch.txt
> HIVE-2362 preventing Hadoop confs from loading in 0.20.204
> ----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167807#comment-13167807 ]
Phabricator commented on HIVE-2643:
-----------------------------------
mareksapotafb has abandoned the revision "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
REVISION DETAIL
https://reviews.facebook.net/D747
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167973#comment-13167973 ]
Phabricator commented on HIVE-2643:
-----------------------------------
jsichi has added reviewers to the revision "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
Added Reviewers: jsichi
REVISION DETAIL
https://reviews.facebook.net/D759
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2643:
---------------------------------
Status: Patch Available (was: Open)
HIVE-2643.D759.2.patch is the latest version of the patch. It is ready for review here: https://reviews.facebook.net/D765
I'm running full tests now.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) HIVE-2362 preventing Hadoop confs from
loading in 0.20.204
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2643:
---------------------------------
Component/s: Configuration
> HIVE-2362 preventing Hadoop confs from loading in 0.20.204
> ----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168107#comment-13168107 ]
Hudson commented on HIVE-2643:
------------------------------
Integrated in Hive-trunk-h0.23.0 #19 (See [https://builds.apache.org/job/Hive-trunk-h0.23.0/19/])
HIVE-2643. Recent patch prevents Hadoop confs from loading in 0.20.204
(Carl Steinbach via jvs)
jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1213514
Files :
* /hive/trunk/build.xml
* /hive/trunk/common/build.xml
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
* /hive/trunk/common/src/test/resources/core-site.xml
* /hive/trunk/common/src/test/resources/hive-site.xml
* /hive/trunk/conf/hive-default.xml
* /hive/trunk/conf/hive-default.xml.template
* /hive/trunk/conf/hive-site.xml
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Fix For: 0.8.0
>
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Jakob Homan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167747#comment-13167747 ]
Jakob Homan commented on HIVE-2643:
-----------------------------------
I've applied HIVE-2643.1.patch.txt (the only ASF-licensed patch; are the others artifacts of the issue name confusion?) to the RC3 tag in svn, built a new package and run against our 204 cluster and everything is working now. Didn't do a code review, but +1 on the result.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) HIVE-2362 preventing Hadoop confs from
loading in 0.20.204
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2643:
------------------------------
Attachment: HIVE-2643.D753.1.patch
cwsteinbach requested code review of "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
Reviewers: JIRA
lint check
After the application of <a href="https://issues.apache.org/jira/browse/HIVE-2362" title="HiveConf properties not appearing in the output of 'set' or 'set -v'"><del>HIVE-2362</del></a>, we're unable to run queries against the Hive cluster (RC2 and 3): <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700 at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680) at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483) at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318) at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116)</pre> </div>
</div>
this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D753
AFFECTED FILES
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/1599/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> HIVE-2362 preventing Hadoop confs from loading in 0.20.204
> ----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "John Sichi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167968#comment-13167968 ]
John Sichi commented on HIVE-2643:
----------------------------------
+1, passed tests, committing to trunk.
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch, HIVE-2643.D759.2.patch, HIVE-2643.D765.1.patch, HIVE-2643.D765.2.patch, HIVE-2643.D789.1.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2643) Recent patch prevents Hadoop confs
from loading in 0.20.204
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2643:
------------------------------
Attachment: HIVE-2643.D759.2.patch
cwsteinbach updated the revision "HIVE-2643 [jira] HIVE-2362 preventing Hadoop confs from loading in 0.20.204".
Reviewers: JIRA
Fix for HIVE-2643
REVISION DETAIL
https://reviews.facebook.net/D759
AFFECTED FILES
common/build.xml
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
common/src/test/resources/core-site.xml
common/src/test/resources/hive-site.xml
ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
> Recent patch prevents Hadoop confs from loading in 0.20.204
> -----------------------------------------------------------
>
> Key: HIVE-2643
> URL: https://issues.apache.org/jira/browse/HIVE-2643
> Project: Hive
> Issue Type: Bug
> Components: Configuration
> Reporter: Jakob Homan
> Assignee: Carl Steinbach
> Priority: Blocker
> Attachments: HIVE-2643.1.patch.txt, HIVE-2643.D747.1.patch, HIVE-2643.D753.1.patch, HIVE-2643.D759.1.patch, HIVE-2643.D759.2.patch
>
>
> After the application of HIVE-2362, we're unable to run queries against the Hive cluster (RC2 and 3):
> {noformat}java.io.IOException: Failed to set permissions of path: /user/jhoman-667872320/.staging to 0700
> at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:680)
> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:653)
> at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:483)
> at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:318)
> at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:183)
> at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:116){noformat}
> this is due to all the hadoop confs not being loaded and the LocalJobRunner (which builds staging directories in this manner) being used instead of the regular one. It's also not possible to access the hdfs since no default fs is specified.
> Reverting 2362 fixes this, but I've not yet looked at that patch to see the exact cause.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira