You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/08/26 16:46:23 UTC
[GitHub] [hadoop] wayneseguin opened a new pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
wayneseguin opened a new pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248
Windows users with default Java install location (c:\Program Files\Java) get errors when running Hadoop due to a space in the path. The fix converts the JAVA_HOME environment variable to the dos (short) path equivalent.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-1023504325
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| -1 :x: | mvninstall | 31m 57s | [/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/artifact/out/branch-mvninstall-root.txt) | root in trunk failed. |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 52m 49s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 1s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 1m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 20s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 4s | | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not generate ASF License warnings. |
| | | 77m 57s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2248 |
| Optional Tests | dupname asflicense mvnsite unit codespell |
| uname | Linux 186dbbf3af10 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e31552d08a58c71eebe1027cabe2846480846af2 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/testReport/ |
| Max. process+thread count | 621 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] wayneseguin commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
wayneseguin commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681186498
Honestly, I would expect that NTFS feature is disabled, and implemented correctly, then what this fix should be a no-op. That said, this fix was mainly aimed at the plethora of windows 10 users who are installing this on simple workstations.
Would you prefer I pull this fix and submit the multi-file fix instead I could do that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] wayneseguin commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
wayneseguin commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681163854
The JIRA is https://issues.apache.org/jira/browse/HADOOP-172304. Is there a specific place I should link it?
As to your questions and comments, while I agree with the potential performance improvement, do we want to force users to reconfigure their systems in a manner possibly incompatible with other requirements they might have? And if so, then that should be part of the published installation instructions, I don't think that is the right thing for the users, however.
The quotes around the %JAVA_HOME% don't help; that line actually works either way. There is an alternate solution where proper quoting in 2 different places can bypass the issue. One in hadoop-config.cmd and one in hadoop.cmd.
Hadoop-config.cmd: for /f "delims=*" %%A in ("'%JAVA% -Xmx32m %HADOOP_JAVA_PLATFORM_OPTS% -classpath %CLASSPATH%"" org.apache.hadoop.util.PlatformName') do set JAVA_PLATFORM=%%A
hadoop.cmd: call "%JAVA%" %JAVA_HEAP_MAX% %HADOOP_OPTS% -classpath %CLASSPATH% %CLASS% %hadoop-command-arguments%
But I thought it more efficient to have it in a single place where everything following gets fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681945141
didn't know you could turn off 8.3 naming in NTFS.
Underlying issue here is that JDKs will go under C:\Program Files\ unless you tell the installer not to. I think our docs all say "don't do that", but, well, it's possibly a bit late by then. Maybe look for a " " in the filename and only shortname if there is one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681183572
@wayneseguin I have linked the PR to the JIRA by creating a "Link" item in the JIRA. You can check the JIRA.
I guess my question was more about: does your fix work with NTFS where short 8.3 character is disabled?
The implementation `~` assumes short 8.3 character is available, which *can* be false if user disables the NTFS feature explicitly for performance reasons. I'm not suggesting we or you should do that, or to force users do that or not do that. I point out the possibility it *may* be disabled. Sorry for previous confusion :)
I'm not a Windows expert (or fan), but quote usually works in Linux/MacOS. So I asked if that is a simpler solution to Windows as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] wayneseguin commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
wayneseguin commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681047087
I did not include tests, since these changes are environmental. Without the patch, Windows launch fails when JAVA_PATH contains a space, and with it it works.
I documented the tests with screenshots in the related Jira, which were the before and after execution screenshots.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681148728
@wayneseguin Thanks for reporting this and fixing this.
One question I have is that, the `~` seems short 8.3 character, see https://support.microsoft.com/en-us/help/121007/how-to-disable-8-3-file-name-creation-on-ntfs-partitions But it can be disabled on NTFS for performance reasons.
Does the double quote work on Windows? e.g.
```
set JAVA_HOME="%JAVA_HOME%"
```
Also, please link the PR to the JIRA manually, if it does not automatically show up there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka edited a comment on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
aajisaka edited a comment on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-689281191
> If it's a no-op instead of a potential bug, then the current code seems fine.
Agreed.
I haven't used Windows for more than 2 years, so I don't have a strong opinion.
Sorry for the late response.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
aajisaka commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-689281191
bq. If it's a no-op instead of a potential bug, then the current code seems fine.
Agreed.
I haven't used Windows for more than 2 years, so I don't have a strong opinion.
Sorry for the late response.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681200251
If it's a no-op instead of a potential bug, then the current code seems fine.
As to which way is better, I think a second opinion will be useful here. @aajisaka
Or if it is not overkill, we apply two approaches together so existing code works and future code using this variable will also work most likely?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-681041182
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 46s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 17s | trunk passed |
| +1 :green_heart: | mvnsite | 1m 32s | trunk passed |
| +1 :green_heart: | shadedclient | 51m 10s | branch has no errors when building and testing our client artifacts. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | the patch passed |
| +1 :green_heart: | mvnsite | 1m 13s | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 15m 56s | patch has no errors when building and testing our client artifacts. |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 59s | hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. |
| | | 77m 20s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2248 |
| Optional Tests | dupname asflicense mvnsite unit |
| uname | Linux a8fab96b3308 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 931adbaa141 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/testReport/ |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] liuml07 commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
liuml07 commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-689290756
Not sure if @goiri or @abmodi is available to give a quick review and commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-731199187
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 1m 8s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 51m 3s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 9s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no whitespace issues. |
| +1 :green_heart: | shadedclient | 14m 47s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 1m 3s | | hadoop-common in the patch passed. |
| -1 :x: | asflicense | 0m 32s | [/patch-asflicense-problems.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/artifact/out/patch-asflicense-problems.txt) | The patch generated 2 ASF License warnings. |
| | | 72m 9s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2248 |
| Optional Tests | dupname asflicense mvnsite unit |
| uname | Linux daaea4e9cfd1 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 747883af4d6 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/testReport/ |
| Max. process+thread count | 612 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2248/1/console |
| versions | git=2.17.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] goiri commented on pull request #2248: HADOOP-17230. Fix JAVA_HOME with spaces
Posted by GitBox <gi...@apache.org>.
goiri commented on pull request #2248:
URL: https://github.com/apache/hadoop/pull/2248#issuecomment-689315668
This looks interesting, I've been having to fight the path limit for a while.
It kind of works, but I get:
```
%%I was unexpected at this time.
```
For my cmd, I actually need:
```
for %I in ("%JAVA_HOME%") do set JAVA_HOME=%%~sI
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org