You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2021/11/12 16:46:39 UTC
[GitHub] [hbase] wchevreuil opened a new pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
wchevreuil opened a new pull request #3845:
URL: https://github.com/apache/hbase/pull/3845
… when creating table FS layout
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-971792927
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 53s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 16s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 36s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 10m 47s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 1m 3s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 38s | the patch passed |
| +1 :green_heart: | javac | 1m 38s | the patch passed |
| +1 :green_heart: | shadedjars | 11m 14s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 50s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 218m 37s | hbase-server in the patch failed. |
| | | 261m 39s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 47bcb02dc0f7 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/testReport/ |
| Max. process+thread count | 3256 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil edited a comment on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil edited a comment on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970195137
> The failed UT is related?
>
> TestMasterFileSystem.testCheckTempDir
Yes, it's failing because it can't rename tabledir to temp:
...
2021-11-16T11:45:56,209 WARN [IPC Server handler 3 on 50183] namenode.FSDirRenameOp(182): DIR* FSDirectory.unprotectedRenameTo: failed to rename /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/data/default/testCheckTempDir to /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/.tmp/data/default/testCheckTempDir because destination's parent does not exist
...
I think this UT is invalid now that we don't need table temp dir. I'm gonna change it to reflect the current behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970657937
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 27s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 29s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 8s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 8m 47s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 10s | the patch passed |
| +1 :green_heart: | compile | 1m 7s | the patch passed |
| +1 :green_heart: | javac | 1m 7s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 49s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 167m 22s | hbase-server in the patch passed. |
| | | 199m 53s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 1a65de5b5288 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/testReport/ |
| Max. process+thread count | 4944 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on a change in pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#discussion_r752265249
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SnapshotScannerHDFSAclHelper.java
##########
@@ -474,8 +474,8 @@ void createTableDirectories(TableName tableName) throws IOException {
*/
List<Path> getTableRootPaths(TableName tableName, boolean includeSnapshotPath)
throws IOException {
- List<Path> paths = Lists.newArrayList(pathHelper.getTmpTableDir(tableName),
Review comment:
Found out this was creating table tmp dir on CreateTableProcedure.CREATE_TABLE_POST_OPERATION when ACL is enabled,
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SnapshotScannerHDFSAclHelper.java
##########
@@ -474,8 +474,8 @@ void createTableDirectories(TableName tableName) throws IOException {
*/
List<Path> getTableRootPaths(TableName tableName, boolean includeSnapshotPath)
throws IOException {
- List<Path> paths = Lists.newArrayList(pathHelper.getTmpTableDir(tableName),
Review comment:
Found out this was creating table tmp dir on CreateTableProcedure.CREATE_TABLE_POST_OPERATION when ACL is enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970372604
> > The failed UT is related?
> > TestMasterFileSystem.testCheckTempDir
>
> Yes, it's failing because it can't rename tabledir to temp:
>
> ... 2021-11-16T11:45:56,209 WARN [IPC Server handler 3 on 50183] namenode.FSDirRenameOp(182): DIR* FSDirectory.unprotectedRenameTo: failed to rename /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/data/default/testCheckTempDir to /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/.tmp/data/default/testCheckTempDir because destination's parent does not exist ...
>
> I think this UT is invalid now that we don't need table temp dir. I'm gonna change it to reflect the current behaviour.
Checked the usage, there are 3 places where we use it, in CreateTableProcedure, CloneSnapshotProcedure and DeleteTableProcedure.deleteFromFs. Since now we do not use temp directory for all these procedures any more, I think we could just remove this temp dir in MasterFileSystem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-973152603
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 57s | Docker mode activated. |
| -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 25s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 16s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 9m 55s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 44s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 46s | the patch passed |
| +1 :green_heart: | compile | 1m 14s | the patch passed |
| +1 :green_heart: | javac | 1m 14s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 58s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 247m 29s | hbase-server in the patch passed. |
| | | 285m 49s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 441d9e70c035 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/testReport/ |
| Max. process+thread count | 3779 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-974022690
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | patch | 0m 4s | https://github.com/apache/hbase/pull/3845 does not apply to HBASE-26067. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/5/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#discussion_r748671886
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
##########
@@ -336,41 +335,21 @@ private void postCreate(final MasterProcedureEnv env)
final TableDescriptor tableDescriptor, List<RegionInfo> newRegions,
final CreateHdfsRegions hdfsRegionHandler) throws IOException {
final MasterFileSystem mfs = env.getMasterServices().getMasterFileSystem();
- final Path tempdir = mfs.getTempDir();
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
- final Path tempTableDir = CommonFSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ final Path tableDir = CommonFSUtils.getTableDir(mfs.getRootDir(), tableDescriptor.getTableName());
Review comment:
We do not need to delete or clean the old tableDir if exists first?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970168392
The failed UT is related?
TestMasterFileSystem.testCheckTempDir
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970196967
> > > Do we need to implement rollback where we will delete the partial table directory?
> >
> >
> > We already delete pre-existing table dir on line on line #104.
>
> This is not the same thing, it is not about retry but about how to clean up the storage when the CreateTableProcedure fails.
>
> But checked the code, we do not support rollback when we already arrive the WRITE_FS_LAYOUT state, so it should not be a problem.
Yeah, thought about adding a rollback logic here, but then thought it wouldn't be necessary as we do the check/cleanup every time in the beginning of the proc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on a change in pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#discussion_r750496366
##########
File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFileSystem.java
##########
@@ -110,28 +108,13 @@ public void testCheckTempDir() throws Exception {
// disable the table so that we can manipulate the files
UTIL.getAdmin().disableTable(tableName);
- final Path tableDir = CommonFSUtils.getTableDir(masterFileSystem.getRootDir(), tableName);
final Path tempDir = masterFileSystem.getTempDir();
- final Path tempTableDir = CommonFSUtils.getTableDir(tempDir, tableName);
+ final Path tempNsDir = CommonFSUtils.getNamespaceDir(tempDir,
+ tableName.getNamespaceAsString());
final FileSystem fs = masterFileSystem.getFileSystem();
- // move the table to the temporary directory
- if (!fs.rename(tableDir, tempTableDir)) {
- fail();
- }
-
- masterFileSystem.checkTempDir(tempDir, UTIL.getConfiguration(), fs);
-
- // check if the temporary directory exists and is empty
- assertTrue(fs.exists(tempDir));
- assertEquals(0, fs.listStatus(tempDir).length);
-
- // check for the existence of the archive directory
- for (HRegion region : regions) {
- Path archiveDir = HFileArchiveTestingUtil.getRegionArchiveDir(UTIL.getConfiguration(),
- region);
- assertTrue(fs.exists(archiveDir));
- }
+ // checks the temporary directory does not exist
+ assertFalse(fs.exists(tempNsDir));
Review comment:
We now make sure no temp dir was created for table namespace during the CreateTableProcedure run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-971815639
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 3m 8s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 9s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 13s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 9m 56s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 45s | the patch passed |
| +1 :green_heart: | compile | 1m 14s | the patch passed |
| +1 :green_heart: | javac | 1m 14s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 52s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 250m 56s | hbase-server in the patch failed. |
| | | 289m 45s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux ff985926c1b2 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/testReport/ |
| Max. process+thread count | 3741 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-972931504
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 5s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 58s | HBASE-26067 passed |
| +1 :green_heart: | compile | 3m 11s | HBASE-26067 passed |
| +1 :green_heart: | checkstyle | 1m 3s | HBASE-26067 passed |
| +1 :green_heart: | spotbugs | 2m 8s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 46s | the patch passed |
| +1 :green_heart: | compile | 3m 10s | the patch passed |
| +1 :green_heart: | javac | 3m 10s | the patch passed |
| -0 :warning: | checkstyle | 1m 6s | hbase-server: The patch generated 13 new + 0 unchanged - 0 fixed = 13 total (was 0) |
| -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | hadoopcheck | 19m 24s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 17s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. |
| | | 49m 26s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 5ad8675cd2bb 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/artifact/yetus-general-check/output/whitespace-eol.txt |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970155959
> > Do we need to implement rollback where we will delete the partial table directory?
>
> We already delete pre-existing table dir on line on line #104.
This is not the same thing, it is not about retry but about how to clean up the storage when the CreateTableProcedure fails.
But checked the code, we do not support rollback when we already arrive the WRITE_FS_LAYOUT state, so it should not be a problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-967543569
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 19s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 21s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 3s | the patch passed |
| +1 :green_heart: | compile | 1m 19s | the patch passed |
| +1 :green_heart: | javac | 1m 19s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 8s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 215m 15s | hbase-server in the patch failed. |
| | | 251m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 95a590ed7572 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/testReport/ |
| Max. process+thread count | 2969 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970495201
> > > The failed UT is related?
> > > TestMasterFileSystem.testCheckTempDir
> >
> >
> > Yes, it's failing because it can't rename tabledir to temp:
> > ... 2021-11-16T11:45:56,209 WARN [IPC Server handler 3 on 50183] namenode.FSDirRenameOp(182): DIR* FSDirectory.unprotectedRenameTo: failed to rename /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/data/default/testCheckTempDir to /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/.tmp/data/default/testCheckTempDir because destination's parent does not exist ...
> > I think this UT is invalid now that we don't need table temp dir. I'm gonna change it to reflect the current behaviour.
>
> Checked the usage, there are 3 places where we use it, in CreateTableProcedure, CloneSnapshotProcedure and DeleteTableProcedure.deleteFromFs. Since now we do not use temp directory for all these procedures any more, I think we could just remove this temp dir in MasterFileSystem?
I think DeleteTableProcedure.deleteFromFs is still relying on temp dir. Let me change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil merged pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil merged pull request #3845:
URL: https://github.com/apache/hbase/pull/3845
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#discussion_r750149466
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
##########
@@ -336,41 +335,21 @@ private void postCreate(final MasterProcedureEnv env)
final TableDescriptor tableDescriptor, List<RegionInfo> newRegions,
final CreateHdfsRegions hdfsRegionHandler) throws IOException {
final MasterFileSystem mfs = env.getMasterServices().getMasterFileSystem();
- final Path tempdir = mfs.getTempDir();
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
- final Path tempTableDir = CommonFSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ final Path tableDir = CommonFSUtils.getTableDir(mfs.getRootDir(), tableDescriptor.getTableName());
Review comment:
OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970152288
> Do we need to implement rollback where we will delete the partial table directory?
We already delete pre-existing table dir on line on line #104.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-971605624
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 4m 17s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 16s | HBASE-26067 passed |
| +1 :green_heart: | compile | 3m 19s | HBASE-26067 passed |
| +1 :green_heart: | checkstyle | 1m 7s | HBASE-26067 passed |
| +1 :green_heart: | spotbugs | 2m 28s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 55s | the patch passed |
| +1 :green_heart: | compile | 3m 18s | the patch passed |
| +1 :green_heart: | javac | 3m 18s | the patch passed |
| -0 :warning: | checkstyle | 1m 15s | hbase-server: The patch generated 13 new + 0 unchanged - 0 fixed = 13 total (was 0) |
| -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | hadoopcheck | 21m 0s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 41s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. |
| | | 56m 0s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fc60184b359d 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/artifact/yetus-general-check/output/whitespace-eol.txt |
| Max. process+thread count | 95 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/3/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-974022885
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on a change in pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#discussion_r750145312
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
##########
@@ -336,41 +335,21 @@ private void postCreate(final MasterProcedureEnv env)
final TableDescriptor tableDescriptor, List<RegionInfo> newRegions,
final CreateHdfsRegions hdfsRegionHandler) throws IOException {
final MasterFileSystem mfs = env.getMasterServices().getMasterFileSystem();
- final Path tempdir = mfs.getTempDir();
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
- final Path tempTableDir = CommonFSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ final Path tableDir = CommonFSUtils.getTableDir(mfs.getRootDir(), tableDescriptor.getTableName());
Review comment:
We alerady call `DeleteTableProcedure.deleteFromFs(env, getTableName(), newRegions, true)` on line #104 before we call `createFsLayout`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-973123279
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 13s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 19s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 43s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 2s | the patch passed |
| +1 :green_heart: | compile | 1m 19s | the patch passed |
| +1 :green_heart: | javac | 1m 19s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 42s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 212m 26s | hbase-server in the patch passed. |
| | | 248m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux d449db9f1636 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/testReport/ |
| Max. process+thread count | 2939 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/4/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-967554391
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 19s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 41s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 5s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 9m 15s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 40s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 26s | the patch passed |
| +1 :green_heart: | compile | 1m 4s | the patch passed |
| +1 :green_heart: | javac | 1m 4s | the patch passed |
| +1 :green_heart: | shadedjars | 9m 7s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 37s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 223m 27s | hbase-server in the patch failed. |
| | | 257m 47s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 7444e9421c12 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/testReport/ |
| Max. process+thread count | 3246 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-967297506
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 6s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 33s | HBASE-26067 passed |
| +1 :green_heart: | compile | 3m 29s | HBASE-26067 passed |
| +1 :green_heart: | checkstyle | 1m 11s | HBASE-26067 passed |
| +1 :green_heart: | spotbugs | 2m 15s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 14s | the patch passed |
| +1 :green_heart: | compile | 3m 24s | the patch passed |
| +1 :green_heart: | javac | 3m 24s | the patch passed |
| -0 :warning: | checkstyle | 1m 12s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 21m 36s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 23s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. |
| | | 54m 28s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6881b11b84d0 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 86 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/1/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970533430
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 29s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 25s | HBASE-26067 passed |
| +1 :green_heart: | compile | 3m 32s | HBASE-26067 passed |
| +1 :green_heart: | checkstyle | 1m 5s | HBASE-26067 passed |
| +1 :green_heart: | spotbugs | 2m 14s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 3s | the patch passed |
| +1 :green_heart: | compile | 3m 32s | the patch passed |
| +1 :green_heart: | javac | 3m 32s | the patch passed |
| -0 :warning: | checkstyle | 1m 11s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | hadoopcheck | 21m 42s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. |
| +1 :green_heart: | spotbugs | 2m 27s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. |
| | | 55m 4s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile |
| uname | Linux ff9ff720354f 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-1.8.0_282-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt |
| Max. process+thread count | 96 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/console |
| versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on a change in pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#discussion_r750145312
##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/CreateTableProcedure.java
##########
@@ -336,41 +335,21 @@ private void postCreate(final MasterProcedureEnv env)
final TableDescriptor tableDescriptor, List<RegionInfo> newRegions,
final CreateHdfsRegions hdfsRegionHandler) throws IOException {
final MasterFileSystem mfs = env.getMasterServices().getMasterFileSystem();
- final Path tempdir = mfs.getTempDir();
// 1. Create Table Descriptor
// using a copy of descriptor, table will be created enabling first
- final Path tempTableDir = CommonFSUtils.getTableDir(tempdir, tableDescriptor.getTableName());
+ final Path tableDir = CommonFSUtils.getTableDir(mfs.getRootDir(), tableDescriptor.getTableName());
Review comment:
We already call `DeleteTableProcedure.deleteFromFs(env, getTableName(), newRegions, true)` on line #104 before we call `createFsLayout`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970646140
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 12s | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck |
||| _ Prechecks _ |
||| _ HBASE-26067 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 51s | HBASE-26067 passed |
| +1 :green_heart: | compile | 1m 15s | HBASE-26067 passed |
| +1 :green_heart: | shadedjars | 8m 22s | branch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 46s | HBASE-26067 passed |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 35s | the patch passed |
| +1 :green_heart: | compile | 1m 13s | the patch passed |
| +1 :green_heart: | javac | 1m 13s | the patch passed |
| +1 :green_heart: | shadedjars | 8m 12s | patch has no errors when building our shaded downstream artifacts. |
| +1 :green_heart: | javadoc | 0m 41s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 150m 24s | hbase-server in the patch passed. |
| | | 183m 41s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/hbase/pull/3845 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux af54e9b41723 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-26067 / 36b6088a26 |
| Default Java | AdoptOpenJDK-11.0.10+9 |
| Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/testReport/ |
| Max. process+thread count | 4098 (vs. ulimit of 30000) |
| modules | C: hbase-server U: hbase-server |
| Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3845/2/console |
| versions | git=2.17.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hbase] wchevreuil commented on pull request #3845: HBASE-26454 CreateTableProcedure still relies on temp dir and renames…
Posted by GitBox <gi...@apache.org>.
wchevreuil commented on pull request #3845:
URL: https://github.com/apache/hbase/pull/3845#issuecomment-970195137
> The failed UT is related?
>
> TestMasterFileSystem.testCheckTempDir
Yes, it's failing because it can't rename tabledir to temp:
...
2021-11-16T11:45:56,209 WARN [IPC Server handler 3 on 50183] namenode.FSDirRenameOp(182): DIR* FSDirectory.unprotectedRenameTo: failed to rename /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/data/default/testCheckTempDir to /user/wellingtonchevreuil/test-data/577b1635-a2e9-6d2d-0e93-e72c07fd3ba7/.tmp/data/default/testCheckTempDir because destination's parent does not exist
...
I think this UT is invalid now that we don't need table temp dir.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org