You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by GitBox <gi...@apache.org> on 2020/03/24 03:23:40 UTC

[GitHub] [hbase] Apache9 opened a new pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Apache9 opened a new pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603041307
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  9s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 34s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m 10s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 40s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 30s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 10s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   6m  1s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 40s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 40s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 27s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 22s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 22s |  hbase-common in the patch passed.  |
   | -1 :x: |  unit  |   0m 38s |  hbase-procedure in the patch failed.  |
   | -1 :x: |  unit  | 143m  1s |  hbase-server in the patch failed.  |
   |  |   | 178m 26s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux ab724df8276b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bdcfd6a4a8 |
   | Default Java | 1.8.0_232 |
   | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-procedure.txt |
   | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/testReport/ |
   | Max. process+thread count | 4127 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache9 commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-604193880
 
 
   The test is strange, we wrap a already created procedure store with mockito and then call the recoverLease method again. Not sure why but after calling the real method of the procStore, the current stream is still fine but after we returned from the method of the mocked instance, the current stream has already been closed and cause the test to fail.
   
   Since the class is only used for data migration and will be removed in 4.0.0, do not want to spend much time on it. Just catch the FSError exception when closing...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603630659
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   6m 16s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 24s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m  6s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 51s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   3m 24s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 53s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 24s |  hbase-common: The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  11m 55s |  Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.  |
   | +1 :green_heart: |  spotbugs  |   3m 53s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 31s |  The patch does not generate ASF License warnings.  |
   |  |   |  50m  5s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 059d23b13054 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eed730e492 |
   | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt |
   | Max. process+thread count | 83 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
ndimiduk commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603569926
 
 
   `TestWALProcedureStore` passes for me locally. Looks like the test could be written a little more defensively to hand reruns. In this particular case, it seems reruns were blocked because the first invocation didn't delete it's directory, and the same directory path is reused over and over. It's not obvious to me why the first run failed, and there's no additional stack trace depth in the test logs.
   
   ```
     <testcase name="testLogFileAleadExists" classname="org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore" time="0.967">
       <error message="java.io.IOException: Stream Closed" type="org.apache.hadoop.fs.FSError"><![CDATA[org.apache.hadoop.fs.FSError: java.io.IOException: Stream Closed
   	at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.tearDown(TestWALProcedureStore.java:104)
   Caused by: java.io.IOException: Stream Closed
   	at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.tearDown(TestWALProcedureStore.java:104)
   ]]></error>
       <system-err><![CDATA[2020-03-24 03:59:21,445 INFO  [Time-limited test] hbase.ResourceChecker(179): after: procedure2.store.wal.TestWALProcedureStore#testProcIdHoles Thread=8 (was 8), OpenFileDescriptor=89 (was 89), MaxFileDescriptor=1048576 (was 1048576), SystemLoadAverage=1007 (was 1007), ProcessCount=22 (was 22), AvailableMemoryMB=26738 (was 26789)
   2020-03-24 03:59:21,453 INFO  [Time-limited test] hbase.ResourceChecker(151): before: procedure2.store.wal.TestWALProcedureStore#testLogFileAleadExists Thread=8, OpenFileDescriptor=89, MaxFileDescriptor=1048576, SystemLoadAverage=1007, ProcessCount=22, AvailableMemoryMB=26736
   2020-03-24 03:59:21,469 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery
   2020-03-24 03:59:21,476 INFO  [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=1
   2020-03-24 03:59:21,477 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=1
   2020-03-24 03:59:21,477 DEBUG [Time-limited test] wal.WALProcedureStore(454): No state logs to replay.
   2020-03-24 03:59:21,482 INFO  [WALProcedureStoreSyncThread] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=2
   2020-03-24 03:59:21,483 INFO  [WALProcedureStoreSyncThread] wal.WALProcedureStore(1221): Remove all state logs with ID less than 1, since no active procedures
   2020-03-24 03:59:21,483 DEBUG [WALProcedureStoreSyncThread] wal.WALProcedureStore(1243): Removed log=/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/c0fb2ae6-9b4d-e64a-6ae8-b8746f2e897c/proc-logs/pv2-00000000000000000001.log, activeLogs=[/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/c0fb2ae6-9b4d-e64a-6ae8-b8746f2e897c/proc-logs/pv2-00000000000000000002.log]
   2020-03-24 03:59:22,393 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery
   2020-03-24 03:59:22,395 WARN  [Time-limited test] wal.WALProcedureStore(1370): Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/c0fb2ae6-9b4d-e64a-6ae8-b8746f2e897c/proc-logs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1585022361000; access_time=1585022361474; owner=; group=; permission=rw-rw-rw-; isSymlink=false}
   2020-03-24 03:59:22,402 INFO  [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=3
   2020-03-24 03:59:22,403 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=3
   2020-03-24 03:59:22,403 INFO  [Time-limited test] wal.WALProcedureStore(330): Stopping the WAL Procedure Store, isAbort=false
   ]]></system-err>
       <rerunError message="java.io.IOException: Stream Closed" type="org.apache.hadoop.fs.FSError">
         <stackTrace><![CDATA[org.apache.hadoop.fs.FSError: java.io.IOException: Stream Closed
   	at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687)
   Caused by: java.io.IOException: Stream Closed
   	at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687)
   ]]></stackTrace>
         <system-err><![CDATA[2020-03-24 03:59:24,459 INFO  [Time-limited test] hbase.ResourceChecker(151): before: procedure2.store.wal.TestWALProcedureStore#testLogFileAleadExists Thread=9, OpenFileDescriptor=91, MaxFileDescriptor=1048576, SystemLoadAverage=1054, ProcessCount=22, AvailableMemoryMB=26379
   2020-03-24 03:59:24,466 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery
   2020-03-24 03:59:24,468 INFO  [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=1
   2020-03-24 03:59:24,469 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=1
   2020-03-24 03:59:24,469 DEBUG [Time-limited test] wal.WALProcedureStore(454): No state logs to replay.
   2020-03-24 03:59:24,480 INFO  [WALProcedureStoreSyncThread] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=2
   2020-03-24 03:59:24,480 INFO  [WALProcedureStoreSyncThread] wal.WALProcedureStore(1221): Remove all state logs with ID less than 1, since no active procedures
   2020-03-24 03:59:24,481 DEBUG [WALProcedureStoreSyncThread] wal.WALProcedureStore(1243): Removed log=/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/8f7fbc71-2e3e-2dbf-99cd-0c61da4671c4/proc-logs/pv2-00000000000000000001.log, activeLogs=[/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/8f7fbc71-2e3e-2dbf-99cd-0c61da4671c4/proc-logs/pv2-00000000000000000002.log]
   2020-03-24 03:59:24,481 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery
   2020-03-24 03:59:24,482 WARN  [Time-limited test] wal.WALProcedureStore(1370): Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/8f7fbc71-2e3e-2dbf-99cd-0c61da4671c4/proc-logs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1585022364000; access_time=1585022364466; owner=; group=; permission=rw-rw-rw-; isSymlink=false}
   2020-03-24 03:59:24,486 INFO  [Time-limited test] wal.WALProcedureStore(330): Stopping the WAL Procedure Store, isAbort=false
   ]]></system-err>
       </rerunError>
       <rerunError message="java.io.IOException: Stream Closed" type="org.apache.hadoop.fs.FSError">
         <stackTrace><![CDATA[org.apache.hadoop.fs.FSError: java.io.IOException: Stream Closed
   	at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687)
   Caused by: java.io.IOException: Stream Closed
   	at org.apache.hadoop.hbase.procedure2.store.wal.TestWALProcedureStore.testLogFileAleadExists(TestWALProcedureStore.java:687)
   ]]></stackTrace>
         <system-err><![CDATA[2020-03-24 03:59:24,545 INFO  [Time-limited test] hbase.ResourceChecker(151): before: procedure2.store.wal.TestWALProcedureStore#testLogFileAleadExists Thread=9, OpenFileDescriptor=92, MaxFileDescriptor=1048576, SystemLoadAverage=1054, ProcessCount=21, AvailableMemoryMB=26507
   2020-03-24 03:59:24,553 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery
   2020-03-24 03:59:24,556 INFO  [Time-limited test] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=1
   2020-03-24 03:59:24,556 DEBUG [Time-limited test] wal.WALProcedureStore(436): Lease acquired for flushLogId=1
   2020-03-24 03:59:24,556 DEBUG [Time-limited test] wal.WALProcedureStore(454): No state logs to replay.
   2020-03-24 03:59:24,559 INFO  [WALProcedureStoreSyncThread] wal.WALProcedureStore(1134): Rolled new Procedure Store WAL, id=2
   2020-03-24 03:59:24,560 INFO  [WALProcedureStoreSyncThread] wal.WALProcedureStore(1221): Remove all state logs with ID less than 1, since no active procedures
   2020-03-24 03:59:24,561 DEBUG [WALProcedureStoreSyncThread] wal.WALProcedureStore(1243): Removed log=/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/dd76aeb2-fe62-be76-3273-d13f409d446f/proc-logs/pv2-00000000000000000001.log, activeLogs=[/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/dd76aeb2-fe62-be76-3273-d13f409d446f/proc-logs/pv2-00000000000000000002.log]
   2020-03-24 03:59:24,561 DEBUG [Time-limited test] wal.WALProcedureStore(401): Starting WAL Procedure Store lease recovery
   2020-03-24 03:59:24,561 WARN  [Time-limited test] wal.WALProcedureStore(1370): Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-procedure/target/test-data/dd76aeb2-fe62-be76-3273-d13f409d446f/proc-logs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1585022364000; access_time=1585022364554; owner=; group=; permission=rw-rw-rw-; isSymlink=false}
   2020-03-24 03:59:24,567 INFO  [Time-limited test] wal.WALProcedureStore(330): Stopping the WAL Procedure Store, isAbort=false
   ]]></system-err>
       </rerunError>
     </testcase>
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 merged pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache9 merged pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache9 commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603725007
 
 
   Seems TestWALProcedureStore is broken... Anyway let me see how to fix it...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-604231759
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 23s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 17s |  master passed  |
   | +1 :green_heart: |  compile  |   2m  2s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 27s |  branch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 17s |  hbase-common in master failed.  |
   | -0 :warning: |  javadoc  |   0m 16s |  hbase-procedure in master failed.  |
   | -0 :warning: |  javadoc  |   0m 41s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 14s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   7m  1s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 57s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 57s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 32s |  patch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 16s |  hbase-common in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 16s |  hbase-procedure in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 40s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 25s |  hbase-common in the patch passed.  |
   | +1 :green_heart: |  unit  |   1m 36s |  hbase-procedure in the patch passed.  |
   | +1 :green_heart: |  unit  | 100m 43s |  hbase-server in the patch passed.  |
   |  |   | 140m 47s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 26e338b4e269 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 76e3db6720 |
   | Default Java | 2020-01-14 |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   |  Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/testReport/ |
   | Max. process+thread count | 4150 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache9 commented on a change in pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache9 commented on a change in pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#discussion_r397588653
 
 

 ##########
 File path: hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java
 ##########
 @@ -759,200 +739,75 @@ public static void checkShortCircuitReadBufferSize(final Configuration conf) {
     conf.setIfUnset(dfsKey, Integer.toString(hbaseSize));
   }
 
-  private static class DfsBuilderUtility {
-    static Class<?> dfsClass = null;
-    static Method createMethod;
-    static Method overwriteMethod;
-    static Method bufferSizeMethod;
-    static Method blockSizeMethod;
-    static Method recursiveMethod;
-    static Method replicateMethod;
-    static Method replicationMethod;
-    static Method buildMethod;
-    static boolean allMethodsPresent = false;
+  private static final class DfsBuilderUtility {
+    private static final Class<?> BUILDER;
+    private static final Method REPLICATE;
 
     static {
-      String dfsName = "org.apache.hadoop.hdfs.DistributedFileSystem";
-      String builderName = dfsName + "$HdfsDataOutputStreamBuilder";
+      String builderName = "org.apache.hadoop.hdfs.DistributedFileSystem$HdfsDataOutputStreamBuilder";
 
 Review comment:
   Yes, hbase-common only depends on hadoop-common so we can only access FileSystem, not DistributedFileSystem. I tried to move this method to hbase-server but then I realized that we also use this method in hbase-procedure, so leave it as is for now. The WALProcedureStore is supposed to be removed in 4.0.0, let's revive this again at that time...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603657374
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   6m 36s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m 40s |  master passed  |
   | +1 :green_heart: |  compile  |   2m  7s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m 38s |  branch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 18s |  hbase-common in master failed.  |
   | -0 :warning: |  javadoc  |   0m 17s |  hbase-procedure in master failed.  |
   | -0 :warning: |  javadoc  |   0m 42s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 17s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   7m  5s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 59s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 59s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   6m 41s |  patch has no errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 17s |  hbase-common in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 16s |  hbase-procedure in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 41s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 31s |  hbase-common in the patch passed.  |
   | +1 :green_heart: |  unit  |   1m 37s |  hbase-procedure in the patch passed.  |
   | +1 :green_heart: |  unit  | 110m  2s |  hbase-server in the patch passed.  |
   |  |   | 157m 19s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux ac1fbbdc10dd 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eed730e492 |
   | Default Java | 2020-01-14 |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   |  Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/testReport/ |
   | Max. process+thread count | 4171 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#discussion_r397525621
 
 

 ##########
 File path: hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java
 ##########
 @@ -525,10 +518,7 @@ private static void invokeSetStoragePolicy(final FileSystem fs, final Path path,
 
     try {
       fs.setStoragePolicy(path, storagePolicy);
-
-      if (LOG.isDebugEnabled()) {
-        LOG.debug("Set storagePolicy={} for path={}", storagePolicy, path);
-      }
+      LOG.debug("Set storagePolicy={} for path={}", storagePolicy, path);
 
 Review comment:
   nit: `path` is not an instance of `String`, so auto-type-conversion means this will call `path.toString()` even when debug is not enabled. That method looks entirely too complicated, with lots of branching, but I guess nothing too hateful. Just and FYI.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
ndimiduk commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603571315
 
 
   `TestWALEntryStream` looks like it read partial data,
   
   ```
   2020-03-24 05:12:43,671 INFO  [Listener at localhost/38169] wal.AbstractFSWAL(756): Rolled WAL /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-server/target/test-data/8b31e9c5-e403-a068-d9ff-fc56f5c00237/WALs/testDifferentCounts/testDifferentCounts.1585026756453 with entries=1500, filesize=184.53 KB; new WAL /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1335/yetus-jdk8-hadoop2-check/src/hbase-server/target/test-data/8b31e9c5-e403-a068-d9ff-fc56f5c00237/WALs/testDifferentCounts/testDifferentCounts.1585026763520
   2020-03-24 05:12:43,672 DEBUG [Listener at localhost/38169] wal.AbstractFSWAL(845): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34733,DS-4271f034-e4f9-408c-9869-e101eb00f24c,DISK], DatanodeInfoWithStorage[127.0.0.1:43925,DS-4f6145c8-5eca-4ff4-afe6-b84495802210,DISK], DatanodeInfoWithStorage[127.0.0.1:40637,DS-e9928388-d36b-42de-bb16-1925e01cd86f,DISK]]
   2020-03-24 05:12:44,270 WARN  [Listener at localhost/38169] wal.ProtobufLogReader(426): Encountered a malformed edit, seeking back to last good position in file, from 188958 to 188957
   java.io.EOFException: Partial PB while reading WAL, probably an unexpected EOF, ignoring. current offset=188958
   	at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:369)
   	at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:98)
   	at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:86)
   	at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.readNextEntryAndRecordReaderPosition(WALEntryStream.java:263)
   	at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:176)
   	at org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:101)
   	at org.apache.hadoop.hbase.replication.regionserver.TestWALEntryStream.testDifferentCounts(TestWALEntryStream.java:169)
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-604207566
 
 
   :confetti_ball: **+1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 28s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  1s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 36s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   5m 36s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 47s |  master passed  |
   | +1 :green_heart: |  spotbugs  |   3m 13s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 22s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 22s |  hbase-common: The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  0s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  11m  0s |  Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.  |
   | +1 :green_heart: |  spotbugs  |   3m 42s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 40s |  The patch does not generate ASF License warnings.  |
   |  |   |  41m 56s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 0fb88ac35eea 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 76e3db6720 |
   | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt |
   | Max. process+thread count | 93 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] ndimiduk commented on a change in pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
ndimiduk commented on a change in pull request #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#discussion_r397528874
 
 

 ##########
 File path: hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java
 ##########
 @@ -759,200 +739,75 @@ public static void checkShortCircuitReadBufferSize(final Configuration conf) {
     conf.setIfUnset(dfsKey, Integer.toString(hbaseSize));
   }
 
-  private static class DfsBuilderUtility {
-    static Class<?> dfsClass = null;
-    static Method createMethod;
-    static Method overwriteMethod;
-    static Method bufferSizeMethod;
-    static Method blockSizeMethod;
-    static Method recursiveMethod;
-    static Method replicateMethod;
-    static Method replicationMethod;
-    static Method buildMethod;
-    static boolean allMethodsPresent = false;
+  private static final class DfsBuilderUtility {
+    private static final Class<?> BUILDER;
+    private static final Method REPLICATE;
 
     static {
-      String dfsName = "org.apache.hadoop.hdfs.DistributedFileSystem";
-      String builderName = dfsName + "$HdfsDataOutputStreamBuilder";
+      String builderName = "org.apache.hadoop.hdfs.DistributedFileSystem$HdfsDataOutputStreamBuilder";
 
 Review comment:
   `DistributedFileSystem` appears to exist since 2.10, so can this string be built from a class reference? For example,
   
   ```java
   String buildName = DistributedFileSystem.class.getName() + "$HdfsDataOutputStreamBuilder";
   ```
   
   Looks like hbase-common module doesn't depend on hadoop-hdfs-client jar, so maybe this gets too weird for a "cleanup" patch. Would be nice though, to use whatever static references we can...

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603653195
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   3m 20s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 30s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   7m  4s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 51s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   6m  9s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 19s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 15s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   6m 44s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m  2s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m  2s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 58s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 27s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 19s |  hbase-common in the patch passed.  |
   | -1 :x: |  unit  |   0m 41s |  hbase-procedure in the patch failed.  |
   | +1 :green_heart: |  unit  | 102m  0s |  hbase-server in the patch passed.  |
   |  |   | 142m 46s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux ce163234bf1b 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / eed730e492 |
   | Default Java | 1.8.0_232 |
   | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-procedure.txt |
   |  Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/testReport/ |
   | Max. process+thread count | 4162 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/2/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603040137
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   0m 39s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  4s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 41s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |  10m  7s |  master passed  |
   | +1 :green_heart: |  compile  |   2m 58s |  master passed  |
   | -1 :x: |  shadedjars  |   0m 13s |  branch has 7 errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 25s |  hbase-common in master failed.  |
   | -0 :warning: |  javadoc  |   0m 21s |  hbase-procedure in master failed.  |
   | -0 :warning: |  javadoc  |   0m 53s |  hbase-server in master failed.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 20s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   9m 56s |  the patch passed  |
   | +1 :green_heart: |  compile  |   2m 56s |  the patch passed  |
   | +1 :green_heart: |  javac  |   2m 56s |  the patch passed  |
   | -1 :x: |  shadedjars  |   0m 15s |  patch has 7 errors when building our shaded downstream artifacts.  |
   | -0 :warning: |  javadoc  |   0m 27s |  hbase-common in the patch failed.  |
   | -0 :warning: |  javadoc  |   0m 29s |  hbase-procedure in the patch failed.  |
   | -0 :warning: |  javadoc  |   1m  9s |  hbase-server in the patch failed.  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   2m 12s |  hbase-common in the patch passed.  |
   | +1 :green_heart: |  unit  |   2m 27s |  hbase-procedure in the patch passed.  |
   | +1 :green_heart: |  unit  | 135m 52s |  hbase-server in the patch passed.  |
   |  |   | 174m 45s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 5024cb9be257 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bdcfd6a4a8 |
   | Default Java | 2020-01-14 |
   | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-common.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt |
   | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-common.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt |
   | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt |
   |  Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/testReport/ |
   | Max. process+thread count | 3970 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-604210063
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m  8s |  Docker mode activated.  |
   | -0 :warning: |  yetus  |   0m  3s |  Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck  |
   ||| _ Prechecks _ |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 23s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m  2s |  master passed  |
   | +1 :green_heart: |  compile  |   1m 43s |  master passed  |
   | +1 :green_heart: |  shadedjars  |   5m 29s |  branch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m 11s |  master passed  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 13s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 55s |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 39s |  the patch passed  |
   | +1 :green_heart: |  javac  |   1m 39s |  the patch passed  |
   | +1 :green_heart: |  shadedjars  |   5m 27s |  patch has no errors when building our shaded downstream artifacts.  |
   | +1 :green_heart: |  javadoc  |   1m  8s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  unit  |   1m 13s |  hbase-common in the patch passed.  |
   | +1 :green_heart: |  unit  |   1m 41s |  hbase-procedure in the patch passed.  |
   | -1 :x: |  unit  |  16m 59s |  hbase-server in the patch failed.  |
   |  |   |  51m 50s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | javac javadoc unit shadedjars compile |
   | uname | Linux 4b00781154a9 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / 76e3db6720 |
   | Default Java | 1.8.0_232 |
   | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt |
   |  Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/testReport/ |
   | Max. process+thread count | 916 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/3/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [hbase] Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0

Posted by GitBox <gi...@apache.org>.
Apache-HBase commented on issue #1335: HBASE-24000 Simplify CommonFSUtils after upgrading to hadoop 2.10.0
URL: https://github.com/apache/hbase/pull/1335#issuecomment-603003398
 
 
   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime | Comment |
   |:----:|----------:|--------:|:--------|
   | +0 :ok: |  reexec  |   1m 10s |  Docker mode activated.  |
   ||| _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  No case conflicting files found.  |
   | +1 :green_heart: |  hbaseanti  |   0m  0s |  Patch does not have any anti-patterns.  |
   | +1 :green_heart: |  @author  |   0m  0s |  The patch does not contain any @author tags.  |
   ||| _ master Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 33s |  Maven dependency ordering for branch  |
   | +1 :green_heart: |  mvninstall  |   6m  6s |  master passed  |
   | +1 :green_heart: |  checkstyle  |   1m 52s |  master passed  |
   | -1 :x: |  spotbugs  |   2m  8s |  hbase-server in master has 1 extant spotbugs warnings.  |
   ||| _ Patch Compile Tests _ |
   | +0 :ok: |  mvndep  |   0m 12s |  Maven dependency ordering for patch  |
   | +1 :green_heart: |  mvninstall  |   5m 52s |  the patch passed  |
   | -0 :warning: |  checkstyle  |   0m 23s |  hbase-common: The patch generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0)  |
   | +1 :green_heart: |  whitespace  |   0m  1s |  The patch has no whitespace issues.  |
   | +1 :green_heart: |  hadoopcheck  |  12m 28s |  Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2.  |
   | +1 :green_heart: |  spotbugs  |   4m 13s |  the patch passed  |
   ||| _ Other Tests _ |
   | +1 :green_heart: |  asflicense  |   0m 31s |  The patch does not generate ASF License warnings.  |
   |  |   |  46m 24s |   |
   
   
   | Subsystem | Report/Notes |
   |----------:|:-------------|
   | Docker | Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-general-check/output/Dockerfile |
   | GITHUB PR | https://github.com/apache/hbase/pull/1335 |
   | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle |
   | uname | Linux 70e73e676523 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/hbase-personality.sh |
   | git revision | master / bdcfd6a4a8 |
   | spotbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html |
   | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-common.txt |
   | Max. process+thread count | 83 (vs. ulimit of 10000) |
   | modules | C: hbase-common hbase-procedure hbase-server U: . |
   | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1335/1/console |
   | versions | git=2.17.1 maven=2018-06-17T18:33:14Z) spotbugs=3.1.12 |
   | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org |
   
   
   This message was automatically generated.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services