You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2015/12/04 12:38:11 UTC
[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping
positional read
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15041464#comment-15041464 ]
Hadoop QA commented on HDFS-8901:
---------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m 10s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 9s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 58s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 46s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 55s {color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 47s {color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 1s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 1s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 56s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 27s {color} | {color:red} Patch generated 4 new checkstyle issues in hadoop-hdfs-project (total was 158, now 159). {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 44s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 30s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 49s {color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 41s {color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 8s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 174m 35s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 45s {color} | {color:green} hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 23s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 2m 50s {color} | {color:red} Patch generated 56 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 313m 28s {color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead |
| | hadoop.hdfs.server.namenode.TestDeleteRace |
| | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | hadoop.hdfs.server.datanode.TestCachingStrategy |
| | hadoop.fs.TestSymlinkHdfsFileContext |
| | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
| | hadoop.hdfs.server.datanode.TestBlockReplacement |
| | hadoop.hdfs.security.TestDelegationTokenForProxyUser |
| | hadoop.fs.contract.hdfs.TestHDFSContractSeek |
| | hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.hdfs.server.datanode.TestBlockScanner |
| | hadoop.hdfs.TestWriteReadStripedFile |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| | hadoop.hdfs.TestHFlush |
| | hadoop.hdfs.server.namenode.TestSecurityTokenEditLog |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles |
| | hadoop.hdfs.TestDFSStripedInputStream |
| | hadoop.hdfs.TestParallelShortCircuitLegacyRead |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.hdfs.server.namenode.TestMetaSave |
| | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| JDK v1.8.0_66 Timed out junit tests | org.apache.hadoop.hdfs.TestParallelShortCircuitRead |
| | org.apache.hadoop.hdfs.TestParallelRead |
| | org.apache.hadoop.hdfs.TestPread |
| | org.apache.hadoop.hdfs.TestParallelShortCircuitReadNoChecksum |
| | org.apache.hadoop.hdfs.TestClientReportBadBlock |
| | org.apache.hadoop.hdfs.TestParallelShortCircuitReadUnCached |
| | org.apache.hadoop.hdfs.TestParallelUnixDomainRead |
| JDK v1.7.0_91 Failed junit tests | hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead |
| | hadoop.hdfs.TestDFSClientRetries |
| | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | hadoop.hdfs.security.TestDelegationTokenForProxyUser |
| | hadoop.hdfs.tools.TestDFSAdminWithHA |
| | hadoop.hdfs.qjournal.TestSecureNNWithQJM |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 |
| | hadoop.hdfs.TestDFSUpgradeFromImage |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 |
| | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.hdfs.TestWriteReadStripedFile |
| | hadoop.hdfs.TestQuota |
| | hadoop.hdfs.TestDataTransferKeepalive |
| | hadoop.hdfs.shortcircuit.TestShortCircuitCache |
| | hadoop.hdfs.TestDFSStripedInputStream |
| | hadoop.hdfs.TestParallelShortCircuitLegacyRead |
| | hadoop.hdfs.TestLocalDFS |
| | hadoop.hdfs.TestReadStripedFileWithDecoding |
| | hadoop.hdfs.TestLeaseRecovery |
| JDK v1.7.0_91 Timed out junit tests | org.apache.hadoop.hdfs.TestParallelRead |
| | org.apache.hadoop.hdfs.tools.TestGetConf |
| | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | org.apache.hadoop.hdfs.TestPread |
| | org.apache.hadoop.hdfs.tools.TestGetGroups |
| | org.apache.hadoop.hdfs.TestClientReportBadBlock |
| | org.apache.hadoop.hdfs.TestParallelShortCircuitReadUnCached |
| | org.apache.hadoop.hdfs.tools.TestDFSAdmin |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12775730/HDFS-8901-v3.patch |
| JIRA Issue | HDFS-8901 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux f7979e9dee1a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e84d6ca |
| findbugs | v3.0.0 |
| checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/13767/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt |
| unit | https://builds.apache.org/job/PreCommit-HDFS-Build/13767/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt |
| unit | https://builds.apache.org/job/PreCommit-HDFS-Build/13767/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt |
| unit test logs | https://builds.apache.org/job/PreCommit-HDFS-Build/13767/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13767/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt |
| JDK v1.7.0_91 Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/13767/testReport/ |
| asflicense | https://builds.apache.org/job/PreCommit-HDFS-Build/13767/artifact/patchprocess/patch-asflicense-problems.txt |
| modules | C: hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project |
| Max memory used | 76MB |
| Powered by | Apache Yetus http://yetus.apache.org |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/13767/console |
This message was automatically generated.
> Use ByteBuffer in striping positional read
> ------------------------------------------
>
> Key: HDFS-8901
> URL: https://issues.apache.org/jira/browse/HDFS-8901
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HDFS-8901-v2.patch, HDFS-8901-v3.patch, initial-poc.patch
>
>
> Native erasure coder prefers to direct ByteBuffer for performance consideration. To prepare for it, this change uses ByteBuffer through the codes in implementing striping position read. It will also fix avoiding unnecessary data copying between striping read chunk buffers and decode input buffers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)