You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2017/07/09 08:48:01 UTC
[jira] [Commented] (HADOOP-14520) WASB: Block compaction for Azure
Block Blobs
[ https://issues.apache.org/jira/browse/HADOOP-14520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16079508#comment-16079508 ]
Hadoop QA commented on HADOOP-14520:
------------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 13s{color} | {color:orange} hadoop-tools/hadoop-azure: The patch generated 80 new + 112 unchanged - 4 fixed = 192 total (was 116) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 34s{color} | {color:red} hadoop-tools/hadoop-azure generated 4 new + 0 unchanged - 0 fixed = 4 total (was 0) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 11s{color} | {color:red} hadoop-azure in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 21s{color} | {color:green} hadoop-azure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 35s{color} | {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-tools/hadoop-azure |
| | Inconsistent synchronization of org.apache.hadoop.fs.azure.BlockBlobAppendStream.activateCompactionBlockCount; locked 50% of time Unsynchronized access at BlockBlobAppendStream.java:50% of time Unsynchronized access at BlockBlobAppendStream.java:[line 807] |
| | Inconsistent synchronization of org.apache.hadoop.fs.azure.BlockBlobAppendStream.bufferSize; locked 60% of time Unsynchronized access at BlockBlobAppendStream.java:60% of time Unsynchronized access at BlockBlobAppendStream.java:[line 875] |
| | Inconsistent synchronization of org.apache.hadoop.fs.azure.BlockBlobAppendStream.outBuffer; locked 50% of time Unsynchronized access at BlockBlobAppendStream.java:50% of time Unsynchronized access at BlockBlobAppendStream.java:[line 413] |
| | Should org.apache.hadoop.fs.azure.BlockBlobAppendStream$ByteArrayOutputStreamInternal be a _static_ inner class? At BlockBlobAppendStream.java:inner class? At BlockBlobAppendStream.java:[lines 527-532] |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HADOOP-14520 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12876296/HADOOP-14520-4.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle |
| uname | Linux 9065c56a0341 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f484a6f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle | https://builds.apache.org/job/PreCommit-HADOOP-Build/12745/artifact/patchprocess/diff-checkstyle-hadoop-tools_hadoop-azure.txt |
| findbugs | https://builds.apache.org/job/PreCommit-HADOOP-Build/12745/artifact/patchprocess/new-findbugs-hadoop-tools_hadoop-azure.html |
| javadoc | https://builds.apache.org/job/PreCommit-HADOOP-Build/12745/artifact/patchprocess/patch-javadoc-hadoop-tools_hadoop-azure.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HADOOP-Build/12745/testReport/ |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output | https://builds.apache.org/job/PreCommit-HADOOP-Build/12745/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> WASB: Block compaction for Azure Block Blobs
> --------------------------------------------
>
> Key: HADOOP-14520
> URL: https://issues.apache.org/jira/browse/HADOOP-14520
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/azure
> Affects Versions: 3.0.0-alpha3
> Reporter: Georgi Chalakov
> Assignee: Georgi Chalakov
> Attachments: HADOOP-14520-01.patch, HADOOP-14520-01-test.txt, HADOOP-14520-03.patch, HADOOP-14520-4.patch
>
>
> Block Compaction for WASB allows uploading new blocks for every hflush/hsync call. When the number of blocks is above 32000, next hflush/hsync triggers the block compaction process. Block compaction replaces a sequence of blocks with one block. From all the sequences with total length less than 4M, compaction chooses the longest one. It is a greedy algorithm that preserve all potential candidates for the next round. Block Compaction for WASB increases data durability and allows using block blobs instead of page blobs. By default, block compaction is disabled. Similar to the configuration for page blobs, the client needs to specify HDFS folders where block compaction over block blobs is enabled.
> Results for HADOOP-14520-04.patch
> tested endpoint: fs.azure.account.key.hdfs4.blob.core.windows.net
> Tests run: 701, Failures: 0, Errors: 0, Skipped: 119
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org