You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "David Johnston (JIRA)" <ji...@apache.org> on 2012/11/07 19:28:12 UTC

[jira] [Created] (FLUME-1688) Bump AsyncHBase version to 1.3.2

David Johnston created FLUME-1688:
-------------------------------------

             Summary: Bump AsyncHBase version to 1.3.2
                 Key: FLUME-1688
                 URL: https://issues.apache.org/jira/browse/FLUME-1688
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
    Affects Versions: v1.3.0, v1.4.0
            Reporter: David Johnston


Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 

Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493638#comment-13493638 ] 

Hari Shreedharan commented on FLUME-1688:
-----------------------------------------

The ZK connection issues might have been due to ZOOKEEPER-1437. I am concerned about the dependencies it introduces. Looks like it will pull in new version of Netty, Guava etc. Let me take a look at the dependencies it is pulling in by doing a build.
                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498257#comment-13498257 ] 

Hari Shreedharan commented on FLUME-1688:
-----------------------------------------

[~ftdave] Do you have any updates on this? It might be a good idea to open an issue with asynchbase project to look into this. You should post the mvn dependency:tree output which I have posted above.
                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "David Johnston (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Johnston updated FLUME-1688:
----------------------------------

    Attachment: FLUME-1688.patch
    
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "David Johnston (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493052#comment-13493052 ] 

David Johnston commented on FLUME-1688:
---------------------------------------

The most significant performance improvement was added in 1.3.0, allowing a Put to contain multiple values (https://github.com/OpenTSDB/asynchbase/commit/bb9168cda1551db4712c1c584cd140a66ec17422) - in 1.2.0, the same operation would require many separate Puts. It also lets us use coprocessors on the region server, which for us at least is a functional requirement.

WRT bugs, we've experienced frequent ZooKeeper connection issues when using Flume 1.3.0 and Hadoop 1.0.3. Recompiling Flume with AsyncHBase 1.3.2 fixed this issue, although I've yet to identify exactly what changeset fixed this (I'm assuming it's one of the Netty changes).

                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "David Johnston (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493894#comment-13493894 ] 

David Johnston commented on FLUME-1688:
---------------------------------------

Ah, the AsyncHBase POM explicitly defines a minimum version. Still doesn't explain why it pulled in 3.4.4 before and 3.3.4 now.
                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492624#comment-13492624 ] 

Hari Shreedharan commented on FLUME-1688:
-----------------------------------------

Do you have information on:
a) compatibility issues.
b) performance?
                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493673#comment-13493673 ] 

Hari Shreedharan commented on FLUME-1688:
-----------------------------------------

Outputs of mvn dependency:tree. Do you know why this is happening:

With asynchbase-1.3.2:
[INFO] +- org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:jar:1.4.0-SNAPSHOT:compile
[INFO] |  \- org.hbase:asynchbase:jar:1.3.2:compile
[INFO] |     +- com.stumbleupon:async:jar:1.2.0:compile
[INFO] |     +- org.apache.zookeeper:zookeeper:jar:3.3.4:compile
[INFO] |     \- org.slf4j:jcl-over-slf4j:jar:1.6.4:runtime


With aynchbase-1.2.0 (current trunk):
[INFO] +- org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:jar:1.4.0-SNAPSHOT:compile
[INFO] |  \- org.hbase:asynchbase:jar:1.2.0:compile
[INFO] |     +- org.jboss.netty:netty:jar:3.2.7.Final:compile
[INFO] |     +- com.stumbleupon:async:jar:1.2.0:compile
[INFO] |     +- org.apache.zookeeper:zookeeper:jar:3.4.4:compile
[INFO] |     \- org.slf4j:jcl-over-slf4j:jar:1.7.2:runtime


                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493648#comment-13493648 ] 

Hari Shreedharan commented on FLUME-1688:
-----------------------------------------

It looks like ZK is being downgraded from 3.4.4 to 3.3.4. I am not sure why this is and whether this affects HBase. I will need to look at hbase and see if this change in ZK version is a concern.
                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "Hari Shreedharan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493673#comment-13493673 ] 

Hari Shreedharan edited comment on FLUME-1688 at 11/9/12 2:00 AM:
------------------------------------------------------------------

Outputs of mvn dependency:tree. Do you know why this is happening:

With asynchbase-1.3.2:
{code}
[INFO] +- org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:jar:1.4.0-SNAPSHOT:compile
[INFO] |  \- org.hbase:asynchbase:jar:1.3.2:compile
[INFO] |     +- com.stumbleupon:async:jar:1.2.0:compile
[INFO] |     +- org.apache.zookeeper:zookeeper:jar:3.3.4:compile
[INFO] |     \- org.slf4j:jcl-over-slf4j:jar:1.6.4:runtime
{code}

With aynchbase-1.2.0 (current trunk):
{code}
[INFO] +- org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:jar:1.4.0-SNAPSHOT:compile
[INFO] |  \- org.hbase:asynchbase:jar:1.2.0:compile
[INFO] |     +- org.jboss.netty:netty:jar:3.2.7.Final:compile
[INFO] |     +- com.stumbleupon:async:jar:1.2.0:compile
[INFO] |     +- org.apache.zookeeper:zookeeper:jar:3.4.4:compile
[INFO] |     \- org.slf4j:jcl-over-slf4j:jar:1.7.2:runtime
{code}
                
      was (Author: hshreedharan):
    Outputs of mvn dependency:tree. Do you know why this is happening:

With asynchbase-1.3.2:
[INFO] +- org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:jar:1.4.0-SNAPSHOT:compile
[INFO] |  \- org.hbase:asynchbase:jar:1.3.2:compile
[INFO] |     +- com.stumbleupon:async:jar:1.2.0:compile
[INFO] |     +- org.apache.zookeeper:zookeeper:jar:3.3.4:compile
[INFO] |     \- org.slf4j:jcl-over-slf4j:jar:1.6.4:runtime


With aynchbase-1.2.0 (current trunk):
[INFO] +- org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:jar:1.4.0-SNAPSHOT:compile
[INFO] |  \- org.hbase:asynchbase:jar:1.2.0:compile
[INFO] |     +- org.jboss.netty:netty:jar:3.2.7.Final:compile
[INFO] |     +- com.stumbleupon:async:jar:1.2.0:compile
[INFO] |     +- org.apache.zookeeper:zookeeper:jar:3.4.4:compile
[INFO] |     \- org.slf4j:jcl-over-slf4j:jar:1.7.2:runtime


                  
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (FLUME-1688) Bump AsyncHBase version to 1.3.2

Posted by "David Johnston (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/FLUME-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13493843#comment-13493843 ] 

David Johnston commented on FLUME-1688:
---------------------------------------

ZOOKEEPER-1437 is indeed the same issue we had. Regretfully I hadn't spotted the change from 3.4.4 to 3.3.4 (curse those similar numbers!)

Looking deeper, AsyncHBase has always had a dependency on ZooKeeper 3.3.1 (in 1.2.0) or 3.3.4 (1.3.0 and higher) - see https://github.com/OpenTSDB/asynchbase/blob/master/third_party/zookeeper/include.mk - which is making me wonder why 3.4.4 is being pulled in at all.
                
> Bump AsyncHBase version to 1.3.2
> --------------------------------
>
>                 Key: FLUME-1688
>                 URL: https://issues.apache.org/jira/browse/FLUME-1688
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.3.0, v1.4.0
>            Reporter: David Johnston
>         Attachments: FLUME-1688.patch
>
>   Original Estimate: 5m
>  Remaining Estimate: 5m
>
> Bump AsyncHBase to 1.3.2 (versus 1.2.0) to take advantage of a range of bug fixes and performance improvements. 
> Only affects the asynchronous HBase sink.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira