You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "nikita15p (via GitHub)" <gi...@apache.org> on 2023/02/13 19:38:53 UTC
[GitHub] [flume] nikita15p opened a new pull request, #396: FLUME-3448 bump up to hadoop 3.2.2
nikita15p opened a new pull request, #396:
URL: https://github.com/apache/flume/pull/396
Signed-off-by: nikita15p <37...@users.noreply.github.com>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1465057944
I tested again today and TestHBaseSink is still failing exactly as I posted above 2 weeks ago. I don't get that error on trunk. From the stack trace it looks like something must have changed in a Google library.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] nikita15p commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "nikita15p (via GitHub)" <gi...@apache.org>.
nikita15p commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1484552069
@rgoers I am working on this and will update soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1442210992
I am still seeing these errors:
```
[INFO] Running org.apache.flume.sink.hbase.TestAsyncHBaseSink
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.193 s <<< FAILURE! - in org.apache.flume.sink.hbase.TestAsyncHBaseSink
[ERROR] org.apache.flume.sink.hbase.TestAsyncHBaseSink Time elapsed: 6.193 s <<< ERROR!
java.io.IOException: Shutting down
at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:73)
Caused by: java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:73)
Caused by: java.lang.NoSuchMethodError: com.google.common.hash.HashFunction.hashString(Ljava/lang/CharSequence;)Lcom/google/common/hash/HashCode;
at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:73)
[INFO] Running org.apache.flume.sink.hbase.TestAsyncHBaseSinkConfiguration
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.54 s - in org.apache.flume.sink.hbase.TestAsyncHBaseSinkConfiguration
[INFO] Running org.apache.flume.sink.hbase.TestRegexHbaseEventSerializer
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.578 s - in org.apache.flume.sink.hbase.TestRegexHbaseEventSerializer
[INFO] Running org.apache.flume.sink.hbase.TestHBaseSink
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.845 s <<< FAILURE! - in org.apache.flume.sink.hbase.TestHBaseSink
[ERROR] org.apache.flume.sink.hbase.TestHBaseSink Time elapsed: 5.845 s <<< ERROR!
java.io.IOException: Shutting down
at org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:84)
Caused by: java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
at org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:84)
Caused by: java.lang.NoSuchMethodError: com.google.common.hash.HashFunction.hashString(Ljava/lang/CharSequence;)Lcom/google/common/hash/HashCode;
at org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:84)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers closed pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers closed pull request #396: FLUME-3448 bump up to hadoop 3.2.2
URL: https://github.com/apache/flume/pull/396
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1465271233
I don't use Flume with Hadoop so I would not be the best qualified person to answer regarding hbase 1.x. I don't see any formal statement about HBase 1.x being EOL and, in fact, I see a 1.7.2 release from Aug 3, 2022.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] nikita15p commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "nikita15p (via GitHub)" <gi...@apache.org>.
nikita15p commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1485160137
Sure @rgoers , if we are ok with removing hbase sink which I think is the best thing as most of the projects have moved out of hbase 1.x version including phoenix.
I was looking into a way where I maintain hadoop2 and hadoop3 profile. Hadoop2 would be default one with both hbase and hbase2, however for hadoop3 profile, we will have all modules but hbase.
I still feel getting removing hbase1.x would be better. Let me know what should I proceed here with.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] nikita15p commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "nikita15p (via GitHub)" <gi...@apache.org>.
nikita15p commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1452990060
@rgoers I have fixed unit tests where hadoop is a dependency. There are 7 tests which were failing even before I raised this PR. Currently ignored them, these are in TestThriftSink.java class , TestMultiportSyslogTCPSource.java and TestThriftSource.java class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1484564061
What will you be doing? If it is just a matter of removing the HBase Sink I can handle that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] nikita15p commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "nikita15p (via GitHub)" <gi...@apache.org>.
nikita15p commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1465183115
@rgoers you are right wrt hbase subproject. I did fix hbase2, hive wrt these changes and it do pass. While I tested hbase subproject was commented, so missed it. At that point of time, I was checking other projects, most of them have moved to hbase 2.x, shall we still support hbase 1.x? Let me know your thoughts.
For Hbase2,
```
-------------------------------------------------------------------------------
Test set: org.apache.flume.sink.hbase2.TestHBase2Sink
-------------------------------------------------------------------------------
Tests run: 16, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 74.042 s - in org.apache.flume.sink.hbase2.TestHBase2Sink
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1440713335
I tried applying this and am getting unit test failures in the Flume Auth module in TestAuthenticator. Did you run a full build after making this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1485838229
I pulled these changes, removed hbase-sink, fixed some checkstyle issues and then merged this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flume] rgoers commented on pull request #396: FLUME-3448 bump up to hadoop 3.2.2
Posted by "rgoers (via GitHub)" <gi...@apache.org>.
rgoers commented on PR #396:
URL: https://github.com/apache/flume/pull/396#issuecomment-1484235323
I have spent several days looking into this and have come to the conclusion that there is no way to include the HBaseSink with these changes. The problem is that Guava seems to make incompatible changes fairly frequently. Since this change upgrades the version of Guava it must also be upgraded for the HBaseSink. Doing so causes a NoSuchMethod error on HashFunction, which removed one of the hashString methods. Not upgrading Guava for only the HBaseSink results in a NoSuchMethod error for the Preconditions.checkState method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@flume.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org