You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2021/04/22 23:13:30 UTC
[GitHub] [camel-quarkus] ffang opened a new pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
ffang opened a new pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506
[ ] An issue should be filed for the change unless this is a trivial change (fixing a typo or similar). One issue should ideally be fixed by not more than one commit and the other way round, each commit should fix just one issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful and properly spelled subject line and body. Copying the title of the associated issue is typically enough. Please include the issue number in the commit message prefixed by #.
[ ] The pull request description should explain what the pull request does, how, and why. If the info is available in the associated issue or some other external document, a link is enough.
[ ] Phrases like Fix #<issueNumber> or Fixes #<issueNumber> will auto-close the named issue upon merging the pull request. Using them is typically a good idea.
[ ] Please run mvn process-resources -Pformat (and amend the changes if necessary) before sending the pull request.
[ ] Contributor guide is your good friend: https://camel.apache.org/camel-quarkus/latest/contributor-guide.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ffang commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ffang commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-827558653
> I'm not aware of the hbase test harness internals, but as far as I can help tests are running on my machine with and without this patch.
> My understanding is that this patch would make use of fixed ports which we tend to avoid ? Is it strictly necessary in order to run on CI ?
> From the reading it looks that overriding hostname resolution would help. I remember a setup, with two containers running in the [same network](https://github.com/apache/camel-quarkus/blob/main/integration-tests/kudu/src/test/java/org/apache/camel/quarkus/component/kudu/it/KuduTestResource.java), with [hostname resolution overriding](https://github.com/apache/camel-quarkus/blob/main/integration-tests/kudu/src/main/java/org/apache/camel/quarkus/component/kudu/it/KuduInfrastructureTestHelper.java) for kudu tests. I would use that with caution and only in case of urgency :)
>
> Hope this helps
Hi @aldettinger ,
Yes, basically this patch intended to use fixed port on the local host running the hbase/zookeeper client, also to set the hostname as the local host name instead of the container name running the hbase/zookeeper server.
We have ./src/main/resources/hbase-site.xml which specifies the hbase.zookeeper.property.clientPort 2181, and I believe this should be the local host port, so we should have some "port forwarding" configuration between container_port and local_host_port. Otherwise on my machine, I always have error like
```
2021-04-22 11:40:58,745 WARN [org.apa.had.hba.zoo.ReadOnlyZKClient] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9) 0x5e541ef9 to 127.0.0.1:2181 failed for get of /hbase/master, code = CONNECTIONLOSS, retries = 1
2021-04-22 11:40:59,748 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2021-04-22 11:40:59,749 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Socket error occurred: localhost/127.0.0.1:2181: Connection refused
2021-04-22 11:40:59,853 WARN [org.apa.had.hba.zoo.ReadOnlyZKClient] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9) 0x5e541ef9 to 127.0.0.1:2181 failed for get of /hbase/master, code = CONNECTIONLOSS, retries = 2
2021-04-22 11:41:00,858 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2021-04-22 11:41:00,858 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Socket error occurred: localhost/127.0.0.1:2181: Connection refused
2021-04-22 11:41:00,964 WARN [org.apa.had.hba.zoo.ReadOnlyZKClient] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9) 0x5e541ef9 to 127.0.0.1:2181 failed for get of /hbase/master, code = CONNECTIONLOSS, retries = 3
2021-04-22 11:41:01,968 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2021-04-22 11:41:01,968 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Socket error occurred: localhost/127.0.0.1:2181: Connection refused
2021-04-22 11:41:02,070 WARN [org.apa.had.hba.zoo.ReadOnlyZKClient] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9) 0x5e541ef9 to 127.0.0.1:2181 failed for get of /hbase/master, code = CONNECTIONLOSS, retries = 4
2021-04-22 11:41:03,074 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2021-04-22 11:41:03,075 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Socket error occurred: localhost/127.0.0.1:2181: Connection refused
2021-04-22 11:41:03,179 WARN [org.apa.had.hba.zoo.ReadOnlyZKClient] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9) 0x5e541ef9 to 127.0.0.1:2181 failed for get of /hbase/master, code = CONNECTIONLOSS, retries = 5
2021-04-22 11:41:04,185 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2021-04-22 11:41:04,185 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Socket error occurred: localhost/127.0.0.1:2181: Connection refused
2021-04-22 11:41:04,289 WARN [org.apa.had.hba.zoo.ReadOnlyZKClient] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9) 0x5e541ef9 to 127.0.0.1:2181 failed for get of /hbase/master, code = CONNECTIONLOSS, retries = 6
2021-04-22 11:41:05,293 INFO [org.apa.zoo.ClientCnxn] (ReadOnlyZKClient-127.0.0.1:2181@0x5e541ef9-SendThread(127.0.0.1:2181)) Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
```
Also the e.withHostName(hostname); in the patch can resolve the CI build failure
```
Caused by: org.apache.hbase.thirdparty.io.netty.channel.unix.Errors$NativeIoException: bind(..) failed: Cannot assign requested address
at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newIOException(Errors.java:117)
at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.bind(Socket.java:285)
at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel.doBind(AbstractEpollChannel.java:714)
at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollServerSocketChannel.doBind(EpollServerSocketChannel.java:70)
at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:558)
at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1283)
```
Thanks!
Freeman
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ffang commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ffang commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-827677088
Thanks @ppalaga @aldettinger !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ppalaga commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-826544093
@ffang does this fix any of the issues mentioned in https://github.com/apache/camel-quarkus/issues/2458 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ppalaga commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-826544093
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ffang commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ffang commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-826785258
> @ffang does this fix any of the issues mentioned in #2458 ?
Hi @ppalaga ,
Yes, I hope we don't need to edit /etc/hosts on the CI build server.
Plus, the way you mentioned to edit /etc/hosts doesn't work for me on my local machine(Probably I'm using Docker desktop on my mac).
Moreover, I don't think previously the way to use testcontainer is correctly. It was exposing the port(2181, 16000, 16020) on the pod container, but not on the host machine which running the client(on the host machine it was random port), so we saw errors. We need the port forwarding configuration. My fix pretty much equals the way to "docker run -p container_port:host_port"
Freeman
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ppalaga commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-826544457
How is this actually related to #2490 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ffang commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ffang commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-826779933
> How is this actually related to #2490 ?
Hi @ppalaga ,
I referred to this build log https://github.com/apache/camel-quarkus/runs/2366932398?check_suite_focus=true in #2490, which caused CI build failure in the first place.
Freeman
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
aldettinger commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-827416565
I'm not aware of the hbase test harness internals, but as far as I can help tests are running on my machine with and without this patch.
My understanding is that this patch would make use of fixed ports which we tend to avoid ? Is it strictly necessary in order to run on CI ?
From the reading it looks that overriding hostname resolution would help. I remember a setup, with two containers running in the [same network](https://github.com/apache/camel-quarkus/blob/main/integration-tests/kudu/src/test/java/org/apache/camel/quarkus/component/kudu/it/KuduTestResource.java), with [hostname resolution overriding](https://github.com/apache/camel-quarkus/blob/main/integration-tests/kudu/src/main/java/org/apache/camel/quarkus/component/kudu/it/KuduInfrastructureTestHelper.java) for kudu tests. I would use that with caution and only in case of urgency :)
Hope this helps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ppalaga commented on pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506#issuecomment-827674204
Let try and see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga merged pull request #2506: #2490 fix camel-quarkus-hbase-integration-test
Posted by GitBox <gi...@apache.org>.
ppalaga merged pull request #2506:
URL: https://github.com/apache/camel-quarkus/pull/2506
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org