You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2020/06/29 08:45:11 UTC
[GitHub] [incubator-heron] nicknezis opened a new pull request #3550: Updates to Kubernetes scheduler to fix edge cases
nicknezis opened a new pull request #3550:
URL: https://github.com/apache/incubator-heron/pull/3550
Fix for #3542
Some notable changes:
1. Kubernetes scheduler now creates a Headless Service to pair with the StatefulSet.
2. Upgraded Kubernetes client from 7.0.0 to 8.0.0
3. Removed old compatibility code that deleted all pods. I don't see the code that used to launch individual pods, so I assume it is no longer needed. For now we only launch StatefulSets, although I do have thoughts on how to better manage the topology pods.
4. Heron Executor now checks for POD_NAME environment variable. If found, it assumed Kubernetes environment in which we use the fully qualified domain name which should be composed of pod name, headless service, namespace and cluster domain.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] windhamwong commented on pull request #3550: Updates to Kubernetes scheduler to fix edge cases
Posted by GitBox <gi...@apache.org>.
windhamwong commented on pull request #3550:
URL: https://github.com/apache/incubator-heron/pull/3550#issuecomment-652193699
Testing environment:
namespace: heron
No additional service added mannually.
Tested with the example topology `ackingTopology`. I am still seeing the `acking-1` complaining about:
File: metricsmgr-1.log.0
```
[2020-07-01 05:05:02 +0000] [INFO] org.apache.heron.common.network.HeronClient: Connecting to endpoint: acking-0:6001
[2020-07-01 05:05:02 +0000] [SEVERE] org.apache.heron.metricsmgr.sink.tmaster.TMasterSink: TMasterClient dies in thread: Thread[Thread-1117,5,main]
java.nio.channels.UnresolvedAddressException
at java.base/sun.nio.ch.Net.checkAddress(Net.java:130)
at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:675)
at org.apache.heron.common.network.HeronClient.start(HeronClient.java:129)
at org.apache.heron.metricsmgr.sink.tmaster.TMasterClient.run(TMasterClient.java:128)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
```
and
File: container_1_exclaim1_1.log.0
```
[INFO] org.apache.heron.network.StreamManagerClient: Stop writing due to not yet connected to Stream Manager.
```
It is still trying to resolve `acking-1` instead of `acking-1.acking.heron.svc.cluster.local` OR `acking1.acking`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on pull request #3550: Updates to Kubernetes scheduler to fix edge cases
Posted by GitBox <gi...@apache.org>.
nicknezis commented on pull request #3550:
URL: https://github.com/apache/incubator-heron/pull/3550#issuecomment-656904962
DrKNa in the Slack was able to test this and verified that it worked in his environment which previously did not work. But I'd like a review if the logic I added is the best option. Any suggestions before merging in?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on a change in pull request #3550: Updates to Kubernetes scheduler to fix edge cases
Posted by GitBox <gi...@apache.org>.
nicknezis commented on a change in pull request #3550:
URL: https://github.com/apache/incubator-heron/pull/3550#discussion_r448016862
##########
File path: third_party/java/BUILD
##########
@@ -227,10 +227,10 @@ java_library(
name = "apache-pulsar-client",
srcs = ["Empty.java"],
exports = [
- "@maven//:org_apache_pulsar_pulsar_client_shaded",
+ "@maven//:org_apache_pulsar_pulsar_client",
],
deps = [
- "@maven//:org_apache_pulsar_pulsar_client_shaded",
+ "@maven//:org_apache_pulsar_pulsar_client",
],
Review comment:
We might want to put it back to the shaded jar. I think I made this change because of a compilation error. I just checked, and I think we changed the dependency in the WORKSPACE file. Here is the PR: https://github.com/apache/incubator-heron/pull/3536/files
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis merged pull request #3550: Updates to Kubernetes scheduler to fix edge cases
Posted by GitBox <gi...@apache.org>.
nicknezis merged pull request #3550:
URL: https://github.com/apache/incubator-heron/pull/3550
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on pull request #3550: Updates to Kubernetes scheduler to fix edge cases
Posted by GitBox <gi...@apache.org>.
nicknezis commented on pull request #3550:
URL: https://github.com/apache/incubator-heron/pull/3550#issuecomment-652213439
@windhamwong What environment are you running in? I'd like to recreate.
A few questions.
1. Do you see the Headless service created at submit time?
2. Does the `acking-0` Executor log not show the FQDN?
3. Does the `acking-1` Executor log also not show the FQDN?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on a change in pull request #3550: Updates to Kubernetes scheduler to fix edge cases
Posted by GitBox <gi...@apache.org>.
nicknezis commented on a change in pull request #3550:
URL: https://github.com/apache/incubator-heron/pull/3550#discussion_r448101404
##########
File path: third_party/java/BUILD
##########
@@ -227,10 +227,10 @@ java_library(
name = "apache-pulsar-client",
srcs = ["Empty.java"],
exports = [
- "@maven//:org_apache_pulsar_pulsar_client_shaded",
+ "@maven//:org_apache_pulsar_pulsar_client",
],
deps = [
- "@maven//:org_apache_pulsar_pulsar_client_shaded",
+ "@maven//:org_apache_pulsar_pulsar_client",
],
Review comment:
Ok I put it back to the shaded version of the jar. Should look more like what it was before the PR we merged 23 days ago.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org