You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/06/29 10:44:55 UTC
[GitHub] [airflow] dan-origami commented on issue #16703: Workers silently crash after memory build up
dan-origami commented on issue #16703:
URL: https://github.com/apache/airflow/issues/16703#issuecomment-870485384
@ashb its `container_memory_usage_bytes` with full query below
```sum(container_memory_usage_bytes{container!="POD",pod=~"airflow.*",image!=""}) by (pod)```
Nothing on the dmesg at all, i checked and due to our affinity/selector the pod was always on the same server
```
[Tue Jun 22 13:05:51 2021] device veth7323bc8f entered promiscuous mode
[Tue Jun 22 13:05:51 2021] cni0: port 6(veth7323bc8f) entered blocking state
[Tue Jun 22 13:05:51 2021] cni0: port 6(veth7323bc8f) entered forwarding state
[Tue Jun 22 13:08:10 2021] cni0: port 2(veth742b6e37) entered disabled state
[Tue Jun 22 13:08:10 2021] device veth742b6e37 left promiscuous mode
[Tue Jun 22 13:08:10 2021] cni0: port 2(veth742b6e37) entered disabled state
[Tue Jun 22 13:08:19 2021] IPv6: ADDRCONF(NETDEV_CHANGE): vethab64c6d9: link becomes ready
[Tue Jun 22 13:08:19 2021] cni0: port 2(vethab64c6d9) entered blocking state
[Tue Jun 22 13:08:19 2021] cni0: port 2(vethab64c6d9) entered disabled state
[Tue Jun 22 13:08:19 2021] device vethab64c6d9 entered promiscuous mode
[Tue Jun 22 13:08:19 2021] cni0: port 2(vethab64c6d9) entered blocking state
[Tue Jun 22 13:08:19 2021] cni0: port 2(vethab64c6d9) entered forwarding state
[Tue Jun 22 14:57:36 2021] cni0: port 5(vethda1d06e5) entered disabled state
[Tue Jun 22 14:57:36 2021] device vethda1d06e5 left promiscuous mode
[Tue Jun 22 14:57:36 2021] cni0: port 5(vethda1d06e5) entered disabled state
[Fri Jun 25 09:38:38 2021] cni0: port 3(vethfa6a98a7) entered disabled state
[Fri Jun 25 09:38:38 2021] device vethfa6a98a7 left promiscuous mode
[Fri Jun 25 09:38:38 2021] cni0: port 3(vethfa6a98a7) entered disabled state
[Mon Jun 28 08:43:32 2021] IPv6: ADDRCONF(NETDEV_CHANGE): vethae505a5a: link becomes ready
[Mon Jun 28 08:43:32 2021] cni0: port 3(vethae505a5a) entered blocking state
[Mon Jun 28 08:43:32 2021] cni0: port 3(vethae505a5a) entered disabled state
[Mon Jun 28 08:43:32 2021] device vethae505a5a entered promiscuous mode
[Mon Jun 28 08:43:32 2021] cni0: port 3(vethae505a5a) entered blocking state
[Mon Jun 28 08:43:32 2021] cni0: port 3(vethae505a5a) entered forwarding state
[Mon Jun 28 08:43:40 2021] cni0: port 6(veth7323bc8f) entered disabled state
[Mon Jun 28 08:43:40 2021] device veth7323bc8f left promiscuous mode
[Mon Jun 28 08:43:40 2021] cni0: port 6(veth7323bc8f) entered disabled state
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org