You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/11 10:30:30 UTC

[GitHub] [airflow] johnchenghk01 opened a new issue #13609: Unable to read log if the pod name is too long

johnchenghk01 opened a new issue #13609:
URL: https://github.com/apache/airflow/issues/13609


   **Apache Airflow version**: 1.10.12
   
   
   **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.15
   
   **Environment**:
   
   - **Cloud provider or hardware configuration**: Not related
   - **OS** (e.g. from /etc/os-release): Not related
   - **Kernel** (e.g. `uname -a`): Not related
   - **Install tools**: Not related
   - **Others**: Not related
   
   **What happened**:
   
   We are using kubernetes executor and try to read logs from webserver directly via the Kubernetes API.
   (https://github.com/apache/airflow/pull/12598)
   We found that if the pod name is longer than 63 characters, it will be 404.
   
   ```
   *** Unable to fetch logs from worker pod thisisaverylooooooooooooooooooooongpodname-8fba3581e4614c469655 ***
   (404)
   Reason: Not Found
   HTTP response headers: HTTPHeaderDict({'Audit-Id': 'xxxxxxxx', 'Content-Type': 'application/json', 'Date': 'Mon, 11 Jan 2021 09:57:37 GMT', 'Content-Length': '294'})
   HTTP response body: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \\"thisisaverylooooooooooooooooooooongpodname-8fba3581e4614c469655\\" not found","reason":"NotFound","details":{"name":"thisisaverylooooooooooooooooooooongpodname-8fba3581e4614c469655","kind":"pods"},"code":404}\n'
   ```
   
   **What you expected to happen**:
   
   ```
   *** Trying to get logs (last 100 lines) from worker pod shortnameeeeeeeeeeeeeeetask-a11e7ef862f641e4a0a4a686482f43a4 ***
   
   [2021-01-11 10:22:29,200] {__init__.py:50} INFO - Using executor LocalExecutor
   [2021-01-11 10:22:29,200] {dagbag.py:417} INFO - Filling up the DagBag from /usr/local/airflow/dags/shortname.py
   ```
   
   **How to reproduce it**: Name a dag / task with long name.
   
   **Anything else we need to know**:
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] brandondtb commented on issue #13609: Unable to read log if the pod name is too long

Posted by GitBox <gi...@apache.org>.
brandondtb commented on issue #13609:
URL: https://github.com/apache/airflow/issues/13609#issuecomment-758059631


   @johnchenghk01Are the pods actually getting scheduled and run correctly? Any errors in your scheduler logs?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] johnchenghk01 closed issue #13609: Unable to read log if the pod name is too long

Posted by GitBox <gi...@apache.org>.
johnchenghk01 closed issue #13609:
URL: https://github.com/apache/airflow/issues/13609


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] johnchenghk01 commented on issue #13609: Unable to read log if the pod name is too long

Posted by GitBox <gi...@apache.org>.
johnchenghk01 commented on issue #13609:
URL: https://github.com/apache/airflow/issues/13609#issuecomment-758364474


   Just found out that it is fix with this PR https://github.com/apache/airflow/pull/10942
   (included in 1.10.13)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #13609: Unable to read log if the pod name is too long

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13609:
URL: https://github.com/apache/airflow/issues/13609#issuecomment-757860750


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] jethrogillgren commented on issue #13609: Unable to read log if the pod name is too long

Posted by GitBox <gi...@apache.org>.
jethrogillgren commented on issue #13609:
URL: https://github.com/apache/airflow/issues/13609#issuecomment-769730038


   Hi, i'm using `apache/airflow:1.10.14-python3.7` and am still seeing this. 
   Did that PR make it in? It looks like it should. I see the same error as johnchenghk01


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] johnchenghk01 commented on issue #13609: Unable to read log if the pod name is too long

Posted by GitBox <gi...@apache.org>.
johnchenghk01 commented on issue #13609:
URL: https://github.com/apache/airflow/issues/13609#issuecomment-758360681


   The pods were started and running correctly. There is no error in the scheduler logs.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] jethrogillgren edited a comment on issue #13609: Unable to read log if the pod name is too long

Posted by GitBox <gi...@apache.org>.
jethrogillgren edited a comment on issue #13609:
URL: https://github.com/apache/airflow/issues/13609#issuecomment-769730038


   Hi, i'm using `apache/airflow:1.10.14-python3.7` and am still seeing this. 
   Did that PR make it in? It looks like it should. I see the same error as johnchenghk01, but am on kubernetes 1.18.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org