You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/07 02:47:32 UTC

[GitHub] [airflow] wkhudgins92 opened a new pull request #8170: Airflow 6515

wkhudgins92 opened a new pull request #8170: Airflow 6515
URL: https://github.com/apache/airflow/pull/8170
 
 
   Per [AIRFLOW-6515](https://issues.apache.org/jira/browse/AIRFLOW-6515), I executed the following to get a list of potential errors logged with the info or warn level:
   
   ```
   grep -iE 'log\.(info|warn).*(error|exceptio|fail|unab|couldn|lost|gone|missing|not fou|abort|exit|could not)' -R *
   ```
   
   In addition to the above, I also ran the following
   ```
   grep -iE 'log\.(info|warn).*(unav)' -R *
   grep -iE 'log\.(info|warn).*(not p)' -R *
   grep -iE 'log\.(info|warn).*(invalid)' -R *
   ```
   
   I changed what I thought were errors to use the error log level. I also changed one or two info entries to warn log level.
   
   The above GREP commands still return some matches but I believe those are likely false positives.
   
   ---
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#discussion_r404861972
 
 

 ##########
 File path: airflow/providers/google/cloud/hooks/bigtable.py
 ##########
 @@ -94,7 +94,7 @@ def delete_instance(self, instance_id: str, project_id: Optional[str] = None) ->
         if instance:
             instance.delete()
         else:
-            self.log.info("The instance '%s' does not exist in project '%s'. Exiting", instance_id,
+            self.log.warn("The instance '%s' does not exist in project '%s'. Exiting", instance_id,
 
 Review comment:
   Here is the pylint problem because the indentation is incorrect ..

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611749011
 
 
   > @wkhudgins92 thanks for the PR! I'm wondering, can we add pre-commit check to avoid regression (i.e. using pygrep)? Also, what in your opinion is the difference between error and warning levels?
   
   @wkhudgins92 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611765011
 
 
   > Another alternative could be clear guidelines on logging in some kind of easy-to-access contributor documentation. We could even make it a checkbox on the PR template.
   
   That's what I'm thinking. However, I am not sure if anyone will search for or follow it...
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wkhudgins92 commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
wkhudgins92 commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#discussion_r406422935
 
 

 ##########
 File path: airflow/providers/hashicorp/secrets/vault.py
 ##########
 @@ -189,7 +189,7 @@ def _get_secret(self, path_prefix: str, secret_id: str) -> Optional[dict]:
                 response = self.client.secrets.kv.v2.read_secret_version(
                     path=secret_path, mount_point=self.mount_point)
         except InvalidPath:
-            self.log.info("Secret %s not found in Path: %s", secret_id, secret_path)
+            self.log.error("Secret %s not found in Path: %s", secret_id, secret_path)
 
 Review comment:
   Thanks, good catch!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611776691
 
 
   Awesome work, congrats on your first merged pull request!
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] boring-cyborg[bot] commented on issue #8170: Airflow 6515

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8170: Airflow 6515
URL: https://github.com/apache/airflow/pull/8170#issuecomment-610142793
 
 
   Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
   Here are some useful points:
   - Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
   - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
   - Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations.
   - Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
   - Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices).
   Apache Airflow is a community-driven project and together we are making it better 🚀.
   In case of doubts contact the developers at:
   Mailing List: dev@airflow.apache.org
   Slack: https://apache-airflow-slack.herokuapp.com/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-610714801
 
 
   I'll fix the failing tests tomorrow (GMT-5)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wkhudgins92 commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
wkhudgins92 commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#discussion_r404958324
 
 

 ##########
 File path: airflow/providers/google/cloud/hooks/bigtable.py
 ##########
 @@ -94,7 +94,7 @@ def delete_instance(self, instance_id: str, project_id: Optional[str] = None) ->
         if instance:
             instance.delete()
         else:
-            self.log.info("The instance '%s' does not exist in project '%s'. Exiting", instance_id,
+            self.log.warn("The instance '%s' does not exist in project '%s'. Exiting", instance_id,
 
 Review comment:
   Thanks @mik-laj , I booted up breeze again and found a few such issues, I think all are fixed now.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#discussion_r405739877
 
 

 ##########
 File path: airflow/providers/hashicorp/secrets/vault.py
 ##########
 @@ -189,7 +189,7 @@ def _get_secret(self, path_prefix: str, secret_id: str) -> Optional[dict]:
                 response = self.client.secrets.kv.v2.read_secret_version(
                     path=secret_path, mount_point=self.mount_point)
         except InvalidPath:
-            self.log.info("Secret %s not found in Path: %s", secret_id, secret_path)
+            self.log.error("Secret %s not found in Path: %s", secret_id, secret_path)
 
 Review comment:
   ```suggestion
               self.log.debug("Secret %s not found in Path: %s", secret_id, secret_path)
   ```
   
   This one is expected, so change it to `debug`. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611762956
 
 
   @turbaszek I would love to add a regression test for this but I agree with @kaxil , it likely is not possible. You could flag certain words in an info message, like I did with the grep statements, but at the end of the day there's always a change that those keywords will appear in a valid info/debug log statement. For example,
   
   ```            
   self.log.info("Secret %s not found in Path: %s", secret_id, secret_path)
   ```
   
   not found in path sounds like an error, or at least a warn, but in reality, in this specific context it should be a debug. 
   
   The best we could do would be a flag/warning but I don't think we can build a sufficiently smart CI check to incorporate it into the official CI processes. Another alternative could be clear guidelines on logging in some kind of easy-to-access contributor documentation. We could even make it a checkbox on the PR template.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611092966
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=h1) Report
   > Merging [#8170](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/68d1714f296989b7aad1a04b75dc033e76afb747&el=desc) will **increase** coverage by `21.43%`.
   > The diff coverage is `51.35%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/8170/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #8170       +/-   ##
   ===========================================
   + Coverage   66.39%   87.83%   +21.43%     
   ===========================================
     Files         935      935               
     Lines       45170    45243       +73     
   ===========================================
   + Hits        29990    39738     +9748     
   + Misses      15180     5505     -9675     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [airflow/contrib/utils/sendgrid.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3V0aWxzL3NlbmRncmlkLnB5) | `88.88% <0.00%> (+68.51%)` | :arrow_up: |
   | [airflow/executors/base\_executor.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvYmFzZV9leGVjdXRvci5weQ==) | `97.46% <0.00%> (+31.64%)` | :arrow_up: |
   | [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `57.11% <ø> (+40.03%)` | :arrow_up: |
   | [airflow/providers/apache/hive/hooks/hive.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvaG9va3MvaGl2ZS5weQ==) | `76.01% <0.00%> (-1.52%)` | :arrow_down: |
   | [...rflow/providers/apache/spark/hooks/spark\_submit.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3NwYXJrL2hvb2tzL3NwYXJrX3N1Ym1pdC5weQ==) | `84.79% <0.00%> (ø)` | |
   | [airflow/providers/google/cloud/hooks/bigquery.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL2hvb2tzL2JpZ3F1ZXJ5LnB5) | `91.63% <0.00%> (ø)` | |
   | [airflow/providers/google/cloud/operators/dlp.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9kbHAucHk=) | `96.25% <0.00%> (ø)` | |
   | [airflow/providers/qubole/hooks/qubole\_check.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcXVib2xlL2hvb2tzL3F1Ym9sZV9jaGVjay5weQ==) | `70.83% <0.00%> (ø)` | |
   | [airflow/providers/salesforce/hooks/salesforce.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvc2FsZXNmb3JjZS9ob29rcy9zYWxlc2ZvcmNlLnB5) | `93.24% <0.00%> (ø)` | |
   | [airflow/serialization/serialized\_objects.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZXJpYWxpemF0aW9uL3NlcmlhbGl6ZWRfb2JqZWN0cy5weQ==) | `90.73% <0.00%> (+9.26%)` | :arrow_up: |
   | ... and [530 more](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=footer). Last update [68d1714...14678ea](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611776575
 
 
   > > Another alternative could be clear guidelines on logging in some kind of easy-to-access contributor documentation. We could even make it a checkbox on the PR template.
   > 
   > That's what I'm thinking. However, I am not sure if anyone will search for or follow it...
   
   Exactly. If we want we can write something at https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611759649
 
 
   > > @wkhudgins92 thanks for the PR! I'm wondering, can we add pre-commit check to avoid regression (i.e. using pygrep)? Also, what in your opinion is the difference between error and warning levels?
   > 
   > @wkhudgins92
   
   Not sure you can avoid regression here. It depends on a case-by-case basis if a particular LOG is DEBUG, WARN, ERROR or INFO

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#discussion_r404862474
 
 

 ##########
 File path: airflow/task/task_runner/cgroup_task_runner.py
 ##########
 @@ -186,7 +186,7 @@ def return_code(self):
         # I wasn't able to track down the root cause of the package install failures, but
         # we might want to revisit that approach at some other point.
         if return_code == 137:
-            self.log.warning("Task failed with return code of 137. This may indicate "
+            self.log.error("Task failed with return code of 137. This may indicate "
 
 Review comment:
   Here is the pylint problem because the indentation is incorrect.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611761887
 
 
   > Not sure you can avoid regression here. It depends on a case-by-case basis if a particular LOG is DEBUG, WARN, ERROR or INFO
   
   True. But some basic check can help. Do we have any guide on selecting log level?
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611092966
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=h1) Report
   > Merging [#8170](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/68d1714f296989b7aad1a04b75dc033e76afb747&el=desc) will **increase** coverage by `21.43%`.
   > The diff coverage is `51.35%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/8170/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #8170       +/-   ##
   ===========================================
   + Coverage   66.39%   87.83%   +21.43%     
   ===========================================
     Files         935      935               
     Lines       45170    45243       +73     
   ===========================================
   + Hits        29990    39738     +9748     
   + Misses      15180     5505     -9675     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [airflow/contrib/utils/sendgrid.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3V0aWxzL3NlbmRncmlkLnB5) | `88.88% <0.00%> (+68.51%)` | :arrow_up: |
   | [airflow/executors/base\_executor.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvYmFzZV9leGVjdXRvci5weQ==) | `97.46% <0.00%> (+31.64%)` | :arrow_up: |
   | [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `57.11% <ø> (+40.03%)` | :arrow_up: |
   | [airflow/providers/apache/hive/hooks/hive.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvaG9va3MvaGl2ZS5weQ==) | `76.01% <0.00%> (-1.52%)` | :arrow_down: |
   | [...rflow/providers/apache/spark/hooks/spark\_submit.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3NwYXJrL2hvb2tzL3NwYXJrX3N1Ym1pdC5weQ==) | `84.79% <0.00%> (ø)` | |
   | [airflow/providers/google/cloud/hooks/bigquery.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL2hvb2tzL2JpZ3F1ZXJ5LnB5) | `91.63% <0.00%> (ø)` | |
   | [airflow/providers/google/cloud/operators/dlp.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9kbHAucHk=) | `96.25% <0.00%> (ø)` | |
   | [airflow/providers/qubole/hooks/qubole\_check.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcXVib2xlL2hvb2tzL3F1Ym9sZV9jaGVjay5weQ==) | `70.83% <0.00%> (ø)` | |
   | [airflow/providers/salesforce/hooks/salesforce.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvc2FsZXNmb3JjZS9ob29rcy9zYWxlc2ZvcmNlLnB5) | `93.24% <0.00%> (ø)` | |
   | [airflow/serialization/serialized\_objects.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZXJpYWxpemF0aW9uL3NlcmlhbGl6ZWRfb2JqZWN0cy5weQ==) | `90.73% <0.00%> (+9.26%)` | :arrow_up: |
   | ... and [530 more](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=footer). Last update [68d1714...14678ea](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-610415394
 
 
   @wkhudgins92 thanks for the PR! I'm wondering, can we add pre-commit check to avoid regression (i.e. using pygrep)? Also, what in your opinion is the difference between error and warning levels?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] kaxil merged pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611772348
 
 
   @turbaszek I think the most effective thing we could try would be a checkbox on the PR template, similar to "Commits follow "How to write a good git commit message", it could link to some markdown file about logging. We would want to have community review and consensus concerning the contents of such a guide. If this is something worth pursuing forward, it might make sense to create a followup ticket in JIRA for it and move this PR and ticket along. I would be willing to write a draft of such a document.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611092966
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=h1) Report
   > Merging [#8170](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/68d1714f296989b7aad1a04b75dc033e76afb747&el=desc) will **increase** coverage by `21.43%`.
   > The diff coverage is `51.35%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/8170/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #8170       +/-   ##
   ===========================================
   + Coverage   66.39%   87.83%   +21.43%     
   ===========================================
     Files         935      935               
     Lines       45170    45243       +73     
   ===========================================
   + Hits        29990    39738     +9748     
   + Misses      15180     5505     -9675     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [airflow/contrib/utils/sendgrid.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3V0aWxzL3NlbmRncmlkLnB5) | `88.88% <0.00%> (+68.51%)` | :arrow_up: |
   | [airflow/executors/base\_executor.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvYmFzZV9leGVjdXRvci5weQ==) | `97.46% <0.00%> (+31.64%)` | :arrow_up: |
   | [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `57.11% <ø> (+40.03%)` | :arrow_up: |
   | [airflow/providers/apache/hive/hooks/hive.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvaG9va3MvaGl2ZS5weQ==) | `76.01% <0.00%> (-1.52%)` | :arrow_down: |
   | [...rflow/providers/apache/spark/hooks/spark\_submit.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3NwYXJrL2hvb2tzL3NwYXJrX3N1Ym1pdC5weQ==) | `84.79% <0.00%> (ø)` | |
   | [airflow/providers/google/cloud/hooks/bigquery.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL2hvb2tzL2JpZ3F1ZXJ5LnB5) | `91.63% <0.00%> (ø)` | |
   | [airflow/providers/google/cloud/operators/dlp.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9kbHAucHk=) | `96.25% <0.00%> (ø)` | |
   | [airflow/providers/qubole/hooks/qubole\_check.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcXVib2xlL2hvb2tzL3F1Ym9sZV9jaGVjay5weQ==) | `70.83% <0.00%> (ø)` | |
   | [airflow/providers/salesforce/hooks/salesforce.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvc2FsZXNmb3JjZS9ob29rcy9zYWxlc2ZvcmNlLnB5) | `93.24% <0.00%> (ø)` | |
   | [airflow/serialization/serialized\_objects.py](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZXJpYWxpemF0aW9uL3NlcmlhbGl6ZWRfb2JqZWN0cy5weQ==) | `90.73% <0.00%> (+9.26%)` | :arrow_up: |
   | ... and [530 more](https://codecov.io/gh/apache/airflow/pull/8170/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=footer). Last update [68d1714...14678ea](https://codecov.io/gh/apache/airflow/pull/8170?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] mik-laj commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error

Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#discussion_r404861634
 
 

 ##########
 File path: airflow/providers/apache/hdfs/hooks/webhdfs.py
 ##########
 @@ -81,10 +81,10 @@ def _find_valid_server(self):
                     host_socket.close()
                     return client
                 else:
-                    self.log.info("Could not connect to %s:%s", connection.host, connection.port)
+                    self.log.error("Could not connect to %s:%s", connection.host, connection.port)
                 host_socket.close()
             except HdfsError as hdfs_error:
-                self.log.info('Read operation on namenode %s failed with error: %s',
+                self.log.error('Read operation on namenode %s failed with error: %s',
 
 Review comment:
   Here is the pylint problem because the indentation is incorrect ..

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services