You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2021/01/11 16:06:15 UTC
[GitHub] [hive] zabetak opened a new pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
zabetak opened a new pull request #1849:
URL: https://github.com/apache/hive/pull/1849
### What changes were proposed in this pull request?
1. Handle dynamic routing appenders via Log4j's `IdlePurgePolicy`
2. Remove in-house management of appenders.
### Why are the changes needed?
1. Solve descriptor leak as shown in HIVE-24590
2. Delegate burden of managing appenders to Log4j.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
`mvn test -Dtest=TestOperationLoggingLayout`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] prasanthj commented on a change in pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
prasanthj commented on a change in pull request #1849:
URL: https://github.com/apache/hive/pull/1849#discussion_r646329733
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/log/LogDivertAppender.java
##########
@@ -242,12 +244,17 @@ public static void registerRoutingAppender(org.apache.hadoop.conf.Configuration
LoggerContext context = (LoggerContext) LogManager.getContext(false);
Configuration configuration = context.getConfiguration();
+ PurgePolicy purgePolicy = IdlePurgePolicy.createPurgePolicy("60", null, "SECONDS", configuration);
Review comment:
IIRC the 60s was configurable from the log4j2.properties earlier. Can we make this configurable via some means (preferably log4j2 properties)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kgyrtkirk commented on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
kgyrtkirk commented on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-868669620
comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
zabetak commented on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-758694697
Closing and reopen to trigger tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] EugeneChung edited a comment on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
EugeneChung edited a comment on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-799393602
@zabetak Yes, I forgot to let you know. It worked well, but in my company's repo. I chose to clear all the log4j MDC. The leak and incorrect operation logging have been gone away.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak closed pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
zabetak closed pull request #1849:
URL: https://github.com/apache/hive/pull/1849
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] closed pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #1849:
URL: https://github.com/apache/hive/pull/1849
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] EugeneChung commented on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
EugeneChung commented on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-799393602
@zabetak Yes, I forgot to let you know. It worked well, but in my company's repo. I chose to clear all the log4j MDC.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-798809787
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
zabetak commented on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-799336894
Hey @EugeneChung did you have a change to try out this fix?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak edited a comment on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
zabetak edited a comment on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-799336894
Hey @EugeneChung did you have a chance to try out this fix?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] zabetak commented on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
zabetak commented on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-801960687
Thanks for testing this out @EugeneChung !
Hey @prasanthj , can you please review this change. The change is very similar to what was done as part of HIVE-24569 that you reviewed previously.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #1849: HIVE-24590: Operation logging still leaks log4j appenders
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #1849:
URL: https://github.com/apache/hive/pull/1849#issuecomment-855489358
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org