You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/08/10 07:28:11 UTC
[GitHub] [ozone] ArafatKhan2198 opened a new pull request, #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
ArafatKhan2198 opened a new pull request, #3671:
URL: https://github.com/apache/ozone/pull/3671
## What changes were proposed in this pull request?
warning messages should be removed from console logs. We can do this by suppressing the respective messages in the log4j.properties file.
for i in $(seq 1 6)
> do
> /opt/cloudera/parcels/CDH/bin/ozone fs -copyFromLocal /tmp/5GB ofs://ozone1/vol2/buck1/key$i
> done
```
22/02/14 12:17:23 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
22/02/14 12:17:24 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
22/02/14 12:17:24 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system started
22/02/14 12:17:24 INFO metrics.MetricRegistries: Loaded MetricRegistries class org.apache.ratis.metrics.impl.MetricRegistriesImpl
22/02/14 12:20:28 WARN io.KeyOutputStream: Encountered exception java.io.IOException: Unexpected Storage Container Exception: java.util.concurrent.CompletionException: Failed to write chunk 109611004723200001_chunk_1 into block conID: 1 locID: 109611004723200001 bcsId: 0 on the pipeline Pipeline[ Id: c665901b-87bf-4943-8ec6-da745db56901, Nodes: c221dcbd-398c-4e76-ab13-dcde074041ae{ip: 172.27.128.2, host: quasar-usholy-5.quasar-usholy.root.hwx.site, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}c9da661f-99ae-4de3-8eb8-306763283723{ip: 172.27.126.198, host: quasar-usholy-6.quasar-usholy.root.hwx.site, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}69a59a13-75d6-4cde-8cb3-57
13ae9de7cd{ip: 172.27.26.1, host: quasar-usholy-2.quasar-usholy.root.hwx.site, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}, ReplicationConfig: RATIS/THREE, State:OPEN, leaderId:c9da661f-99ae-4de3-8eb8-306763283723, CreationTimestamp2022-02-14T05:43:11.228Z[UTC]]. The last committed block length is 0, uncommitted data length is 33554432 retry count 0
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7104
## How was this patch tested?
Tested it out on the local cluster, the messages will get suppressed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on code in PR #3671:
URL: https://github.com/apache/ozone/pull/3671#discussion_r952722710
##########
hadoop-ozone/dist/src/shell/ozone/ozone:
##########
@@ -187,6 +187,7 @@ function ozonecmd_case
;;
fs)
OZONE_CLASSNAME=org.apache.hadoop.fs.ozone.OzoneFsShell
+ OZONE_CLIENT_OPTS="${OZONE_CLIENT_OPTS} -Dhadoop.log.file=ozone-shell.log -Dlog4j.configuration=file:${ozone_shell_log4j}"
Review Comment:
I would suggest adding it in a different order, to avoid the problem HDDS-7153 is fixing for `ozone sh`.
```suggestion
OZONE_CLIENT_OPTS="-Dhadoop.log.file=ozone-shell.log -Dlog4j.configuration=file:${ozone_shell_log4j} ${OZONE_CLIENT_OPTS}"
```
##########
hadoop-ozone/dist/src/shell/conf/ozone-shell-log4j.properties:
##########
@@ -31,4 +31,4 @@ log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR
log4j.logger.org.apache.ratis.conf.ConfUtils=WARN
log4j.logger.org.apache.hadoop.security.ShellBasedUnixGroupsMapping=ERROR
log4j.logger.org.apache.ratis.grpc.client.GrpcClientProtocolClient=WARN
-log4j.logger.org.apache.hadoop.io.retry.RetryInvocationHandler=WARN
+log4j.logger.org.apache.hadoop.io.retry.RetryInvocationHandler=WARN
Review Comment:
Nit: This file now only has whitespace change. Can you please omit this from the PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ArafatKhan2198 commented on pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
ArafatKhan2198 commented on PR #3671:
URL: https://github.com/apache/ozone/pull/3671#issuecomment-1219598017
> Thanks @ArafatKhan2198 , can you tried other fs/CLI commands, to see if their WARNING messages are resolved by this PR?
I have tested out most of the FileSystem commands out there and none of them give out the suppressed warnings :-
1. Creating Volumes
2. Creating FSO and OBS buckets
3. Creating directories inside buckets
4. Listing the buckets
5. Creating keys inside buckets
6. Move Commands on keys
7. Deleting a key
8. Deleting a Bucket
9. Deleting a Volume
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi merged pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
ChenSammi merged PR #3671:
URL: https://github.com/apache/ozone/pull/3671
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ArafatKhan2198 commented on a diff in pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
ArafatKhan2198 commented on code in PR #3671:
URL: https://github.com/apache/ozone/pull/3671#discussion_r952777630
##########
hadoop-ozone/dist/src/shell/conf/ozone-shell-log4j.properties:
##########
@@ -31,4 +31,4 @@ log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR
log4j.logger.org.apache.ratis.conf.ConfUtils=WARN
log4j.logger.org.apache.hadoop.security.ShellBasedUnixGroupsMapping=ERROR
log4j.logger.org.apache.ratis.grpc.client.GrpcClientProtocolClient=WARN
-log4j.logger.org.apache.hadoop.io.retry.RetryInvocationHandler=WARN
+log4j.logger.org.apache.hadoop.io.retry.RetryInvocationHandler=WARN
Review Comment:
Understood will do !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ChenSammi commented on pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
ChenSammi commented on PR #3671:
URL: https://github.com/apache/ozone/pull/3671#issuecomment-1211676345
Thanks @ArafatKhan2198 , can you tried other fs/CLI commands, to see if their WARNING messages are resolved by this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ArafatKhan2198 commented on a diff in pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
ArafatKhan2198 commented on code in PR #3671:
URL: https://github.com/apache/ozone/pull/3671#discussion_r952766124
##########
hadoop-ozone/dist/src/shell/ozone/ozone:
##########
@@ -187,6 +187,7 @@ function ozonecmd_case
;;
fs)
OZONE_CLASSNAME=org.apache.hadoop.fs.ozone.OzoneFsShell
+ OZONE_CLIENT_OPTS="${OZONE_CLIENT_OPTS} -Dhadoop.log.file=ozone-shell.log -Dlog4j.configuration=file:${ozone_shell_log4j}"
Review Comment:
Understood will do !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ArafatKhan2198 commented on pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
ArafatKhan2198 commented on PR #3671:
URL: https://github.com/apache/ozone/pull/3671#issuecomment-1211031486
@ChenSammi @siddhantsangwan can you help review this !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] ArafatKhan2198 commented on pull request #3671: HDDS-7104. Remove WARNINGs while running ozone fs/CLI commands.
Posted by GitBox <gi...@apache.org>.
ArafatKhan2198 commented on PR #3671:
URL: https://github.com/apache/ozone/pull/3671#issuecomment-1223676452
> @ArafatKhan2198 `ozone-shell-log4j.properties` does not need to be changed. I think logging to a file is enough to avoid annoying users with useless warnings.
>
> However, we need to configure `ozone fs` command to use `ozone-shell-log4j.properties`:
>
> https://github.com/apache/ozone/blob/5f84fd1c9f35d82f2558b9c05dbe9c4d0b252900/hadoop-ozone/dist/src/shell/ozone/ozone#L188-L191
>
> similar to how it's done for `ozone sh`:
>
> https://github.com/apache/ozone/blob/5f84fd1c9f35d82f2558b9c05dbe9c4d0b252900/hadoop-ozone/dist/src/shell/ozone/ozone#L149-L154
>
> See also #3706 for an improvement being done for `ozone sh` to allow custom log filenames.
Yes I am testing it out now, Ill be updating the patch soon with the changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org