You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/01/29 18:42:43 UTC
[GitHub] [hadoop-ozone] adoroszlai opened a new pull request #502:
HDDS-2955. Unnecessary log messages in DBStoreBuilder
adoroszlai opened a new pull request #502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
URL: https://github.com/apache/hadoop-ozone/pull/502
## What changes were proposed in this pull request?
1. Reduce log level of "using default/custom profile ..." messages to `debug`.
2. Avoid logging both "default" and "custom" for the same table by refactoring the code.
3. Add constant for `String` version of `DEFAULT_COLUMN_FAMILY`.
https://issues.apache.org/jira/browse/HDDS-2955
## How was this patch tested?
Verified log in compose cluster.
with default info level:
```
recon_1 | 2020-01-29 17:14:39,085 [pool-8-thread-1] INFO impl.OzoneManagerServiceProviderImpl: Got new checkpoint from OM : /data/metadata/recon/om.snapshot.db_1580318078998
recon_1 | 2020-01-29 17:14:39,085 [pool-8-thread-1] INFO recovery.ReconOmMetadataManagerImpl: Cleaning up old OM snapshot db at /data/metadata/recon/om.snapshot.db_1580317479076.
recon_1 | 2020-01-29 17:14:39,231 [pool-8-thread-1] INFO recovery.ReconOmMetadataManagerImpl: Created OM DB handle from snapshot at /data/metadata/recon/om.snapshot.db_1580318078998.
```
and with debug level:
```
recon_1 | 2020-01-29 17:00:12,434 [pool-8-thread-1] INFO impl.OzoneManagerServiceProviderImpl: Got new checkpoint from OM : /data/metadata/recon/om.snapshot.db_1580317212331
recon_1 | 2020-01-29 17:00:12,434 [pool-8-thread-1] INFO recovery.ReconOmMetadataManagerImpl: Cleaning up old OM snapshot db at /data/metadata/recon/om.snapshot.db_1580316612245.
recon_1 | 2020-01-29 17:00:12,438 [pool-8-thread-1] DEBUG db.DBStoreBuilder: default profile:DBProfile.DISK
recon_1 | 2020-01-29 17:00:12,438 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:userTable
recon_1 | 2020-01-29 17:00:12,442 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:volumeTable
recon_1 | 2020-01-29 17:00:12,442 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:bucketTable
recon_1 | 2020-01-29 17:00:12,443 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:keyTable
recon_1 | 2020-01-29 17:00:12,443 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:deletedTable
recon_1 | 2020-01-29 17:00:12,443 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:openKeyTable
recon_1 | 2020-01-29 17:00:12,444 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:s3Table
recon_1 | 2020-01-29 17:00:12,444 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:multipartInfoTable
recon_1 | 2020-01-29 17:00:12,444 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:dTokenTable
recon_1 | 2020-01-29 17:00:12,445 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:s3SecretTable
recon_1 | 2020-01-29 17:00:12,445 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:prefixTable
recon_1 | 2020-01-29 17:00:12,447 [pool-8-thread-1] DEBUG db.DBStoreBuilder: using default profile for table:default
recon_1 | 2020-01-29 17:00:12,451 [pool-8-thread-1] DEBUG db.DBStoreBuilder: Using default options: DBProfile.DISK
recon_1 | 2020-01-29 17:00:12,561 [pool-8-thread-1] INFO recovery.ReconOmMetadataManagerImpl: Created OM DB handle from snapshot at /data/metadata/recon/om.snapshot.db_1580317212331.
```
https://github.com/adoroszlai/hadoop-ozone/runs/415576820
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] anuengineer merged pull request #502: HDDS-2955.
Unnecessary log messages in DBStoreBuilder
Posted by GitBox <gi...@apache.org>.
anuengineer merged pull request #502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
URL: https://github.com/apache/hadoop-ozone/pull/502
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on issue #502: HDDS-2955.
Unnecessary log messages in DBStoreBuilder
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on issue #502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
URL: https://github.com/apache/hadoop-ozone/pull/502#issuecomment-585439345
Thanks @avijayanhwx and @xiaoyuyao for the review, and @anuengineer for reviewing and merging this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request
#502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
Posted by GitBox <gi...@apache.org>.
xiaoyuyao commented on a change in pull request #502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
URL: https://github.com/apache/hadoop-ozone/pull/502#discussion_r376165006
##########
File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/DBStoreBuilder.java
##########
@@ -174,21 +184,16 @@ private void processDBProfile() {
dbProfile = this.configuration.getEnum(HDDS_DB_PROFILE,
HDDS_DEFAULT_DB_PROFILE);
}
+ LOG.debug("default profile:{}", dbProfile);
}
private void processTables() throws IOException {
- if (tableNames.size() > 0) {
- for (String name : tableNames) {
- addTable(name, dbProfile.getColumnFamilyOptions());
- LOG.info("Using default column profile:{} for Table:{}",
- dbProfile.toString(), name);
- }
+ List<String> list = new ArrayList<>(tableNames);
+ list.add(DEFAULT_COLUMN_FAMILY_NAME);
+ for (String name : list) {
+ LOG.debug("using default profile for table:{}", name);
+ addTableDefinition(name, dbProfile.getColumnFamilyOptions());
}
- addTable(StringUtils.bytes2String(RocksDB.DEFAULT_COLUMN_FAMILY),
Review comment:
Can we pass the dbProfile's name to addTable so that we can LOG the profile name inside addTable?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] adoroszlai commented on a change in pull request
#502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on a change in pull request #502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
URL: https://github.com/apache/hadoop-ozone/pull/502#discussion_r377084074
##########
File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/db/DBStoreBuilder.java
##########
@@ -174,21 +184,16 @@ private void processDBProfile() {
dbProfile = this.configuration.getEnum(HDDS_DB_PROFILE,
HDDS_DEFAULT_DB_PROFILE);
}
+ LOG.debug("default profile:{}", dbProfile);
}
private void processTables() throws IOException {
- if (tableNames.size() > 0) {
- for (String name : tableNames) {
- addTable(name, dbProfile.getColumnFamilyOptions());
- LOG.info("Using default column profile:{} for Table:{}",
- dbProfile.toString(), name);
- }
+ List<String> list = new ArrayList<>(tableNames);
+ list.add(DEFAULT_COLUMN_FAMILY_NAME);
+ for (String name : list) {
+ LOG.debug("using default profile for table:{}", name);
+ addTableDefinition(name, dbProfile.getColumnFamilyOptions());
}
- addTable(StringUtils.bytes2String(RocksDB.DEFAULT_COLUMN_FAMILY),
Review comment:
Thanks @xiaoyuyao for the review.
Both previously and with this patch only the default profile name is logged, no custom profile names. (I guess the reason for that is that it's only a `ColumnFamilyOptions` object being passed to `addTable`, not a `DBProfile`, and only the latter has meaningful name). The patch changes logging of the default profile to happen only once instead of for all tables using it, to avoid repetition.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on issue #502: HDDS-2955.
Unnecessary log messages in DBStoreBuilder
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on issue #502: HDDS-2955. Unnecessary log messages in DBStoreBuilder
URL: https://github.com/apache/hadoop-ozone/pull/502#issuecomment-579997161
LGTM +1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org