You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/03/09 22:27:16 UTC
[GitHub] [hadoop-ozone] bharatviswa504 opened a new pull request #654:
HDDS-3150.
bharatviswa504 opened a new pull request #654: HDDS-3150.
URL: https://github.com/apache/hadoop-ozone/pull/654
## What changes were proposed in this pull request?
Implement getIfExist in Table and use it in CreateKey/File
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3150
## How was this patch tested?
Added UT.
And also deployed this fix on a laptop and ran to benchmark.
**Using db.getIfExist**
```
$ support-tools-0.1.0-SNAPSHOT/bin/ozone-tools benchmark om write -d=35m -w=50
Final Stats!
Time elapsed: 2100 sec.
Number of Threads: 50
Number of Keys written: 6335987
Average Key write time (CPU Time): 16.5624 milliseconds.
Average Key write time (Real): .3312 milliseconds.
Max Key write time (CPU Time): 558 milliseconds.
****************************************************************
```
Metrics
So, out of 6.33 million calls of getIfExist, only 15599 calls performed db.get and all those are misses. So, it proves that using db.getIfExist in a negative case is better.
"NumDBKeyGetIfExistChecks": 6335987, (Total calls to API)
"NumDBKeyGetIfExistMisses": 15599, (keyMay Exist returned true, we call db.get, this metric value says how many times we called db.get())
"NumDBKeyGetIfExistGets": 15599 (keyMay Exist returned true, we called db.get(), but turns out that db.get() returns null, so this is a miss where keyMayExist was not able to find out correctly)
**Without getIfExist**
```
$ support-tools-0.1.0-SNAPSHOT/bin/ozone-tools benchmark om write -d=60m -w=50
================================================================
2020-03-09T14:06:33.169
================================================================
Time elapsed: 2100 sec.
Number of Threads: 50
Number of Keys written: 5815634
Average Key write time (CPU Time): 18.0492 milliseconds.
Average Key write time (Real): .3610 milliseconds.
Max Key write time (CPU Time): 1080 milliseconds.
****************************************************************
```
**Conclusion:**
With getIfExists:
6335987 Keys created in 35 mts
Throughput: 3017 keys/sec
With get:
5815634 Keys created in 35 mts
Throughput: 2769 keys/sec
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request
#654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#discussion_r390670650
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
##########
@@ -155,6 +155,32 @@ public boolean isExist(byte[] key) throws IOException {
}
}
+ @Override
+ public byte[] getIfExist(byte[] key) throws IOException {
+ try {
+ // RocksDB#keyMayExist
+ // If the key definitely does not exist in the database, then this
+ // method returns false, else true.
+ rdbMetrics.incNumDBKeyGetIfExistChecks();
+ StringBuilder outValue = new StringBuilder();
+ boolean keyMayExist = db.keyMayExist(handle, key, outValue);
+ if (keyMayExist) {
+ // Not using out value from string builder, as that is causing
+ // IllegalArgumentException during protobuf parsing.
Review comment:
@Override
```
public byte[] get(byte[] key) throws IOException {
try {
// RocksDB#keyMayExist
// If the key definitely does not exist in the database, then this
// method returns false, else true.
rdbMetrics.incNumDBKeyIfExistChecks();
StringBuilder outValue = new StringBuilder();
boolean keyMayExist = db.keyMayExist(handle, key, outValue);
if (keyMayExist) {
byte[] val;
if (outValue.length() > 0) {
val = outValue.toString().getBytes(UTF_8);
} else {
val = db.get(handle, key);
}
if (val != null) {
rdbMetrics.incNumDBKeyIfExistMisses();
}
return val;
}
return null;
} catch (RocksDBException e) {
throw toIOException(
"Error in accessing DB. ", e);
}
}
```
I tried with the above code, getting IllegalException during parsing. So, for now, removed not to use outVal.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] elek commented on issue #654: HDDS-3150. Implement
getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
elek commented on issue #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#issuecomment-597632158
> How was this patch tested?
> $ support-tools-0.1.0-SNAPSHOT/bin/ozone-tools benchmark om write -d=35m -w=50
Can you please share more details how was it tested (how can I reproduce the test?). This seems to be an external tool which is not included and I am not familiar with it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] mukul1987 commented on a change in pull request
#654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
mukul1987 commented on a change in pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#discussion_r390066571
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
##########
@@ -155,6 +155,32 @@ public boolean isExist(byte[] key) throws IOException {
}
}
+ @Override
+ public byte[] getIfExist(byte[] key) throws IOException {
+ try {
+ // RocksDB#keyMayExist
+ // If the key definitely does not exist in the database, then this
+ // method returns false, else true.
+ rdbMetrics.incNumDBKeyGetIfExistChecks();
+ StringBuilder outValue = new StringBuilder();
+ boolean keyMayExist = db.keyMayExist(handle, key, outValue);
+ if (keyMayExist) {
+ // Not using out value from string builder, as that is causing
+ // IllegalArgumentException during protobuf parsing.
Review comment:
whats the reason for the illegal arguement exception ? Do we have a stack trace?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request
#654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on a change in pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#discussion_r390664487
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
##########
@@ -155,6 +155,32 @@ public boolean isExist(byte[] key) throws IOException {
}
}
+ @Override
+ public byte[] getIfExist(byte[] key) throws IOException {
+ try {
+ // RocksDB#keyMayExist
+ // If the key definitely does not exist in the database, then this
+ // method returns false, else true.
+ rdbMetrics.incNumDBKeyGetIfExistChecks();
+ StringBuilder outValue = new StringBuilder();
+ boolean keyMayExist = db.keyMayExist(handle, key, outValue);
+ if (keyMayExist) {
+ // Not using out value from string builder, as that is causing
+ // IllegalArgumentException during protobuf parsing.
Review comment:
Does this mean we cannot use the outValue that we pass in? If 'keyMayExist' returns true, and the value is indeed present in the block cache, I believe the outValue will have the data.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] nandakumar131 merged pull request #654: HDDS-3150.
Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
nandakumar131 merged pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on issue #654:
HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on issue #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#issuecomment-597708204
> > How was this patch tested?
> > $ support-tools-0.1.0-SNAPSHOT/bin/ozone-tools benchmark om write -d=35m -w=50
>
> Can you please share more details how was it tested (how can I reproduce the test?). This seems to be an external tool which is not included and I am not familiar with it.
https://github.com/nandakumar131/support-tools
This is the repo used for testing. In this test used OM and SCM only, no data path involved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request
#654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#discussion_r390670650
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
##########
@@ -155,6 +155,32 @@ public boolean isExist(byte[] key) throws IOException {
}
}
+ @Override
+ public byte[] getIfExist(byte[] key) throws IOException {
+ try {
+ // RocksDB#keyMayExist
+ // If the key definitely does not exist in the database, then this
+ // method returns false, else true.
+ rdbMetrics.incNumDBKeyGetIfExistChecks();
+ StringBuilder outValue = new StringBuilder();
+ boolean keyMayExist = db.keyMayExist(handle, key, outValue);
+ if (keyMayExist) {
+ // Not using out value from string builder, as that is causing
+ // IllegalArgumentException during protobuf parsing.
Review comment:
```
@Override
public byte[] get(byte[] key) throws IOException {
try {
// RocksDB#keyMayExist
// If the key definitely does not exist in the database, then this
// method returns false, else true.
rdbMetrics.incNumDBKeyIfExistChecks();
StringBuilder outValue = new StringBuilder();
boolean keyMayExist = db.keyMayExist(handle, key, outValue);
if (keyMayExist) {
byte[] val;
if (outValue.length() > 0) {
val = outValue.toString().getBytes(UTF_8);
} else {
val = db.get(handle, key);
}
if (val != null) {
rdbMetrics.incNumDBKeyIfExistMisses();
}
return val;
}
return null;
} catch (RocksDBException e) {
throw toIOException(
"Error in accessing DB. ", e);
}
}
```
I tried with the above code, getting IllegalException during parsing. So, for now, removed not to use outVal.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request
#654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#discussion_r390670650
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
##########
@@ -155,6 +155,32 @@ public boolean isExist(byte[] key) throws IOException {
}
}
+ @Override
+ public byte[] getIfExist(byte[] key) throws IOException {
+ try {
+ // RocksDB#keyMayExist
+ // If the key definitely does not exist in the database, then this
+ // method returns false, else true.
+ rdbMetrics.incNumDBKeyGetIfExistChecks();
+ StringBuilder outValue = new StringBuilder();
+ boolean keyMayExist = db.keyMayExist(handle, key, outValue);
+ if (keyMayExist) {
+ // Not using out value from string builder, as that is causing
+ // IllegalArgumentException during protobuf parsing.
Review comment:
@Override
public byte[] get(byte[] key) throws IOException {
try {
// RocksDB#keyMayExist
// If the key definitely does not exist in the database, then this
// method returns false, else true.
rdbMetrics.incNumDBKeyIfExistChecks();
StringBuilder outValue = new StringBuilder();
boolean keyMayExist = db.keyMayExist(handle, key, outValue);
if (keyMayExist) {
byte[] val;
if (outValue.length() > 0) {
val = outValue.toString().getBytes(UTF_8);
} else {
val = db.get(handle, key);
}
if (val != null) {
rdbMetrics.incNumDBKeyIfExistMisses();
}
return val;
}
return null;
} catch (RocksDBException e) {
throw toIOException(
"Error in accessing DB. ", e);
}
}
I tried with the above code, getting IllegalException during parsing. So, for now, removed not to use outVal.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request
#654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#discussion_r390669860
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
##########
@@ -155,6 +155,32 @@ public boolean isExist(byte[] key) throws IOException {
}
}
+ @Override
+ public byte[] getIfExist(byte[] key) throws IOException {
+ try {
+ // RocksDB#keyMayExist
+ // If the key definitely does not exist in the database, then this
+ // method returns false, else true.
+ rdbMetrics.incNumDBKeyGetIfExistChecks();
+ StringBuilder outValue = new StringBuilder();
+ boolean keyMayExist = db.keyMayExist(handle, key, outValue);
+ if (keyMayExist) {
+ // Not using out value from string builder, as that is causing
+ // IllegalArgumentException during protobuf parsing.
Review comment:
Yes, using that is causing the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on issue #654: HDDS-3150.
Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on issue #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#issuecomment-596850620
Why do we need a new API for this? Can the caller not just use the isExist API first and then get?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #654: HDDS-3150.
Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#issuecomment-596895075
> Why do we need a new API for this? Can the caller not just use the isExist API first and then get?
Because if we use isExist(), if keyMayExist returns true, then we will do 2 times db.get
isExist()
```
boolean keyMayExist = db.keyMayExist(handle, key, outValue);
if (keyMayExist) {
boolean keyExists = (outValue.length() > 0) ||
(db.get(handle, key) != null);
```
Now to use in OMKeyCreateRequest
```
if(omMetadataManager.getKeyTable().isExist(dbKeyName)) {
dbKeyInfo =
omMetadataManager.getKeyTable().getIfExist(dbKeyName);
.
.
```
So if keyMayExist returns true, then we do 2 times db.get, to avoid that new API is introduced.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request
#654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on a change in pull request #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#discussion_r390089153
##########
File path: hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBTable.java
##########
@@ -155,6 +155,32 @@ public boolean isExist(byte[] key) throws IOException {
}
}
+ @Override
+ public byte[] getIfExist(byte[] key) throws IOException {
+ try {
+ // RocksDB#keyMayExist
+ // If the key definitely does not exist in the database, then this
+ // method returns false, else true.
+ rdbMetrics.incNumDBKeyGetIfExistChecks();
+ StringBuilder outValue = new StringBuilder();
+ boolean keyMayExist = db.keyMayExist(handle, key, outValue);
+ if (keyMayExist) {
+ // Not using out value from string builder, as that is causing
+ // IllegalArgumentException during protobuf parsing.
Review comment:
Because value returned from StringBuilder then to a string, which is converted to bytes[] and then later if it is used in parseFrom byte[] data.
```
java.lang.IllegalArgumentException: Can't encode the the raw data from the byte array
at org.apache.hadoop.ozone.om.codec.OmKeyInfoCodec.fromPersistedFormat(OmKeyInfoCodec.java:48)
at org.apache.hadoop.ozone.om.codec.OmKeyInfoCodec.fromPersistedFormat(OmKeyInfoCodec.java:31)
at org.apache.hadoop.hdds.utils.db.CodecRegistry.asObject(CodecRegistry.java:55)
at org.apache.hadoop.hdds.utils.db.TypedTable.getFromTableIfExist(TypedTable.java:207)
at org.apache.hadoop.hdds.utils.db.TypedTable.getIfExist(TypedTable.java:194)
at org.apache.hadoop.ozone.om.request.key.OMKeyCreateRequest.validateAndUpdateCache(OMKeyCreateRequest.java:196)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleWriteRequest(OzoneManagerRequestHandler.java:230)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequestDirectlyToOM(OzoneManagerProtocolServerSideTranslatorPB.java:210)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:130)
at org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:72)
at org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:98)
at org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #654: HDDS-3150.
Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 commented on issue #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#issuecomment-597708204
> > How was this patch tested?
> > $ support-tools-0.1.0-SNAPSHOT/bin/ozone-tools benchmark om write -d=35m -w=50
>
> Can you please share more details how was it tested (how can I reproduce the test?). This seems to be an external tool which is not included and I am not familiar with it.
https://github.com/nandakumar131/support-tools
This is the repo used for testing. Which uses OM and SCM only.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] bharatviswa504 edited a comment on issue #654:
HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
Posted by GitBox <gi...@apache.org>.
bharatviswa504 edited a comment on issue #654: HDDS-3150. Implement getIfExist in Table and use it in CreateKey/File
URL: https://github.com/apache/hadoop-ozone/pull/654#issuecomment-596895075
> Why do we need a new API for this? Can the caller not just use the isExist API first and then get?
Because if we use isExist(), if keyMayExist returns true, then we will do 2 times db.get
isExist()
```
boolean keyMayExist = db.keyMayExist(handle, key, outValue);
if (keyMayExist) {
boolean keyExists = (outValue.length() > 0) ||
(db.get(handle, key) != null);
```
Now to use in OMKeyCreateRequest
```
if(omMetadataManager.getKeyTable().isExist(dbKeyName)) {
dbKeyInfo =
omMetadataManager.getKeyTable().get(dbKeyName);
.
.
```
So if keyMayExist returns true, then we do 2 times db.get, to avoid that new API is introduced.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org