You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2022/01/19 12:32:11 UTC
[GitHub] [incubator-pegasus] cauchy1988 opened a new issue #884: add ‘BATCH_GET
cauchy1988 opened a new issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884
## Feature Request
**Is your feature request related to a problem? Please describe:**
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
**Describe the feature you'd like:**
<!-- A clear and concise description of what you want to happen. -->
**Describe alternatives you've considered:**
<!-- A clear and concise description of any alternative solutions or features you've considered. -->
**Teachability, Documentation, Adoption, Migration Strategy:**
<!-- If you can, explain some scenarios how users might use this, situations it would be helpful in. Any API designs, mockups, or diagrams are also helpful. -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021022365
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018397096
> Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
>
> One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
>
> Refer to `checkAndMutate`, You may need define rdb like this:
>
> ```thrift
> struct batch_request
> {
> ...
> 1:list<request> batch_list;
> ...
> }
>
> struct request {
> 1:operation_type operation; // it is very important, you need judge the request type
> 2: base.blob hash_key // this is diff from checkAndmutate, which don't include it
> 3:base.blob sort_key;
> 4:base.blob value;
> 5:i32 set_expire_ts_seconds;
> }
> ```
>
> On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
>
> This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.
Thank you, it is very helpful to me; I will think more roughly and come up with a more considerable solution, when this is done, i'll consult you for later 'niubility' advise
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018397096
> Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
>
> One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
>
> Refer to `checkAndMutate`, You may need define rdb like this:
>
> ```thrift
> struct batch_request
> {
> ...
> 1:list<request> batch_list;
> ...
> }
>
> struct request {
> 1:operation_type operation; // it is very important, you need judge the request type
> 2: base.blob hash_key // this is diff from checkAndmutate, which don't include it
> 3:base.blob sort_key;
> 4:base.blob value;
> 5:i32 set_expire_ts_seconds;
> }
> ```
>
> On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
>
> This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.
Thank you, it is very helpful to me; I will think more roughly and come up with a more considerable solution, when this is done, i'll consult you for later niubility advise
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691
Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
Refer to `checkAndMutate`, You may need define rdb like this:
```thrift
struct batch_request
{
...
1:list<request> batch_list;
...
}
struct request {
1:operation_type operation; // it is very important, you need judge the request is which type
2: base.blob hash_key // this is diff from checkAndmutate, which don't include it
3:base.blob sort_key;
4:base.blob value;
5:i32 set_expire_ts_seconds;
}
```
On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021084767
> Yeah, so I say that define two type, `read_request` and `write_request`, read support `get` and so on, write support `set`, `del` and so on.
>
> dynamodb don't have `multi` operation, We can consider that if support `batch_multi_get`.
read operation only have 'one type' that is 'get'; so if i have not misunderstood your ideas, i can implement the "batch read" rpc interface based on following thrift definitions
```
struct read_request {
1:dsn.blob hash_key;
2:dsn.blob sort_key;
}
struct batch_read_request {
1:list<read_request> requests;
}
struct read_response {
1:dsn.blob hash_key;
2:dsn.blob sort_key;
3:dsn.blob value;
4:bool exists // true mean data exists, false means not
}
struct batch_read_response {
1:i32 error;
2:list<read_response> data;
3:i32 app_id;
4:i32 partition_index;
6:string server;
}
```
it's almostly the same with what i have initally comed up;
'batch set' and ' batch del' rpc interface use common thrift definition and add a 'operation_type' field to the definition to differentiate them ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] acelyc111 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019689019
checkAndMutate is a SET type interface in all cases, it is different from batchGet/Set/Del interfaces.
1. The responses are different, for SET/DEL type interfaces, there are no values responed, but for GET type interfaces, you should design a thrfit struct to contain values, and the related hashkeys and sortkeys along.
2. Mix all type of requests in one RPC will make code coupled. Imagine the pains we suffered from `multi_get_request`.
So I support to distinguish these RPCs, and try to avoid overdesign.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021046213
Yeah, so I say that define two type, `read_request` and `write_request`, read support `get` and so on, write support `set`, `del` and so on.
dynamodb don't have `multi` operation, We can consider that if support `batch_multi_get`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021085689
> Yeah, so I say that define two type, `read_request` and `write_request`, read support `get` and so on, write support `set`, `del` and so on.
>
> dynamodb don't have `multi` operation, We can consider that if support `batch_multi_get`.
if i misunderstood you ideas, then i want chat with you in weixin; heihei
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] acelyc111 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
acelyc111 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019689019
checkAndMutate is a SET type interface in all cases, it is different from batchGet/Set/Del interfaces.
1. The responses are different, for SET/DEL type interfaces, there are no values responed, but for GET type interfaces, you should design a thrfit struct to contain values, and the related hashkeys and sortkeys along.
2. Mix all type of requests in one RPC will make code coupled. Imagine the pains we suffered from `multi_get_request`.
So I support to distinguish these RPCs, and try to avoid overdesign.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019469312
> > You may need define rdb like this:
>
> You may need provide two rdb type for `on_client_write` and `on_client_read`
I feel that adding a batch interface that mixes read, write, and delete operations is not very appropriate, for the following reasons:
1. The example of check_and_mutate you quoted is an example of cas-like operation on a hashkey, which is different from the usual read, write, and delete.
2. The rpc interface with a single responsibility is simple and clear. As far as the kv database is concerned, there are actually not many rpc interfaces for real reading and writing.
3. 'BATCH_GET' rpc interface style is consistent with the previous interface styles of GET, SET, MULTI_GET, etc.
4. The subsequent functions of "batch set" and "batch del" are still under discussion. I think pegasus may not support these two functions, because the success of "partial modification" will bring additional design and consideration, while "batch get" is only for reading, Even if the read fails, it will not affect the data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] acelyc111 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021043216
Doesn't dynamodb's BatchWriteItem API only operate on puts and deletes ? BatchGetItem is for get operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018397096
> Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
>
> One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
>
> Refer to `checkAndMutate`, You may need define rdb like this:
>
> ```thrift
> struct batch_request
> {
> ...
> 1:list<request> batch_list;
> ...
> }
>
> struct request {
> 1:operation_type operation; // it is very important, you need judge the request type
> 2: base.blob hash_key // this is diff from checkAndmutate, which don't include it
> 3:base.blob sort_key;
> 4:base.blob value;
> 5:i32 set_expire_ts_seconds;
> }
> ```
>
> On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
>
> This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.
Thank you, it is very helpful to me; I will think more roughly and come up a more considerable solution, when this is done, i'll consult you for later niubility advise
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691
Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
Refer to `checkAndMutate`, You may need define rdb like this:
```thrift
struct batch_request
{
...
1:list<request> batch_list;
...
}
struct request {
1:operation_type operation; // it is very important, you need judge the request type
2: base.blob hash_key // this is diff from checkAndmutate, which don't include it
3:base.blob sort_key;
4:base.blob value;
5:i32 set_expire_ts_seconds;
}
```
On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018250264
> You may need define rdb like this:
You may need provide two rdb type for `on_client_write` and `on_client_read`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019469312
> > You may need define rdb like this:
>
> You may need provide two rdb type for `on_client_write` and `on_client_read`
I feel that adding a batch interface that mixes read, write, and delete operations is not very appropriate, for the following reasons:
1. The example of check_and_mutate you quoted is an example of cas-like operation on a hashkey, which is different from the usual read, write, and delete.
2. The rpc interfacethe with a single responsibility is simple and clear. As far as the kv database is concerned, there are actually not many rpc interfaces for real reading and writing.
3. 'BATCH_GET' rpc interface style is consistent with the previous interface styles of GET, SET, MULTI_GET, etc.
4. The subsequent functions of "batch set" and "batch del" are still under discussion. I think pegasus may not support these two functions, because the success of "partial modification" will bring additional design and consideration, while "batch get" is only for reading, Even if the read fails, it will not affect the data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021022365
## 1
> 1. there are no values responed, but for GET type interfaces,
Some explanations added before as follow:
> You may need provide two rdb type for on_client_write and on_client_read.
So actually we need define `wrtite_request` and `read_request` rpc
## 2
> 2. Mix all type of requests in one RPC will make code coupled
On the contrary, I think that using a unified interface will make code maintenance easier. In addition, we can refer https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html to consider it again
## 3
> 4. The subsequent functions of "batch set" and "batch del" are still under discussion. I think pegasus may not support these two functions, because the success of "partial modification" will bring additional design and consideration
Haha, actually, I hope support `batch set` to improve online irrigation data rate.
I think may need others join to decide it. @neverchanje @hycdong @levy5307 @Smityz @GiantKing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021084767
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] acelyc111 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021043216
Doesn't dynamodb's BatchWriteItem API only operate on puts and deletes ? BatchGetItem is for get operations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691
Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
Refer to `checkAndMutate`, You may need define rdb like this:
```thrift
struct batch_request
{
...
1:list<request> batch_list;
...
}
struct request {
1:operation_type operation; // it is very important, you need judge the request type
2: base.blob hash_key // this is diff from checkAndmutate, which don't include it
3:base.blob sort_key;
4:base.blob value;
5:i32 set_expire_ts_seconds;
}
```
On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization
Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691
Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
Refer to `checkAndMutate`, You may need define rdb like this:
```thrift
struct batch_request
{
...
1:list<request> batch_list;
...
}
struct request {
1:operation_type operation;
2: base.blob hash_key // this is diff from checkAndmutate, which don't include it
3:base.blob sort_key;
4:base.blob value;
5:i32 set_expire_ts_seconds;
}
```
On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org