You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2022/01/19 12:32:11 UTC

[GitHub] [incubator-pegasus] cauchy1988 opened a new issue #884: add ‘BATCH_GET

cauchy1988 opened a new issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884


   ## Feature Request
   
   **Is your feature request related to a problem? Please describe:**
   <!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
   
   **Describe the feature you'd like:**
   <!-- A clear and concise description of what you want to happen. -->
   
   **Describe alternatives you've considered:**
   <!-- A clear and concise description of any alternative solutions or features you've considered. -->
   
   **Teachability, Documentation, Adoption, Migration Strategy:**
   <!-- If you can, explain some scenarios how users might use this, situations it would be helpful in. Any API designs, mockups, or diagrams are also helpful. -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021022365






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018397096


   > Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
   > 
   > One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
   > 
   > Refer to `checkAndMutate`, You may need define rdb like this:
   > 
   > ```thrift
   > struct batch_request
   > {
   > ...
   >     1:list<request>   batch_list;
   > ...
   > }
   > 
   > struct request {
   >     1:operation_type operation; // it is very important, you need judge the request  type
   >     2: base.blob       hash_key // this is diff from checkAndmutate, which don't include it
   >     3:base.blob        sort_key;
   >     4:base.blob        value;
   >     5:i32              set_expire_ts_seconds;
   > }
   > ```
   > 
   > On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
   > 
   > This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.
   
   Thank you, it is very helpful to me; I will think more roughly and come up with a more considerable solution, when this is done, i'll consult you for later 'niubility' advise 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018397096


   > Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
   > 
   > One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
   > 
   > Refer to `checkAndMutate`, You may need define rdb like this:
   > 
   > ```thrift
   > struct batch_request
   > {
   > ...
   >     1:list<request>   batch_list;
   > ...
   > }
   > 
   > struct request {
   >     1:operation_type operation; // it is very important, you need judge the request  type
   >     2: base.blob       hash_key // this is diff from checkAndmutate, which don't include it
   >     3:base.blob        sort_key;
   >     4:base.blob        value;
   >     5:i32              set_expire_ts_seconds;
   > }
   > ```
   > 
   > On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
   > 
   > This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.
   
   Thank you, it is very helpful to me; I will think more roughly and come up with a more considerable solution, when this is done, i'll consult you for later niubility advise 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691


   Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
   
   One example you can  refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
   
   Refer to `checkAndMutate`,  You may need define  rdb like this:
   ```thrift
   struct batch_request
   {
   ...
       1:list<request>   batch_list;
   ...
   }
   
   struct request {
       1:operation_type operation; // it is very important, you need judge the request is which type
       2: base.blob       hash_key // this is diff from checkAndmutate, which don't include it
       3:base.blob        sort_key;
       4:base.blob        value;
       5:i32              set_expire_ts_seconds;
   }
   ```
   
   On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
   
   
   This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021084767


   > Yeah, so I say that define two type, `read_request` and `write_request`, read support `get` and so on, write support `set`, `del` and so on.
   > 
   > dynamodb don't have `multi` operation, We can consider that if support `batch_multi_get`.
   
   read operation only have 'one type' that is 'get'; so  if i have not misunderstood your ideas, i can implement the "batch read" rpc interface based on following thrift definitions
   ```
   struct read_request {
       1:dsn.blob hash_key;
       2:dsn.blob sort_key;
   }
   
   struct batch_read_request {
       1:list<read_request> requests;
   }
   
   struct read_response {
       1:dsn.blob hash_key;
       2:dsn.blob sort_key;
       3:dsn.blob value;
       4:bool exists // true mean data exists, false means not
   }
   
   struct batch_read_response {
       1:i32               error;
       2:list<read_response>   data;
       3:i32               app_id;
       4:i32               partition_index;
       6:string            server;
   }
   ``` 
   it's almostly the same with what i have initally comed up;
   
   'batch set' and ' batch del' rpc interface use common thrift definition and  add a 'operation_type' field to the definition to differentiate them ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] acelyc111 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019689019


   checkAndMutate is a SET type interface in all cases, it is different from batchGet/Set/Del interfaces.
   1. The responses are different, for SET/DEL type interfaces, there are no values responed, but for GET type interfaces, you should design a thrfit struct to contain values, and the related hashkeys and sortkeys along.
   2. Mix all type of requests in one RPC will make code coupled. Imagine the pains we suffered from `multi_get_request`.
   So I support to distinguish these RPCs, and try to avoid overdesign.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021046213


   Yeah, so I say that define two type, `read_request` and `write_request`,  read support `get` and so on, write support `set`, `del` and so on.
   
   dynamodb don't have `multi` operation, We can consider that if support `batch_multi_get`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021085689


   > Yeah, so I say that define two type, `read_request` and `write_request`, read support `get` and so on, write support `set`, `del` and so on.
   > 
   > dynamodb don't have `multi` operation, We can consider that if support `batch_multi_get`.
   
   if i misunderstood you ideas, then i want chat  with you in weixin; heihei


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] acelyc111 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
acelyc111 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019689019


   checkAndMutate is a SET type interface in all cases, it is different from batchGet/Set/Del interfaces.
   1. The responses are different, for SET/DEL type interfaces, there are no values responed, but for GET type interfaces, you should design a thrfit struct to contain values, and the related hashkeys and sortkeys along.
   2. Mix all type of requests in one RPC will make code coupled. Imagine the pains we suffered from `multi_get_request`.
   
   So I support to distinguish these RPCs, and try to avoid overdesign.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019469312


   > > You may need define rdb like this:
   > 
   > You may need provide two rdb type for `on_client_write` and `on_client_read`
   
   I feel that adding a batch interface that mixes read, write, and delete operations is not very appropriate, for the following reasons:
   1. The example of check_and_mutate you quoted is an example of cas-like operation on a hashkey, which is different from the usual read, write, and delete.
   2. The rpc interface with a single responsibility is simple and clear. As far as the kv database is concerned, there are actually not many rpc interfaces for real reading and writing.
   3. 'BATCH_GET' rpc interface style is consistent with the previous interface styles of GET, SET, MULTI_GET, etc.
   4. The subsequent functions of "batch set" and "batch del" are still under discussion. I think pegasus may not support these two functions, because the success of "partial modification" will bring additional design and consideration, while "batch get" is only for reading, Even if the read fails, it will not affect the data
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] acelyc111 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021043216


   Doesn't dynamodb's BatchWriteItem API only operate on puts and deletes ? BatchGetItem is for get operations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018397096


   > Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
   > 
   > One example you can refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
   > 
   > Refer to `checkAndMutate`, You may need define rdb like this:
   > 
   > ```thrift
   > struct batch_request
   > {
   > ...
   >     1:list<request>   batch_list;
   > ...
   > }
   > 
   > struct request {
   >     1:operation_type operation; // it is very important, you need judge the request  type
   >     2: base.blob       hash_key // this is diff from checkAndmutate, which don't include it
   >     3:base.blob        sort_key;
   >     4:base.blob        value;
   >     5:i32              set_expire_ts_seconds;
   > }
   > ```
   > 
   > On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
   > 
   > This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.
   
   Thank you, it is very helpful to me; I will think more roughly and come up a more considerable solution, when this is done, i'll consult you for later niubility advise 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691


   Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
   
   One example you can  refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
   
   Refer to `checkAndMutate`,  You may need define  rdb like this:
   ```thrift
   struct batch_request
   {
   ...
       1:list<request>   batch_list;
   ...
   }
   
   struct request {
       1:operation_type operation; // it is very important, you need judge the request  type
       2: base.blob       hash_key // this is diff from checkAndmutate, which don't include it
       3:base.blob        sort_key;
       4:base.blob        value;
       5:i32              set_expire_ts_seconds;
   }
   ```
   
   On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
   
   
   This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018250264


   > You may need define rdb like this:
   
   You may need provide two  rdb type for `on_client_write` and `on_client_read`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1019469312


   > > You may need define rdb like this:
   > 
   > You may need provide two rdb type for `on_client_write` and `on_client_read`
   
   I feel that adding a batch interface that mixes read, write, and delete operations is not very appropriate, for the following reasons:
   1. The example of check_and_mutate you quoted is an example of cas-like operation on a hashkey, which is different from the usual read, write, and delete.
   2. The rpc interfacethe with a single responsibility is simple and clear. As far as the kv database is concerned, there are actually not many rpc interfaces for real reading and writing.
   3. 'BATCH_GET' rpc interface style is consistent with the previous interface styles of GET, SET, MULTI_GET, etc.
   4. The subsequent functions of "batch set" and "batch del" are still under discussion. I think pegasus may not support these two functions, because the success of "partial modification" will bring additional design and consideration, while "batch get" is only for reading, Even if the read fails, it will not affect the data
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021022365


   ## 1
   > 1. there are no values responed, but for GET type interfaces,
   
   Some explanations added before as follow:
   
   > You may need provide two rdb type for on_client_write and on_client_read.
   
   So actually we need define `wrtite_request` and `read_request` rpc
   
   ## 2 
   > 2. Mix all type of requests in one RPC will make code coupled
   
   On the contrary, I  think that using a unified interface will make code maintenance easier. In addition, we can refer https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchWriteItem.html to consider it again
   
   ## 3
   > 4. The subsequent functions of "batch set" and "batch del" are still under discussion. I think pegasus may not support these two functions, because the success of "partial modification" will bring additional design and consideration
    
   Haha, actually, I hope support `batch set` to improve online irrigation data rate.
   
   
   
   
   I think may need others join to decide it. @neverchanje @hycdong @levy5307 @Smityz @GiantKing 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] cauchy1988 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021084767






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] acelyc111 commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1021043216


   Doesn't dynamodb's BatchWriteItem API only operate on puts and deletes ? BatchGetItem is for get operations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia edited a comment on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia edited a comment on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691


   Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
   
   One example you can  refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
   
   Refer to `checkAndMutate`,  You may need define  rdb like this:
   ```thrift
   struct batch_request
   {
   ...
       1:list<request>   batch_list;
   ...
   }
   
   struct request {
       1:operation_type operation; // it is very important, you need judge the request  type
       2: base.blob       hash_key // this is diff from checkAndmutate, which don't include it
       3:base.blob        sort_key;
       4:base.blob        value;
       5:i32              set_expire_ts_seconds;
   }
   ```
   
   On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
   
   
   This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org


[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #884: add ‘BATCH_GET’ rpc request for read optimization

Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #884:
URL: https://github.com/apache/incubator-pegasus/issues/884#issuecomment-1018246691


   Consider whether there is a more general resolution to support all operations, including get, set, del and so on.
   
   One example you can  refer to [checkAndMutate](https://github.com/XiaoMi/pegasus-java-client/blob/master/idl/rrdb.thrift#L202), which can batch send different `write` operation, and only support same hashKey.
   
   Refer to `checkAndMutate`,  You may need define  rdb like this:
   ```thrift
   struct batch_request
   {
   ...
       1:list<request>   batch_list;
   ...
   }
   
   struct request {
       1:operation_type operation; 
       2: base.blob       hash_key // this is diff from checkAndmutate, which don't include it
       3:base.blob        sort_key;
       4:base.blob        value;
       5:i32              set_expire_ts_seconds;
   }
   ```
   
   On server side, you just need one method to judge `operation_type` and forward corresponding existed method, of course, which is also like `checkAndMutate` [handler of server side](https://github.com/apache/incubator-pegasus/blob/master/src/server/pegasus_write_service_impl.h#L358).
   
   
   This just is one suggestion, I haven't make sure whether it is feasible, However, I think the interface should be as general as possible, rather than one requirement need one interface


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org