[GitHub] [incubator-teaclave] litongxin1991 commented on issue #138: [RFC][Do Not Merge] RPC protocols

[GitBox <gi...@apache.org>]

litongxin1991 commented on issue #138: [RFC][Do Not Merge] RPC protocols
URL: https://github.com/apache/incubator-teaclave/pull/138#issuecomment-568115612
 
 
   # Description for Task protocols
   This is a draft protocol for Task.
   
   Task defines a computation process, that means, the computation logic (Function) and the input/output data (Data). 
   ## Task in Clients' View
   ```
   message TaskAgreement {
       // Task identifier
       required string task_id = 1;
       // Function identifier
       required string function_id = 2;
       // Input arguments
       map<string, InputArg> input_arguments = 3;
       // Output arguments
       map<string, OutputArg> output_argments = 4;
       // Participants
       repeated string participants = 5;
       // Signatures of the participants
       map<string, bytes> signatures = 6;
   } 
   ```
   Participants need to consent to the ```TaskAgreement```.  They can know what function will be executed, who provides the certain input, who will get the certain output.  
   
   ```TaskAgreement``` doesn't contain the script content or input content, which are represented by function_id and data_id.
   ```
   // input argument in TaskAgreement
   message InputArg {
       // private input or fusion input
       required bool is_private = 1;
       // owners of the input: private - single owner, fusion - multiple owners
       repeated string owners = 2;
       // data id of the input
       required string data_id = 3;
   }
   // output argument in TaskAgreement
   message OutputArg {
       // private output or fusion output
       required bool is_private = 1;
       // owners of the input: private - single owner, fusion - multiple owners
       repeated string owners = 2;
       // data id of the output
       // fusion output may not have a data id before the TaskAgreement is accepted by the TMS
       optional string data_id = 3;
   }
   ```
   
   Because there is no sensitive information in the ```TaskAgreement```, multiple participants can consent to the TaskAgreement offline. Participants need to approve the ```Task``` by signing the ```TaskAgreement```
   
   And an output can belong to multiple users (data fusion), no one can use or access it without all the owners' approvals. Owners can get the data id of a fusion output after the ```Task``` is accepted by the platform.  
   
   ## Task in Workers' View
   ```
   message ExecutableTask {
     // Task identifer
     required string task_id = 1;
     // Task token is used for the worker to update the Task status and output meta
     required string task_token = 2;
     // Function information
     required Function function = 3;
     // Input Arguments
     map<string, InputArg> input_arguments = 4;
     // Output Arguments
     map<string, OutputArg> output_argments = 5;
   }
   ```
   For workers, ```ExecutableTask``` provides all the required information (function and input/output) to finish a computation process without the need to query additional information from the platform. 
   ### Function
   - If ```Function``` is a built-in function, then the ```Function``` is represented by a pre-defined function id.
   - If ```Function``` is a script, then the ```Function``` is represented by the bytes of the script content.
   ```
   // Function information
   message Function {
     // Built-in function only needs function_id
     optional string function_id = 1;
     // script content
     required bytes script = 2;
   }
   ```
   ### Input Data
   Workers don't need to know the data id of an input data. But they need to know where to access the data, how to decrypt the data and how to verify the data integrity before use.
   So in ```ExecutableTask```, the input data is represented by the meta information, physical access information and logical access information.
   ```
   // Input Arugment
   message InputArg {
     // Meta information is used to verify the data integrity. 
     required data_common.DataMeta meta = 1; 
     // Storage information is used to access the physical data. 
     required data_common.DataStorageInfo storage_info = 2;
     // Key config is used to decrypt the data.
     required kms_proto.KeyConfig config = 3; 
   }
   ``` 
   ### Output Data
   Workers don't need to know the data id of an output data. But they need to know where to to access the data and how to encrypt the data.
   So in ```ExecutableTask```, the output data is represented by the physical information and logical access information.
   ```
   // Output Argument
   message OutputArg {
     // Storage information is used to access the physical data. 
     required data_common.DataStorageInfo storage_info = 1;
     // Key config is used to encrypt the data.
     required kms_proto.KeyConfig config = 2; 
   }
   ```
   ### Stateless
   Because the ```ExecutableTask``` contains all the required information and a worker doesn't need to query additional information from the platform, a worker is considered as a stateless computation component. 
   
   Once the function and data are prepared, the worker can be used to do the computation without the platform. Thus, we can directly test a worker without the platform.
   
   ## RPC 
   ### TaskService for User
   see mesatee_services/tms/tms_proto/src/proto/task_external.proto
   ```
   service TaskService {
       // Client can use this request to propose a Task
       rpc RegisterTask(RegisterTaskRequest) returns (RegisterTaskResponse);
       // Client can use this request to query Task information.
       rpc GetTask(GetTaskRequest) returns (GetTaskResponse);
       // Client can use this request to invoke a Task.
       rpc InvokeTask(InvokeTaskRequest) returns (InvokeTaskResponse);
   }
   ```
   ### TaskService for Platform
   see mesatee_services/tms/tms_proto/src/proto/task_internal.proto
   ```
   service TaskService {
     // Worker can use this request to ask the scheduler to update the task status and output meta.
     // Scheduler can use this request to ask TMS to update the task status and output meta
     rpc UpdateTask(UpdateTaskRequest) returns (UpdateTaskResponse);
   }
   ```
   
    
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org