You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@teaclave.apache.org by GitBox <gi...@apache.org> on 2022/01/30 20:06:19 UTC
[GitHub] [incubator-teaclave] qinkunbao opened a new pull request #603: Support optional input/output files
qinkunbao opened a new pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603
## Description
This PR introduces optional input/output files.
For some tasks in biometric computing, the number of input/output files varies case by case. Therefore, this PR adds a new feature called optional files. Once a file is marked with `option: True`, users do not need to register the files before creating tasks. The C, Python, Rust SDKs have been updated accordingly.
Fixes # (issue)
## Type of change (select or add applied and delete the others)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] API change with a documentation update
- [ ] Additional test coverage
- [ ] Code cleanup or just sync with upstream third-party crates
## How has this been tested?
## Checklist
- [x] Fork the repo and create your branch from `master`.
- [x] If you've added code that should be tested, add tests.
- [x] If you've changed APIs, update the documentation.
- [x] Ensure the tests pass (see CI results).
- [x] Make sure your code lints/format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] dingelish commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
dingelish commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r798220805
##########
File path: types/src/task_state.rs
##########
@@ -135,12 +136,29 @@ impl Task<Create> {
// check input fkeys
let inputs_spec: HashSet<&String> = function.inputs.iter().map(|f| &f.name).collect();
- let req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ let mut req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ // If an input/output file is marked with `optional: True`, users do not need to
+ // register the file.
+ let option_inputs_spec: HashSet<&String> = function
+ .inputs
+ .iter()
+ .filter(|f| f.optional)
+ .map(|f| &f.name)
Review comment:
nsdd. pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] qinkunbao commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
qinkunbao commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r798205304
##########
File path: services/management/enclave/src/service.rs
##########
@@ -438,8 +438,8 @@ impl TeaclaveManagement for TeaclaveManagementService {
// access control: none
// when a task is created, following rules will be verified:
// 1) arugments match function definition
- // 2) input match function definition
- // 3) output match function definition
+ // 2) input files match function definition
+ // 3) output files match function definition
Review comment:
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] qinkunbao commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
qinkunbao commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r798977658
##########
File path: types/src/task_state.rs
##########
@@ -127,6 +127,7 @@ impl Task<Create> {
if !function.public {
participants.insert(function.owner.clone());
}
+ log::info!("participants {:?}", &participants);
Review comment:
I have removed this log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] qinkunbao commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
qinkunbao commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r796192000
##########
File path: sdk/python/teaclave.py
##########
@@ -222,10 +224,12 @@ class FunctionOutput:
Args:
name: Name of output data.
description: Description of the output data.
+ option: [Default: False] Data owners do not need to register the data.
"""
- def __init__(self, name: str, description: str):
+ def __init__(self, name: str, description: str, option=False):
Review comment:
Sounds good!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun merged pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
mssun merged pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
mssun commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r798834748
##########
File path: types/src/task_state.rs
##########
@@ -127,6 +127,7 @@ impl Task<Create> {
if !function.public {
participants.insert(function.owner.clone());
}
+ log::info!("participants {:?}", &participants);
Review comment:
Why do we need this log? Also, participants should be a sensitive data, this is a info log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] ya0guang commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
ya0guang commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r795997582
##########
File path: types/src/function.rs
##########
@@ -38,13 +40,15 @@ impl FunctionInput {
#[derive(Debug, Deserialize, Serialize)]
pub struct FunctionOutput {
pub name: String,
+ pub option: bool,
pub description: String,
}
impl FunctionOutput {
- pub fn new(name: impl Into<String>, description: impl Into<String>) -> Self {
+ pub fn new(name: impl Into<String>, option: bool, description: impl Into<String>) -> Self {
Review comment:
Do we need to think about future compatibility? For example, we may have some "decrypted" input file, and we can have an argument called `property`, and `property='do'` stands for decrypted, optional.
##########
File path: services/proto/src/teaclave_frontend_service.rs
##########
@@ -1072,6 +1074,7 @@ impl std::convert::TryFrom<proto::FunctionOutput> for FunctionOutput {
fn try_from(proto: proto::FunctionOutput) -> Result<Self> {
let ret = Self {
name: proto.name,
+ option: proto.option,
Review comment:
In Python API, `option` is the 3rd argument. Do you think it's better to make them consistent?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] bigdata-memory commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
bigdata-memory commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r795960747
##########
File path: sdk/python/teaclave.py
##########
@@ -222,10 +224,12 @@ class FunctionOutput:
Args:
name: Name of output data.
description: Description of the output data.
+ option: [Default: False] Data owners do not need to register the data.
"""
- def __init__(self, name: str, description: str):
+ def __init__(self, name: str, description: str, option=False):
Review comment:
Probably using `optional = False` could be clearer and it is also consistent with the usage in TOML files. thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
mssun commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r798835763
##########
File path: types/src/lib.rs
##########
@@ -20,6 +20,8 @@
#[macro_use]
extern crate sgx_tstd as std;
+extern crate log;
Review comment:
`extern crate` is not suggested. Use the log directly: e.g., "use log::info".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] qinkunbao commented on pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
qinkunbao commented on pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#issuecomment-1028618579
> Please rebase to the latest master for CI checking.
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] dingelish commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
dingelish commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r798191319
##########
File path: services/management/enclave/src/service.rs
##########
@@ -438,8 +438,8 @@ impl TeaclaveManagement for TeaclaveManagementService {
// access control: none
// when a task is created, following rules will be verified:
// 1) arugments match function definition
- // 2) input match function definition
- // 3) output match function definition
+ // 2) input files match function definition
+ // 3) output files match function definition
Review comment:
please remove additional spaces
##########
File path: types/src/task_state.rs
##########
@@ -135,12 +136,29 @@ impl Task<Create> {
// check input fkeys
let inputs_spec: HashSet<&String> = function.inputs.iter().map(|f| &f.name).collect();
- let req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ let mut req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ // If an input/output file is marked with `optional: True`, users do not need to
+ // register the file.
+ let option_inputs_spec: HashSet<&String> = function
+ .inputs
+ .iter()
+ .filter(|f| f.optional)
+ .map(|f| &f.name)
+ .collect();
+ req_input_fkeys.extend(&option_inputs_spec);
+
ensure!(inputs_spec == req_input_fkeys, "input keys mismatch");
// check output fkeys
let outputs_spec: HashSet<&String> = function.outputs.iter().map(|f| &f.name).collect();
- let req_output_fkeys: HashSet<&String> = req_output_owners.keys().collect();
+ let mut req_output_fkeys: HashSet<&String> = req_output_owners.keys().collect();
+ let option_outputs_spec: HashSet<&String> = function
+ .outputs
+ .iter()
+ .filter(|f| f.optional)
+ .map(|f| &f.name)
Review comment:
checkout filter_map https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter_map
##########
File path: types/src/task_state.rs
##########
@@ -135,12 +136,29 @@ impl Task<Create> {
// check input fkeys
let inputs_spec: HashSet<&String> = function.inputs.iter().map(|f| &f.name).collect();
- let req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ let mut req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ // If an input/output file is marked with `optional: True`, users do not need to
+ // register the file.
+ let option_inputs_spec: HashSet<&String> = function
+ .inputs
+ .iter()
+ .filter(|f| f.optional)
+ .map(|f| &f.name)
Review comment:
checkout filter_map https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.filter_map
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] qinkunbao commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
qinkunbao commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r798207225
##########
File path: types/src/task_state.rs
##########
@@ -135,12 +136,29 @@ impl Task<Create> {
// check input fkeys
let inputs_spec: HashSet<&String> = function.inputs.iter().map(|f| &f.name).collect();
- let req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ let mut req_input_fkeys: HashSet<&String> = req_input_owners.keys().collect();
+ // If an input/output file is marked with `optional: True`, users do not need to
+ // register the file.
+ let option_inputs_spec: HashSet<&String> = function
+ .inputs
+ .iter()
+ .filter(|f| f.optional)
+ .map(|f| &f.name)
Review comment:
I cannot come up with better ways of rewriting the code with `filter_map`. Do we have to replace it with the `filter_map`?
```Rust
let option_inputs_spec: HashSet<&String> = function
.outputs
.iter()
.filter_map(|f| if f.optional { Some(&f.name) } else { None })
.collect();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
mssun commented on pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#issuecomment-1028614263
Please rebase to the latest master for CI checking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on a change in pull request #603: Support optional input/output files
Posted by GitBox <gi...@apache.org>.
mssun commented on a change in pull request #603:
URL: https://github.com/apache/incubator-teaclave/pull/603#discussion_r795998306
##########
File path: sdk/python/teaclave.py
##########
@@ -222,10 +224,12 @@ class FunctionOutput:
Args:
name: Name of output data.
description: Description of the output data.
+ option: [Default: False] Data owners do not need to register the data.
"""
- def __init__(self, name: str, description: str):
+ def __init__(self, name: str, description: str, option=False):
Review comment:
@qinkunbao, I also suggest to use `optional` instead of `option` for all definitions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org