You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@teaclave.apache.org by GitBox <gi...@apache.org> on 2020/06/02 09:59:13 UTC
[GitHub] [incubator-teaclave] czzmmc opened a new issue #329: How to register local files?
czzmmc opened a new issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329
Hi,
Registering files in Teaclave is upgraded by adding a new feature that it can support remote files, which is really great. But does it still supports local files? If do, how should I do? THX
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] yc-huang commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
yc-huang commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-650944770
Need to make sure that the local file url is accessible from the teaclave-execution-service but not the teaclave client.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
mssun commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-649191686
Thanks @yc-huang. I think I find the issue.
When registering the function, you should also provide input information if used. Here is an example:
https://github.com/apache/incubator-teaclave/blob/70f02555f58ad3f58640f75f4aadab3307230a0b/examples/python/builtin_gbdt_train.py#L36-L48
The register_function API is like this: https://teaclave.apache.org/docs/client-sdk-python/teaclave.html#teaclave.FrontendClient.register_function
Sorry, the document of this API is not completed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] yc-huang commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
yc-huang commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-649186151
Here's the python client:
cat incubator-teaclave/examples/python/builtin_rustface.py
```
#!/usr/bin/env python3
import sys
import base64
from teaclave import (AuthenticationService, FrontendService,
AuthenticationClient, FrontendClient, DataMap)
from utils import (AUTHENTICATION_SERVICE_ADDRESS, FRONTEND_SERVICE_ADDRESS,
AS_ROOT_CA_CERT_PATH, ENCLAVE_INFO_PATH, USER_ID,
USER_PASSWORD)
class BuiltinRustfaceExample:
def __init__(self, user_id, user_password):
self.user_id = user_id
self.user_password = user_password
def detect_face(self, image_base64):
client = AuthenticationService(
AUTHENTICATION_SERVICE_ADDRESS, AS_ROOT_CA_CERT_PATH,
ENCLAVE_INFO_PATH).connect().get_client()
print("[+] registering user")
client.user_register(self.user_id, self.user_password)
print("[+] login")
token = client.user_login(self.user_id, self.user_password)
client = FrontendService(FRONTEND_SERVICE_ADDRESS,
AS_ROOT_CA_CERT_PATH,
ENCLAVE_INFO_PATH).connect().get_client()
metadata = {"id": self.user_id, "token": token}
client.metadata = metadata
print("[+] registering key file")
image_id = register_input_file(client)
print("[+] key file id" + image_id)
print("[+] registering function")
function_id = client.register_function(
name="builtin-rustface-detector",
description="Native Face Detection Function",
executor_type="builtin",
arguments=["image_base64"])
print("[+] creating task")
task_id = client.create_task(function_id=function_id,
function_arguments={"image_base64": image_base64},
executor="builtin")
print("[+] assigning data to task")
client.assign_data_to_task(task_id, [DataMap("image_key", image_id)], [])
print("[+] invoking task")
client.invoke_task(task_id)
print("[+] getting result")
result = client.get_task_result(task_id)
print("[+] done")
return bytes(result)
def register_input_file(client):
url = "file:///home/ubuntu/incubator-teaclave/1.jpg.enc"
cmac = "cf8a2eec4ab2fb53cd5c0b2ce3c0a828"
schema = "teaclave-file-128"
key = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3]
iv = []
key_data_id = client.register_input_file(url, schema, key, iv, cmac)
return key_data_id
def main():
example = BuiltinRustfaceExample(USER_ID, USER_PASSWORD)
if len(sys.argv) > 1:
image = sys.argv[1]
else:
image = "1.jpg"
with open(image, 'rb') as binary_file:
binary_file_data = binary_file.read()
base64_encoded_data = base64.b64encode(binary_file_data)
base64_message = base64_encoded_data.decode('utf-8')
rt = example.detect_face(base64_message)
print("[+] function return: ", rt)
if __name__ == '__main__':
main()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] yc-huang edited a comment on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
yc-huang edited a comment on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-648703515
I'm trying to use file agent to pass a local file as well and failed with "permission denied" error, do you have any clue that I might missed? Thanks...
Following are the code snippet:
--------------------------------------------------------------
in the builtin function:
`let mut f = _runtime.open_input("image_key")?;`
and in the client:
```
image_id = register_input_file(client)
...
print("[+] assigning data to task")
client.assign_data_to_task(task_id, [DataMap("image_key", image_id)], [])
...
def register_input_file(client):
url = "file:///home/ubuntu/incubator-teaclave/1.jpg.enc"
cmac = "cf8a2eec4ab2fb53cd5c0b2ce3c0a828"
schema = "teaclave-file-128"
key = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3]
iv = []
key_data_id = client.register_input_file(url, schema, key, iv, cmac)
return key_data_id
```
----------------------------------------------------------
and following are the errors (I have modified teaclave.py to print out the response)
> [+] assigning data to task
> b'{"result":"err","request_error":"permission denied"}'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] yc-huang edited a comment on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
yc-huang edited a comment on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-649258156
Thanks @mssun and @qinkunbao , we need to specify the "input" parameter when register the function and "inputs_ownership" when create the task:
```
function_id = client.register_function(
name="builtin-rustface-detector",
description="Native Face Detection Function",
executor_type="builtin",
inputs=[FunctionInput("image_key", "Input key file.")],
arguments=["image_base64"])
task_id = client.create_task(function_id=function_id,
function_arguments={"image_base64": image_base64},
inputs_ownership=[OwnerList("image_key", [self.user_id])],
executor="builtin")
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] yc-huang commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
yc-huang commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-649258156
Thanks @mssun and @qinkunbao , we need to specify the "input" parameter when register the function and "inputs_ownership" when create the task:
```
function_id = client.register_function(
name="builtin-rustface-detector",
description="Native Face Detection Function",
executor_type="builtin",
inputs=[FunctionInput("image_key", "Input key file.")],
arguments=["image_base64"])
task_id = client.create_task(function_id=function_id,
function_arguments={"image_base64": image_base64},
inputs_ownership=[OwnerList("image_key", [self.user_id])],
executor="builtin")
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun edited a comment on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
mssun edited a comment on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-637699653
You can find a component called file agent (https://github.com/apache/incubator-teaclave/tree/master/file_agent), which provides functionalities to interactive (download and upload files) with data storage services. As you can see from the code, the `file://` schema is for local file storage. That is to say, you can register local files with `file://` in the prefix of the URL.
However, note that the `file://` schema is not recommended right now. We need time to improve the security and usability.
At last, I don't really understand why you want to use local files. Does this mean your client and services are in the same machine?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
mssun commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-648972881
This is not a issue about registering local files.
A lot of cases can cause "permission denied". How do you register the input file and create the task? If possible, can you provide the sources of the client side? Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] yc-huang commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
yc-huang commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-648703515
I'm trying to use file agent to pass a local file as well and failed with "permission denied" error, do you have any clue that I might missed? Thanks...
Following are the code snippet:
--------------------------------------------------------------
in the builtin function:
let mut f = _runtime.open_input("image_key")?;
and in the client:
image_id = register_input_file(client)
...
print("[+] assigning data to task")
client.assign_data_to_task(task_id, [DataMap("image_key", image_id)], [])
...
def register_input_file(client):
url = "file:///home/ubuntu/incubator-teaclave/1.jpg.enc"
cmac = "cf8a2eec4ab2fb53cd5c0b2ce3c0a828"
schema = "teaclave-file-128"
key = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3]
iv = []
key_data_id = client.register_input_file(url, schema, key, iv, cmac)
return key_data_id
----------------------------------------------------------
and following are the errors (I have modified teaclave.py to print out the response)
[+] assigning data to task
b'{"result":"err","request_error":"permission denied"}'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] czzmmc commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
czzmmc commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-638172725
@mssun Great! What I want to say is u guys made a lot of useful test samplecode, but they usually are located on different files. If u dont know where to go, u may spend a lot of time on finding them over massive folders. U may consider a better way to get them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
mssun commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-637699653
You can find a component called file agent (https://github.com/apache/incubator-teaclave/tree/master/file_agent), which provides functionalities to interactive (download and upload files) with data storage services. As you can see from the code, the `file://` schema is for local file storage. That is to say, you can register local files with `file://` in the prefix of the URL.
However, note that the `file://` schema is not recommended right now. We need time to improve the security and usability.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] mssun commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
mssun commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-650946538
Yes, thanks for explanation! @yc-huang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] qinkunbao commented on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
qinkunbao commented on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-649198155
Thanks @mssun and @yc-huang . The gbdt example also specifies the ownership of the file, which is missing in the above code. Is it necessary to define the `ownership` if it is a single user task?
https://github.com/apache/incubator-teaclave/blob/70f02555f58ad3f58640f75f4aadab3307230a0b/examples/python/builtin_gbdt_train.py#L70-L85
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org
[GitHub] [incubator-teaclave] yc-huang edited a comment on issue #329: How to register local files?
Posted by GitBox <gi...@apache.org>.
yc-huang edited a comment on issue #329:
URL: https://github.com/apache/incubator-teaclave/issues/329#issuecomment-648703515
I'm trying to use file agent to pass a local file as well and failed with "permission denied" error, do you have any clue that I might missed? Thanks...
Following are the code snippet:
--------------------------------------------------------------
in the builtin function:
`let mut f = _runtime.open_input("image_key")?;`
and in the client:
```
image_id = register_input_file(client)
...
print("[+] assigning data to task")
client.assign_data_to_task(task_id, [DataMap("image_key", image_id)], [])
...
def register_input_file(client):
url = "file:///home/ubuntu/incubator-teaclave/1.jpg.enc"
cmac = "cf8a2eec4ab2fb53cd5c0b2ce3c0a828"
schema = "teaclave-file-128"
key = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3]
iv = []
key_data_id = client.register_input_file(url, schema, key, iv, cmac)
return key_data_id
```
----------------------------------------------------------
and following are the errors (I have modified teaclave.py to print out the response)
[+] assigning data to task
b'{"result":"err","request_error":"permission denied"}'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@teaclave.apache.org
For additional commands, e-mail: notifications-help@teaclave.apache.org