You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@teaclave.apache.org by "TheButlah (via GitHub)" <gi...@apache.org> on 2026/02/09 06:40:08 UTC

[PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

TheButlah opened a new pull request, #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278

   Fixes https://github.com/apache/teaclave-trustzone-sdk/issues/273
   
   **Summarizing that issue**, the [current api](https://github.com/apache/teaclave-trustzone-sdk/blob/e6511571ef29dc7bd2dd971098027cdcfea86ded/optee-utee/src/parameter.rs#L79) to access a `ParamMemref`'s buffer is unsound when having to account for malicious CAs (and probably benign ones too). The main issue is that since this memory is not synchronized, Linux can change its contents at any time, and therefore it is essentially volatile memory. Constructing a reference to volatile memory is always unsound.
   
   Additionally, OP-TEE *only* allows reads when the memref is an `[in]` or `[inout]` param, and writes when its an `[out]` or `[inout]` param. The original API doesn't check the `ParamType` at all, which means that the end user can easily write to a buffer they can only read from, or read from a buffer they can only write to.
   
   **This PR fixes the issue by introducing two concepts**:
   * An "access" api in `access.rs`. This api has traits and zero sized types that allow us to encode the access permissions into the type system via a typestate pattern. This sort of approach is a zero cost abstraction and allows us to check the param_type *only once*, as opposed to every interaction with the memref. It also helps ensure that only the appropriate functions (such as .read() or .write()) are expose for a given access type. The typestate approach is also extremely common in the bare-metal rust ecosystem (see embedded-hal crates), so I figured it isn't overly unfamiliar for this domain of work.
   * A `VolatileBuf`, which is a type that ensures that all access to its memory is done through `core::ptr::read_volatile` or `core::ptr::write_volatile`, and leverages the access api to expose only the appropriate read/write functionality.
   
   *A few things reviewers should be aware of:*
   * I added a dependency on `bytemuck::Pod`. This was done because I wanted to support *arbitrary* buffers of `T` rather than just `u8`. This was done in an attempt to allow faster accessing of data without needing to repeatedly call deserialization code, as if it were only u8 some sort of deserialization (or subsequent bytemuck cast) would be required. The downside is the additional dependency, but bytemuck is a 1.0 crate and well known and commonly used. I didn't want to introduce a first-party Pod trait of our own, because it would have less compatibility with the broader ecosystem and would require us to then implement bytemuck-style derive macros for user-defined types.
   * I *didn't* use the [volatile crate](https://docs.rs/volatile) because so much of the crate's surface area requires nightly compiler features. Additionally, I found the API limiting in some ways, and too complicated to easily use in others.
   
   I am definitely open to workshopping this submission, please let me know your thoughts and I will adjust the PR accordingly. Thanks for your time :+1: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-3870434618

   i'm seeing errors related to error_in_core, but I thought that was resolved by https://github.com/apache/teaclave-trustzone-sdk/pull/262


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-3870666996

   OK let me know what you think :+1: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-3869986568

   Thanks for your commit!
   The approach looks good to me, and I've left two tiny comments.
   Additionally, could you help with the following:
   - fix the license ci;
   - modify one example to demonstrate the API change
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2802019037


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   Suggestions on `NotBiggerThanCapacityErr`:
   - the name is a bit confusing, how about `ShortBufferErr` or any other you prefer
   - It should be converted into `ErrorKind::ShortBuffer`, as TA notifies CA that it needs more capacity
   
   Suggestions on `request_more_capacity()`, how about changing to:
   ```
   pub fn ensure_capacity(&mut self, required_len: usize) -> Result<(), ShortBufferErr> {
       if required_len > self.capacity {
           let memref = unsafe { self.raw.as_mut() };
           memref.size = required_len;
           return Err(ShortBufferErr);
       }
       Ok(())
   }
   ```
   In the example:
   ```
   let len = bytes.len();
   // no need to manually check the len
   p.ensure_capacity(len)?
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and in the future we will instead actually invoke a PTA that talks to journald.
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   Side note: some of these error types are more than just richer error messages. They actually have very important data. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the access fails, the caller doesn't drop data.



##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and in the future we will instead actually invoke a PTA that talks to journald.
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   Side note: some of these error types are more than just richer error messages. They actually have very important data. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the access fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and in the future we will instead actually invoke a PTA that talks to journald.
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   Side note: some of these error types are more than just richer error messages. They actually have very important data. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the conversion fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2803476525


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   I think that approach makes sense! I've updated the PR



##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   I think that approach is a big improvement! I've updated the PR



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "ivila (via GitHub)" <gi...@apache.org>.
ivila commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-4323709206

   > I believe the Rust SDK should provide primitives that are as safe as possible. In our context, checking capacity, copying data, and updating the size should be treated as a single atomic operation to maintain consistency. Decoupling them only creates more opportunities for logic bugs (like forgetting to update the size) and leads to redundant error handling.
   > 
   > > ~what if someone _doesn't_ want to update the capacity?~
   > 
   > I cannot think of a valid use case for skipping the update... If the capacity is insufficient, why not updating the capacity to inform the client?
   > 
   > > maybe we _should_ just always ensure capacity and set updated size when doing copy_from?
   > 
   > It also makes sense to me. To keep the API explicit, we may need a more descriptive name than just copy_from to indicate it handles the full lifecycle (check, copy, and size update).
   > 
   > Also requesting feedback from @ivila for this API design (once he is available). Since we are heading into the Chinese New Year holiday, our responses would be slower than usual. Thanks for your patience!
   
   Sorry for the late response — I must have missed the GitHub notification. I don’t have any other comments; only a few parts of the code need to be updated (for example, we should use core::error directly rather than renaming it to CoreError, just check previous comments).😄


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-3903360167

   My concern would be, what if someone *doesn't* want to update the capacity?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2784866405


##########
optee-utee/src/parameter.rs:
##########
@@ -127,9 +246,11 @@ impl Parameter {
         match self.param_type {
             ParamType::MemrefInout | ParamType::MemrefInput | ParamType::MemrefOutput => {
                 Ok(ParamMemref {
-                    raw: &mut (*self.raw).memref,
+                    raw: NonNull::new_unchecked(addr_of_mut!((*self.raw).memref)),
                     param_type: self.param_type,
-                    _marker: marker::PhantomData,
+                    capacity: unsafe { *self.raw }.memref.size,

Review Comment:
   Done. BTW I would consider this to be an anti-pattern. 2024 edition turns on warnings by default if the `unsafe` keyword is not used explicitly. It might be good for teaclave to `#![forbid(unsafe_op_in_unsafe_fn)]` by default.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2785576937


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   I'm a bit confused about the logic here:
   
   `request_more_capacity()` could return error according its inner logic, right?
   Regarding that, it should raise the error to the caller.
   
   Current code:
   ```
                   if p.request_more_capacity(len).is_err() {
                       unreachable!()
                   }
   ```
   means it should never return error?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and having the ability to clearly see in logs the chain of errors and their descriptions (without having to clutter every call site with tons of if lets and printlns), is very valuable.
   
   There are other reasons too. For example, in the system we use at work, our serial is not accessible therefore if we want to see what errors happened, we need to return the error to the CA. So we can't rely on `trace_*!()` macros. If we used only `ErrorKind` we would have *very* little information necessary to debug the system, since so little information can be encoded. Instead, it makes sense to have richer error types that can be returned to the CA. Additionally, trace_println!() is extremely slow and becomes a performance bottleneck very fast.
   
   
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   
   
   Side note: some of these error types are more than just richer error messages. They actually have very important data. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the conversion fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and in the future we will instead actually invoke a PTA that talks to journald. Additionally in production, we don't utilize trace_println!() almost at all, since we think it reveals too much info about the system (and it can lag things due to slow serial).
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   Side note: some of these error types are more than just richer error messages. They actually have very important data. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the conversion fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2784866405


##########
optee-utee/src/parameter.rs:
##########
@@ -127,9 +246,11 @@ impl Parameter {
         match self.param_type {
             ParamType::MemrefInout | ParamType::MemrefInput | ParamType::MemrefOutput => {
                 Ok(ParamMemref {
-                    raw: &mut (*self.raw).memref,
+                    raw: NonNull::new_unchecked(addr_of_mut!((*self.raw).memref)),
                     param_type: self.param_type,
-                    _marker: marker::PhantomData,
+                    capacity: unsafe { *self.raw }.memref.size,

Review Comment:
   Done. BTW 2024 edition turns on warnings by default if the `unsafe` keyword is not used explicitly. It might be good for teaclave to `#![forbid(unsafe_op_in_unsafe_fn)]` by default.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and in the future we will instead actually invoke a PTA that talks to journald.
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "ivila (via GitHub)" <gi...@apache.org>.
ivila commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-5349983260

   Already Done by #317 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2782740885


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");
                 return Err(ErrorKind::BadParameters.into());
             }
 
             // Copy the serialized JSON string into the buffer.
-            buffer[..len].copy_from_slice(bytes);
+            buffer.copy_from(bytes);

Review Comment:
   Suggest to handle the result of `copy_from()`



##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   We should avoid panic in TAs, could you revise the interface for better error handling?
   Same in line81.



##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -54,8 +54,8 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
     trace_println!("[+] TA invoke command");
     match Command::from(cmd_id) {
         Command::DefaultOp => {
-            let mut p = unsafe { params.0.as_memref()? };
-            let buffer = p.buffer();
+            let mut p = unsafe { params.0.as_memref()? }.output()?;
+            let mut buffer = p.buffer().ok_or(TeeError::new(ErrorKind::ShortBuffer))?;

Review Comment:
   Seems we always need to cast the option to error, how about buffer() directly returns Result?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and in the future we will instead actually invoke a PTA that talks to journald.
   
   The way this PR does it is a middle ground: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2785591083


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   To keep the SDK simple, why not use `trace_println!` at the call site and eliminate these extra types? I believe the code would be much clean and simple.
   
   BTW, you can use `ErrorKind::Overflow.into()` to convert directly to the OP-TEE Error.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and having the ability to clearly see in logs the chain of errors and their descriptions (without having to clutter every call site with tons of if lets and printlns), is very valuable.
   
   There are other reasons too. For example, in the system we use at work, our serial is not accessible therefore if we want to see what errors happened, we need to return the error to the CA. So we can't rely on `trace_*!()` macros. If we used only `ErrorKind` we would have *very* little information necessary to debug the system, since so little information can be encoded. Instead, it makes sense to have richer error types that can be returned to the CA. Additionally, trace_println!() is extremely slow and becomes a performance bottleneck very fast.
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   Side note: some of these error types are more than just richer error messages. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the conversion fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-3900522469

   Thanks for the changes! Since this is a core part of the API design, it's worth taking the extra time to consider it carefully. I really appreciate your patience and collaboration on this!
   
   Some new suggestions:
   I’m wondering if we can also eliminate the another `unreachable!()` block in updating size.
   Since checking capacity, copying memory, and updating the size are almost always performed together, how about  merging them into a single atomic method? This makes the example code much cleaner and user-friendly, also prevents developers from forgetting necessary checks or size updates.
   Like this:
   ```
   impl ParamMemref<'_, _> {
       /// Copies data into the buffer and automatically updates the memref size.
       /// Returns `ShortBufferErr` and updates the size hint if the capacity is insufficient.
       pub fn copy_from_and_update_size(&mut self, src: &[u8]) -> Result<(), Error> {
           let len = src.len();
           
           // 1. Check capacity
           self.ensure_capacity(len)?;
   
           // 2. Access the buffer and perform the copy
           let mut buf = self.buffer()?;
           buf.copy_from(src)?;
   
           // 3. Update the raw memref size, the len has been checked
           unsafe { self.raw.as_mut() }.size = len;
   
           Ok(())
       }
   }
   ```
   In the example:
   ```
   let bytes = serialized.as_bytes();
   
   // Atomically check capacity, copy data, and update the size
   p.copy_from_and_update_size(bytes)?;
   ```
   Let me know what you think, thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2801940668


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   Sounds good, thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789470658


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   the request_more_capacity() function will error if the requested capacity `len` is less than or equal to the current capacity. We already checked this on line 71, which is why the function will never return an error. To indicate the intent more clearly, I used `unreachable!()` although normally when I write code, I just `.expect("infallible")`. We could bubble the error up with `?` but imo obscures the fact that this error will never happen in the first place.
   
   let me know how to proceed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and having the ability to clearly see in logs the chain of errors and their descriptions (without having to clutter every call site with tons of if lets and printlns), is very valuable.
   
   There are other reasons too. For example, in the system we use at work, our serial is not accessible therefore if we want to see what errors happened, we need to return the error to the CA. So we can't rely on `trace_*!()` macros. If we used only `ErrorKind` we would have *very* little information necessary to debug the system, since so little information can be encoded. Instead, it makes sense to have richer error types that can be returned to the CA, and we get to defer the printing of those errors instead of serializing them via `format!()` or `trace_println!()` Additionally, trace_println!() is extremely slow and becomes a performance bottleneck very fast.
   
   
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   
   
   Side note: some of these error types are more than just richer error messages. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the conversion fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#issuecomment-3906077920

   I believe the Rust SDK should provide primitives that are as safe as possible. In our context, checking capacity, copying data, and updating the size should be treated as a single atomic operation to maintain consistency. Decoupling them only creates more opportunities for logic bugs (like forgetting to update the size) and leads to redundant error handling.
   
   > ~what if someone _doesn't_ want to update the capacity?~
   
   I cannot think of a valid use case for skipping the update... If the capacity is insufficient, why not updating the capacity to inform the client?
   
   > maybe we _should_ just always ensure capacity and set updated size when doing copy_from?
   
   It also makes sense to me. To keep the API explicit, we may need a more descriptive name than just copy_from to indicate it handles the full lifecycle (check, copy, and size update).
   
   ----
   Also requesting feedback from @ivila for this API design (once he is available). Since we are heading into the Chinese New Year holiday, our responses would be slower than usual. Thanks for your patience!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2784695440


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   ok, let me know if `unreachable!()` is more clear. The point is that this code should never panic.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "ivila (via GitHub)" <gi...@apache.org>.
ivila closed pull request #278: fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2782810649


##########
optee-utee/src/parameter.rs:
##########
@@ -127,9 +246,11 @@ impl Parameter {
         match self.param_type {
             ParamType::MemrefInout | ParamType::MemrefInput | ParamType::MemrefOutput => {
                 Ok(ParamMemref {
-                    raw: &mut (*self.raw).memref,
+                    raw: NonNull::new_unchecked(addr_of_mut!((*self.raw).memref)),
                     param_type: self.param_type,
-                    _marker: marker::PhantomData,
+                    capacity: unsafe { *self.raw }.memref.size,

Review Comment:
   An internal `unsafe` block isn't strictly necessary because the entire function is already marked `unsafe`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "DemesneGH (via GitHub)" <gi...@apache.org>.
DemesneGH commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2781151787


##########
optee-utee/src/parameter.rs:
##########
@@ -69,27 +77,137 @@ impl<'parameter> ParamValue<'parameter> {
     }
 }
 
-pub struct ParamMemref<'parameter> {
-    raw: *mut raw::Memref,
+/// A [`Parameter`] that is a reference to CA-controlled shared memory.
+///
+/// Note: The memory it points to is volatile and can contain maliciously crafted
+/// data, so care should be taken when accessing it. For a safe api, first check
+/// the type of the memref via [`Self::in_`], [`Self::out`], or [`Self::inout`].
+///
+/// These will return type type-safe versions that unsure that read or write access
+/// is only allowed based on the underlying [`ParamType`].
+///
+/// Then, call [`Self::buffer`] which returns a [`VolatileBuf`] that can be accessed
+/// safely.
+pub struct ParamMemref<'parameter, A = NoAccess> {
+    raw: NonNull<raw::Memref>,
     param_type: ParamType,
-    _marker: marker::PhantomData<&'parameter mut [u8]>,
+    capacity: usize,
+    _phantom_param: marker::PhantomData<&'parameter mut [u8]>,
+    _phantom_access: marker::PhantomData<A>,
 }
 
-impl<'parameter> ParamMemref<'parameter> {
-    pub fn buffer(&mut self) -> &mut [u8] {
-        unsafe { slice::from_raw_parts_mut((*self.raw).buffer as *mut u8, (*self.raw).size) }
+impl<'parameter, A> ParamMemref<'parameter, A> {
+    // Helper function to cast access.
+    fn access<NewAccess>(
+        self,
+        expected_param_type: ParamType,
+    ) -> Result<ParamMemref<'parameter, NewAccess>, InvalidAccessErr<Self, NewAccess>> {
+        if self.param_type != expected_param_type {
+            return Err(InvalidAccessErr {
+                param_type: self.param_type,
+                value: self,
+                _phantom: marker::PhantomData,
+            });
+        }
+
+        Ok(ParamMemref::<'parameter, NewAccess> {
+            raw: self.raw,
+            param_type: self.param_type,
+            capacity: self.capacity,
+            _phantom_param: marker::PhantomData,
+            _phantom_access: marker::PhantomData,
+        })
+    }
+
+    /// Checks that this param is a [`ParamType::MemrefInput`] and casts access to [`Read`].
+    pub fn in_(self) -> Result<ParamMemref<'parameter, Read>, InvalidAccessErr<Self, Read>> {

Review Comment:
   I see `in` is a reserved keyword in Rust, but `in_()` is a little awkward for user interface. How about naming as `input()`, `output()`?



##########
optee-utee/src/parameter.rs:
##########
@@ -171,6 +291,20 @@ pub enum ParamType {
     MemrefInout = 7,
 }
 
+impl Display for ParamType {

Review Comment:
   Recommend using strum_macros::Display



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789470658


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   the request_more_capacity() function will error if the requested capacity `len` is less than or equal to the current capacity. We already checked this on line 71, which is why the function will never return an error. To indicate the intent more clearly, I used `unreachable!()` although normally when I write code, I just `.expect("infallible")`. We could bubble the error up with `?` but that obscures the fact that this error will never happen in the first place



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789470658


##########
examples/serde-rs/ta/src/main.rs:
##########
@@ -68,16 +68,17 @@ fn invoke_command(cmd_id: u32, params: &mut Parameters) -> Result<()> {
 
             // Ensure the buffer is large enough to hold the serialized data.
             let len = bytes.len();
-            if len > buffer.len() {
+            if len > buffer.capacity() {
                 trace_println!("Buffer too small, cannot copy all bytes");
+                p.request_more_capacity(len).expect("infallible");

Review Comment:
   the request_more_capacity() function will error if the requested capacity `len` is less than or equal to the current capacity. We already checked this on line 71, which is why the function will never return an error.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and having the ability to clearly see in logs the chain of errors and their descriptions (without having to clutter every call site with tons of if lets and printlns), is very valuable.
   
   There are other reasons too. For example, in the system we use at work, our serial is not accessible therefore if we want to see what errors happened, we need to return the error to the CA. So we can't rely on `trace_*!()` macros. If we used only `ErrorKind` we would have *very* little information necessary to debug the system, since so little information can be encoded. Instead, it makes sense to have richer error types that can be returned to the CA. Additionally, trace_println!() is extremely slow and becomes a performance bottleneck very fast.
   
   
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   
   
   Side note: some of these error types are more than just richer error messages. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the conversion fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org


Re: [PR] fix!: introduce access api and VolatileBuf to safely interact with volatile ParamMemref buffers [teaclave-trustzone-sdk]

Posted by "TheButlah (via GitHub)" <gi...@apache.org>.
TheButlah commented on code in PR #278:
URL: https://github.com/apache/teaclave-trustzone-sdk/pull/278#discussion_r2789491237


##########
optee-utee/src/parameter.rs:
##########
@@ -185,3 +306,101 @@ impl From<u32> for ParamType {
         }
     }
 }
+
+#[derive(Debug)]
+pub struct BiggerThanCapacityErr {
+    requested_size: usize,
+    capacity: usize,
+}
+
+impl Display for BiggerThanCapacityErr {
+    fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
+        write!(
+            f,
+            "requested size {} but this is bigger than the capacity {}",
+            self.requested_size, self.capacity,
+        )
+    }
+}
+
+impl CoreError for BiggerThanCapacityErr {}
+
+impl From<BiggerThanCapacityErr> for crate::Error {
+    fn from(_value: BiggerThanCapacityErr) -> Self {
+        crate::Error::new(ErrorKind::Overflow)
+    }
+}

Review Comment:
   > Seems these new error types are only used for printing error messages before being converted into a standard optee Error?
   
   I provide the `From` conversion to make the example code more terse, but I believe there is value in more clearly communicating what the nature of the error is (along with a richer error message). In production code for example, we actually wrap all of our errors with `anyhow` and then right before we return to optee, we log that more comprehensive error out, and having the ability to clearly see in logs the chain of errors and their descriptions (without having to clutter every call site with tons of if lets and printlns), is very valuable.
   
   There are other reasons too. For example, in the system we use at work, our serial is not accessible therefore if we want to see what errors happened, we need to return the error to the CA. So we can't rely on `trace_*!()` macros. If we used only `ErrorKind` we would have *very* little information necessary to debug the system, since so little information can be encoded. Instead, it makes sense to have richer error types that can be returned to the CA.
   
   Additionally, trace_println!() is extremely slow and becomes a performance bottleneck very fast.
   
   
   
   The way this PR does it is the best of both worlds: it provides the conversion for those that just want a `?` and tersely writing code, but it retains the structured error for those that need/want it.
   
   
   
   Side note: some of these error types are more than just richer error messages. They actually have very important data. The error returned when converting access for example, contains the original type (since the function takes `self`) so that if the conversion fails, the caller doesn't drop data and can try again with a different access type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@teaclave.apache.org
For additional commands, e-mail: dev-help@teaclave.apache.org