You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "Kev1n8 (via GitHub)" <gi...@apache.org> on 2024/08/23 04:07:26 UTC

[PR] Add a api to append view by u128 in for view_builder [arrow-rs]

Kev1n8 opened a new pull request, #6291:
URL: https://github.com/apache/arrow-rs/pull/6291

   # Which issue does this PR close?
   None
   <!--
   We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
   -->
   
   # Rationale for this change
    The idea is to make the builder a way to append a view by a given u128 (unchecked). Check out this [comment](https://github.com/apache/datafusion/pull/12044#discussion_r1727098779).
   <!--
   Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
   Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
   -->
   
   # What changes are included in this PR?
   A function is added to the view builder.
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   # Are there any user-facing changes?
   no
   
   <!--
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   <!---
   If there are any breaking changes to public APIs, please add the `breaking change` label.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add an API to append view by u128 in for view_builder [arrow-rs]

Posted by "Kev1n8 (via GitHub)" <gi...@apache.org>.
Kev1n8 commented on code in PR #6291:
URL: https://github.com/apache/arrow-rs/pull/6291#discussion_r1728654052


##########
arrow-array/src/builder/generic_bytes_view_builder.rs:
##########
@@ -205,6 +205,19 @@ impl<T: ByteViewType + ?Sized> GenericByteViewBuilder<T> {
         self.null_buffer_builder.append_non_null();
     }
 
+    /// Append the given `u128` as a view
+    ///
+    /// # Safety
+    /// (1) The block of the given view must have been added
+    /// (2) The range `offset...offset+length` of the given view
+    /// must be within the bounds of the block
+    /// (3) The data in the block must be valid type `T`
+    /// (4) The view must be not null
+    pub unsafe fn append_view_u128_unchecked(&mut self, view: u128) {

Review Comment:
   That's a good point. However, I believe that StringViewArray does not currently provide a `From<Vec<u128>>` trait directly. The related PR is intended to manually modify views to obtain the substr_view, which can then be added to the builder. Additionally, since the core implementation of StringView is based on the builder pattern, it seemed natural to me to use it. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add an API to append view by u128 in for view_builder [arrow-rs]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on code in PR #6291:
URL: https://github.com/apache/arrow-rs/pull/6291#discussion_r1728662441


##########
arrow-array/src/builder/generic_bytes_view_builder.rs:
##########
@@ -205,6 +205,19 @@ impl<T: ByteViewType + ?Sized> GenericByteViewBuilder<T> {
         self.null_buffer_builder.append_non_null();
     }
 
+    /// Append the given `u128` as a view
+    ///
+    /// # Safety
+    /// (1) The block of the given view must have been added
+    /// (2) The range `offset...offset+length` of the given view
+    /// must be within the bounds of the block
+    /// (3) The data in the block must be valid type `T`
+    /// (4) The view must be not null
+    pub unsafe fn append_view_u128_unchecked(&mut self, view: u128) {

Review Comment:
   https://docs.rs/arrow-array/latest/arrow_array/array/struct.GenericByteViewArray.html#method.new_unchecked and https://docs.rs/arrow-array/latest/arrow_array/array/struct.GenericByteViewArray.html#method.into_parts are the relevant methods FWIW



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add an API to append view by u128 in for view_builder [arrow-rs]

Posted by "Kev1n8 (via GitHub)" <gi...@apache.org>.
Kev1n8 commented on code in PR #6291:
URL: https://github.com/apache/arrow-rs/pull/6291#discussion_r1728670325


##########
arrow-array/src/builder/generic_bytes_view_builder.rs:
##########
@@ -205,6 +205,19 @@ impl<T: ByteViewType + ?Sized> GenericByteViewBuilder<T> {
         self.null_buffer_builder.append_non_null();
     }
 
+    /// Append the given `u128` as a view
+    ///
+    /// # Safety
+    /// (1) The block of the given view must have been added
+    /// (2) The range `offset...offset+length` of the given view
+    /// must be within the bounds of the block
+    /// (3) The data in the block must be valid type `T`
+    /// (4) The view must be not null
+    pub unsafe fn append_view_u128_unchecked(&mut self, view: u128) {

Review Comment:
   I see. That's useful. Thanks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add an API to append view by u128 in for view_builder [arrow-rs]

Posted by "Kev1n8 (via GitHub)" <gi...@apache.org>.
Kev1n8 commented on code in PR #6291:
URL: https://github.com/apache/arrow-rs/pull/6291#discussion_r1728654052


##########
arrow-array/src/builder/generic_bytes_view_builder.rs:
##########
@@ -205,6 +205,19 @@ impl<T: ByteViewType + ?Sized> GenericByteViewBuilder<T> {
         self.null_buffer_builder.append_non_null();
     }
 
+    /// Append the given `u128` as a view
+    ///
+    /// # Safety
+    /// (1) The block of the given view must have been added
+    /// (2) The range `offset...offset+length` of the given view
+    /// must be within the bounds of the block
+    /// (3) The data in the block must be valid type `T`
+    /// (4) The view must be not null
+    pub unsafe fn append_view_u128_unchecked(&mut self, view: u128) {

Review Comment:
   That's a good point. However, I believe that StringViewArray does not currently provide a From<Vec<u128>> trait directly. The related PR is intended to manually modify views to obtain the substr_view, which can then be added to the builder. Additionally, since the core implementation of StringView is based on the builder pattern, it seemed natural to me to use it. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add an API to append view by u128 in for view_builder [arrow-rs]

Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on code in PR #6291:
URL: https://github.com/apache/arrow-rs/pull/6291#discussion_r1728629989


##########
arrow-array/src/builder/generic_bytes_view_builder.rs:
##########
@@ -205,6 +205,19 @@ impl<T: ByteViewType + ?Sized> GenericByteViewBuilder<T> {
         self.null_buffer_builder.append_non_null();
     }
 
+    /// Append the given `u128` as a view
+    ///
+    /// # Safety
+    /// (1) The block of the given view must have been added
+    /// (2) The range `offset...offset+length` of the given view
+    /// must be within the bounds of the block
+    /// (3) The data in the block must be valid type `T`
+    /// (4) The view must be not null
+    pub unsafe fn append_view_u128_unchecked(&mut self, view: u128) {

Review Comment:
   If appending raw `u128` what is the benefit of using the builder at all, as opposed to just using a `Vec<u128>` directly? Is the intended use-case to intermix this method with some of the others?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Add an API to append view by u128 in for view_builder [arrow-rs]

Posted by "Kev1n8 (via GitHub)" <gi...@apache.org>.
Kev1n8 closed pull request #6291: Add an API to append view by u128 in for view_builder
URL: https://github.com/apache/arrow-rs/pull/6291


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org