You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/11/26 12:14:15 UTC

[GitHub] [arrow-datafusion] tobyhede opened a new issue #1364: Documentation for using scalar functions with the the DataFrame API

tobyhede opened a new issue #1364:
URL: https://github.com/apache/arrow-datafusion/issues/1364


   I've framed this as a documentation feature request, but it is also a request for assistance. I''ve been attempting to get the `starts_with` function working with the Dataframe API and after banging my head for a couple of hours I am a bit stumped as to how to make it actually work. 
   
   **Describe the solution you'd like**
   More documentation and extended examples for the Dataframe API that include usage of some of the other availablefunctions. 
   Current docs demonstrate comparison (lt_eq) and aggrgeation. 
   
   **Describe alternatives you've considered**
   -
   
   **Additional context**
   -
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] alamb closed issue #1364: Documentation for using scalar functions with the the DataFrame API

Posted by GitBox <gi...@apache.org>.
alamb closed issue #1364:
URL: https://github.com/apache/arrow-datafusion/issues/1364


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] tobyhede edited a comment on issue #1364: Documentation for using scalar functions with the the DataFrame API

Posted by GitBox <gi...@apache.org>.
tobyhede edited a comment on issue #1364:
URL: https://github.com/apache/arrow-datafusion/issues/1364#issuecomment-980450411


   Thanks @xudong963. The core docs are very good, but don't seem to include any examples of using the scalar functions  `lower, upper, starts_with, etc` . I've been trying permutations with no luck.
   
   eg My assumption (based on the equality operators) was that something like this should work:
   ```
     .filter(col("some_col").starts_with(lit("alpha")))
   ```
   
   In the above, `starts_with` isn't found in the scope.
   
   I can do this to get it compiling:
   ```
   .filter(starts_with(lit("alpha")))
   ```
   But this (obviously) fails at runtime. Message is:
   ```
   Error: Plan("Coercion from [Utf8] to the signature OneOf([Exact([Utf8, Utf8]), Exact([Utf8, LargeUtf8]), Exact([LargeUtf8, Utf8]), Exact([LargeUtf8, LargeUtf8])]) failed.")
   ```
          
   I know I am missing something a bit obvious :)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] xudong963 commented on issue #1364: Documentation for using scalar functions with the the DataFrame API

Posted by GitBox <gi...@apache.org>.
xudong963 commented on issue #1364:
URL: https://github.com/apache/arrow-datafusion/issues/1364#issuecomment-979948699


   Hi @tobyhede, thanks for your interest in Datafusion. 
   I think you can find all DF API usages in https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/dataframe.rs which has rich notes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] tobyhede commented on issue #1364: Documentation for using scalar functions with the the DataFrame API

Posted by GitBox <gi...@apache.org>.
tobyhede commented on issue #1364:
URL: https://github.com/apache/arrow-datafusion/issues/1364#issuecomment-980450411


   Thanks @xudong963. The core docs are very good, but don't seem to include any examples of using the scalar functions  `lower, upper, starts_with, etc` . I've been trying permutations with no luck.
   
   eg My assumption (based on the equality operators) was that something like this should work:
   ```
     .filter(col("some_col").starts_with(lit("alpha")))
   ```
   
   In the above, `starts_with` isn't found in the scope.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] tobyhede edited a comment on issue #1364: Documentation for using scalar functions with the the DataFrame API

Posted by GitBox <gi...@apache.org>.
tobyhede edited a comment on issue #1364:
URL: https://github.com/apache/arrow-datafusion/issues/1364#issuecomment-980450411


   Thanks @xudong963. The core docs are very good, but don't seem to include any examples of using the scalar functions  `lower, upper, starts_with, etc` . I've been trying permutations with no luck.
   
   eg My assumption (based on the equality operators) was that something like this should work:
   ```
     .filter(col("some_col").starts_with(lit("alpha")))
   ```
   
   In the above, `starts_with` isn't found in the scope.
   
   I can do this:
   ```
   .filter(starts_with(lit("alpha")))
   ```
   But this fails at runtime saying:
   ```
   Error: Plan("Coercion from [Utf8] to the signature OneOf([Exact([Utf8, Utf8]), Exact([Utf8, LargeUtf8]), Exact([LargeUtf8, Utf8]), Exact([LargeUtf8, LargeUtf8])]) failed.")
   ```
          
   I know I am missing something a bit obvious :)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-datafusion] tobyhede commented on issue #1364: Documentation for using scalar functions with the the DataFrame API

Posted by GitBox <gi...@apache.org>.
tobyhede commented on issue #1364:
URL: https://github.com/apache/arrow-datafusion/issues/1364#issuecomment-980817896


   Digging further, it looks like the root cause here is a bug in the functions - many are being constructed without the correct arity. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org