You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/24 13:15:53 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue, #3936: Error using `IN` list on dictionary encoded data
alamb opened a new issue, #3936:
URL: https://github.com/apache/arrow-datafusion/issues/3936
**Describe the bug**
I am writing a query to select some values from a dictionary encoded string column `trace_id` and I get an error
```
select * from spans where trace_id IN ('187dcbbc68a83a0d', '335424324532');
```
Results in this error
```
This feature is not implemented: InList does not support datatype Dictionary(Int32, Utf8).
```
**To Reproduce**
Run this test:
```diff
diff --git a/datafusion/core/tests/sql/predicates.rs b/datafusion/core/tests/sql/predicates.rs
index 07e016a27..5109f0167 100644
--- a/datafusion/core/tests/sql/predicates.rs
+++ b/datafusion/core/tests/sql/predicates.rs
@@ -427,6 +427,32 @@ async fn csv_in_set_test() -> Result<()> {
Ok(())
}
+#[tokio::test]
+async fn in_set_string_dictionaries() -> Result<()> {
+ let input = vec![Some("foo"), Some("bar"), None, Some("fazzz")]
+ .into_iter()
+ .collect::<DictionaryArray<Int32Type>>();
+
+ let batch = RecordBatch::try_from_iter(vec![("c1", Arc::new(input) as _)]).unwrap();
+
+ let ctx = SessionContext::new();
+ ctx.register_batch("test", batch)?;
+
+ let sql = "SELECT * FROM test WHERE c1 IN ('foo', 'Bar', 'fazz')";
+ let actual = execute_to_batches(&ctx, sql).await;
+ let expected = vec![
+ "+-------+",
+ "| c1 |",
+ "+-------+",
+ "| foo |",
+ "| fazzz |",
+ "+-------+",
+ ];
+
+ assert_batches_eq!(expected, &actual);
+ Ok(())
+}
+
```
Results in
```
thread 'sql::predicates::in_set_string_dictionaries' panicked at 'called `Result::unwrap()` on an `Err` value: "ArrowError(ExternalError(NotImplemented(\"InList does not support datatype Dictionary(Int32, Utf8).\"))) at ...
```
**Expected behavior**
Test should pass
**Additional context**
Found this in IOx as part of https://github.com/influxdata/influxdb_iox/issues/5959
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] tustvold closed issue #3936: Error using `IN` list on dictionary encoded data: `InList does not support datatype Dictionary(Int32, Utf8).`
Posted by GitBox <gi...@apache.org>.
tustvold closed issue #3936: Error using `IN` list on dictionary encoded data: `InList does not support datatype Dictionary(Int32, Utf8).`
URL: https://github.com/apache/arrow-datafusion/issues/3936
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #3936: Error using `IN` list on dictionary encoded data: `InList does not support datatype Dictionary(Int32, Utf8).`
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #3936:
URL: https://github.com/apache/arrow-datafusion/issues/3936#issuecomment-1289603064
@NGA-TRAN says she plans to work on this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on issue #3936: Error using `IN` list on dictionary encoded data: `InList does not support datatype Dictionary(Int32, Utf8).`
Posted by GitBox <gi...@apache.org>.
jackwener commented on issue #3936:
URL: https://github.com/apache/arrow-datafusion/issues/3936#issuecomment-1289065648
related issue: #3766
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #3936: Error using `IN` list on dictionary encoded data
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #3936:
URL: https://github.com/apache/arrow-datafusion/issues/3936#issuecomment-1289018830
I plan to work on this later in the week if no one beats me to it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] NGA-TRAN commented on issue #3936: Error using `IN` list on dictionary encoded data: `InList does not support datatype Dictionary(Int32, Utf8).`
Posted by GitBox <gi...@apache.org>.
NGA-TRAN commented on issue #3936:
URL: https://github.com/apache/arrow-datafusion/issues/3936#issuecomment-1292082278
I am actively working on this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org