[GitHub] [arrow-datafusion] alamb commented on a change in pull request #628: Added composite identifiers to get field of struct.

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #628:
URL: https://github.com/apache/arrow-datafusion/pull/628#discussion_r660894440



##########
File path: datafusion/src/physical_plan/expressions/get_field.rs
##########
@@ -0,0 +1,100 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! get field of a struct array
+
+use std::{any::Any, sync::Arc};
+
+use arrow::{
+    array::StructArray,
+    datatypes::{DataType, Schema},
+    record_batch::RecordBatch,
+};
+
+use crate::{
+    error::DataFusionError,
+    error::Result,
+    physical_plan::{ColumnarValue, PhysicalExpr},
+    utils::get_field as get_data_type_field,
+};
+
+/// expression to get a field of a struct array.
+#[derive(Debug)]
+pub struct GetFieldExpr {
+    arg: Arc<dyn PhysicalExpr>,
+    name: String,
+}
+
+impl GetFieldExpr {
+    /// Create new get field expression
+    pub fn new(arg: Arc<dyn PhysicalExpr>, name: String) -> Self {
+        Self { arg, name }
+    }
+
+    /// Get the input expression
+    pub fn arg(&self) -> &Arc<dyn PhysicalExpr> {
+        &self.arg
+    }
+}
+
+impl std::fmt::Display for GetFieldExpr {
+    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
+        write!(f, "({}).{}", self.arg, self.name)
+    }
+}
+
+impl PhysicalExpr for GetFieldExpr {
+    fn as_any(&self) -> &dyn Any {
+        self
+    }
+
+    fn data_type(&self, input_schema: &Schema) -> Result<DataType> {
+        let data_type = self.arg.data_type(input_schema)?;
+        get_data_type_field(&data_type, &self.name).map(|f| f.data_type().clone())
+    }
+
+    fn nullable(&self, input_schema: &Schema) -> Result<bool> {
+        let data_type = self.arg.data_type(input_schema)?;
+        get_data_type_field(&data_type, &self.name).map(|f| f.is_nullable())
+    }
+
+    fn evaluate(&self, batch: &RecordBatch) -> Result<ColumnarValue> {
+        let arg = self.arg.evaluate(batch)?;
+        match arg {
+            ColumnarValue::Array(array) => Ok(ColumnarValue::Array(
+                array
+                    .as_any()
+                    .downcast_ref::<StructArray>()
+                    .unwrap()
+                    .column_by_name(&self.name)

Review comment:
       Very cool -- I was wondering if `column_by_name` returning `None` could be a legit error. For example what happens when trying to select a non existent field? it seems like the calls to `get_field` will have failed earlier with a real error so it is probably fine to treat this as infallible.

##########
File path: datafusion/tests/sql.rs
##########
@@ -2860,6 +2854,31 @@ async fn query_is_not_null() -> Result<()> {
     Ok(())
 }
 
+#[tokio::test]
+async fn query_get_field() -> Result<()> {
+    let inner_field = Field::new("inner", DataType::Float64, true);
+    let field = Field::new("c1", DataType::Struct(vec![inner_field.clone()]), true);
+    let schema = Arc::new(Schema::new(vec![field]));
+
+    let array = Arc::new(Float64Array::from(vec![Some(1.1), None])) as ArrayRef;
+
+    let data = RecordBatch::try_new(
+        schema.clone(),
+        vec![Arc::new(StructArray::from(vec![(inner_field, array)]))],
+    )?;
+
+    let table = MemTable::try_new(schema, vec![vec![data]])?;
+
+    let mut ctx = ExecutionContext::new();
+    ctx.register_table("test", Arc::new(table))?;
+    let sql = "SELECT test.c1.inner FROM test";
+    let actual = execute(&mut ctx, sql).await;
+    let expected = vec![vec!["1.1"], vec!["NULL"]];
+

Review comment:
       It would probably be good to add a negative test here too (e.g. `SELECT test.c1.nonexistent FROM test`) to ensure we get an error (and not a `panic!`)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org