You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/04 16:45:09 UTC
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #4107: support `SET Timezone`
alamb commented on code in PR #4107:
URL: https://github.com/apache/arrow-datafusion/pull/4107#discussion_r1014242315
##########
datafusion/core/src/execution/context.rs:
##########
@@ -1741,6 +1741,12 @@ impl ContextProvider for SessionState {
.as_ref()
.and_then(|provider| provider.get(&provider_type)?.get_type(variable_names))
}
+
+ fn get_config_option(&self, variable: &str) -> Option<ScalarValue> {
Review Comment:
👍
##########
datafusion/sql/src/planner.rs:
##########
@@ -2671,6 +2673,113 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
Ok(lit(ScalarValue::new_list(Some(values), data_type)))
}
}
+
+ fn convert_data_type(&self, sql_type: &SQLDataType) -> Result<DataType> {
+ match sql_type {
+ SQLDataType::Array(inner_sql_type) => {
+ let data_type = self.convert_simple_data_type(inner_sql_type)?;
+
+ Ok(DataType::List(Box::new(Field::new(
+ "field", data_type, true,
+ ))))
+ }
+ other => self.convert_simple_data_type(other),
+ }
+ }
+ fn convert_simple_data_type(&self, sql_type: &SQLDataType) -> Result<DataType> {
+ match sql_type {
+ SQLDataType::Boolean => Ok(DataType::Boolean),
+ SQLDataType::TinyInt(_) => Ok(DataType::Int8),
+ SQLDataType::SmallInt(_) => Ok(DataType::Int16),
+ SQLDataType::Int(_) | SQLDataType::Integer(_) => Ok(DataType::Int32),
+ SQLDataType::BigInt(_) => Ok(DataType::Int64),
+ SQLDataType::UnsignedTinyInt(_) => Ok(DataType::UInt8),
+ SQLDataType::UnsignedSmallInt(_) => Ok(DataType::UInt16),
+ SQLDataType::UnsignedInt(_) | SQLDataType::UnsignedInteger(_) => {
+ Ok(DataType::UInt32)
+ }
+ SQLDataType::UnsignedBigInt(_) => Ok(DataType::UInt64),
+ SQLDataType::Float(_) => Ok(DataType::Float32),
+ SQLDataType::Real => Ok(DataType::Float32),
+ SQLDataType::Double | SQLDataType::DoublePrecision => Ok(DataType::Float64),
+ SQLDataType::Char(_)
+ | SQLDataType::Varchar(_)
+ | SQLDataType::Text
+ | SQLDataType::String => Ok(DataType::Utf8),
+ SQLDataType::Timestamp(tz_info) => {
+ let tz = if matches!(tz_info, TimezoneInfo::Tz)
+ || matches!(tz_info, TimezoneInfo::WithTimeZone)
+ {
+ match self.schema_provider.get_config_option("datafusion.execution.time_zone") {
Review Comment:
It would be nice to use a named constant instead of the string here
##########
datafusion/sql/src/planner.rs:
##########
@@ -2671,6 +2673,113 @@ impl<'a, S: ContextProvider> SqlToRel<'a, S> {
Ok(lit(ScalarValue::new_list(Some(values), data_type)))
}
}
+
+ fn convert_data_type(&self, sql_type: &SQLDataType) -> Result<DataType> {
+ match sql_type {
+ SQLDataType::Array(inner_sql_type) => {
+ let data_type = self.convert_simple_data_type(inner_sql_type)?;
+
+ Ok(DataType::List(Box::new(Field::new(
+ "field", data_type, true,
+ ))))
+ }
+ other => self.convert_simple_data_type(other),
+ }
+ }
+ fn convert_simple_data_type(&self, sql_type: &SQLDataType) -> Result<DataType> {
+ match sql_type {
+ SQLDataType::Boolean => Ok(DataType::Boolean),
+ SQLDataType::TinyInt(_) => Ok(DataType::Int8),
+ SQLDataType::SmallInt(_) => Ok(DataType::Int16),
+ SQLDataType::Int(_) | SQLDataType::Integer(_) => Ok(DataType::Int32),
+ SQLDataType::BigInt(_) => Ok(DataType::Int64),
+ SQLDataType::UnsignedTinyInt(_) => Ok(DataType::UInt8),
+ SQLDataType::UnsignedSmallInt(_) => Ok(DataType::UInt16),
+ SQLDataType::UnsignedInt(_) | SQLDataType::UnsignedInteger(_) => {
+ Ok(DataType::UInt32)
+ }
+ SQLDataType::UnsignedBigInt(_) => Ok(DataType::UInt64),
+ SQLDataType::Float(_) => Ok(DataType::Float32),
+ SQLDataType::Real => Ok(DataType::Float32),
+ SQLDataType::Double | SQLDataType::DoublePrecision => Ok(DataType::Float64),
+ SQLDataType::Char(_)
+ | SQLDataType::Varchar(_)
+ | SQLDataType::Text
+ | SQLDataType::String => Ok(DataType::Utf8),
+ SQLDataType::Timestamp(tz_info) => {
+ let tz = if matches!(tz_info, TimezoneInfo::Tz)
+ || matches!(tz_info, TimezoneInfo::WithTimeZone)
+ {
+ match self.schema_provider.get_config_option("datafusion.execution.time_zone") {
+ Some(ScalarValue::Utf8(s)) => {
+ s
+ }
+ Some(_) => {
+ None
+ }
+ None => None
+ }
+ //Some("+00:00".to_string())
+
+ } else {
+ None
+ };
+ Ok(DataType::Timestamp(TimeUnit::Nanosecond, tz))
Review Comment:
It is somewhat strange to me that the type of a table that is created depends on the current session 's timezone setting -- but I can't think of anything better short of the user explicitly providing the timezone themselves
##########
datafusion/sql/src/planner.rs:
##########
@@ -2809,6 +2918,7 @@ fn extract_possible_join_keys(
}
/// Convert SQL simple data type to relational representation of data type
+/*
pub fn convert_simple_data_type(sql_type: &SQLDataType) -> Result<DataType> {
match sql_type {
Review Comment:
I think we should remove it personally or of we want to deprecate it for a release cycle that would be fine
Bonus points for somehow calling the new implementation (like making a dummy `SqlToRel` somehow)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org