You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "ArnavBalyan (via GitHub)" <gi...@apache.org> on 2025/01/30 02:41:07 UTC
[PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
ArnavBalyan opened a new pull request, #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
(Fixes: \#ISSUE-ID)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2695091532
> Please refer to this one [facebookincubator/velox#12041](https://github.com/facebookincubator/velox/pull/12041) in the PR description, but it is not merged in oap-project/velox, please wait for #8867
Got it, updated the ref, will wait for the Velox to update
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2703623798
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2708863881
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2727911705
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2733373239
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1976693065
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -250,8 +250,8 @@ bool SubstraitToVeloxPlanValidator::validateCast(
const auto& toType = SubstraitParser::parseType(castExpr.type());
core::TypedExprPtr input = exprConverter_->toVeloxExpr(castExpr.input(), inputType);
- // Only support cast from date to timestamp
- if (toType->kind() == TypeKind::TIMESTAMP && !input->type()->isDate()) {
+ // Only support cast of certain types to timestamp (date and double)
+ if (toType->kind() == TypeKind::TIMESTAMP && !input->type()->isDate() && input->type()->kind() != TypeKind::DOUBLE) {
Review Comment:
This one https://github.com/facebookincubator/velox/pull/12041 has been merged, please update to also support float, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2708973780
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2727326127
cc @jinchengchenghh this is ready for review thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2623393319
https://github.com/apache/incubator-gluten/issues/8639
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2709899333
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1997827397
##########
backends-velox/src/test/scala/org/apache/spark/sql/catalyst/expressions/VeloxCastSuite.scala:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.catalyst.expressions
+
+import org.apache.gluten.execution.VeloxWholeStageTransformerSuite
+
+import org.apache.spark.sql.catalyst.util.DateTimeTestUtils.UTC_OPT
+import org.apache.spark.sql.types._
+
+import java.sql.Timestamp
+import java.util.TimeZone
+
+class VeloxCastSuite extends VeloxWholeStageTransformerSuite with ExpressionEvalHelper {
+ def cast(v: Any, targetType: DataType, timeZoneId: Option[String] = None): Cast = {
+ v match {
+ case lit: Expression =>
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ case _ =>
+ val lit = Literal(v)
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ }
+ }
+
+ test("cast from double to timestamp format") {
Review Comment:
Is this test derived from Spark test "cast from timestamp II"? If yes, better to add a comment to clarify.
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -242,6 +242,9 @@ bool SubstraitToVeloxPlanValidator::isAllowedCast(const TypePtr& fromType, const
// 3. Timestamp to most categories except few supported types is not allowed.
// 4. Certain complex types are not allowed.
+ auto fromKind = fromType->kind();
+ auto toKind = toType->kind();
Review Comment:
Please remove these two if unused.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2629301343
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2689928559
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1998461822
##########
gluten-ut/spark35/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala:
##########
@@ -100,6 +100,7 @@ class VeloxTestSettings extends BackendTestSettings {
.exclude(
"Process Infinity, -Infinity, NaN in case insensitive manner" // +inf not supported in folly.
)
+ .exclude("cast from timestamp II") // Rewrite test for Gluten
Review Comment:
Hi @jackylee-ch have added the reason and fix here https://github.com/apache/incubator-gluten/pull/9011. We are running the wrong tests, once it's merged I'll enable/re-write for Gluten. With ANSI on, Spark had additional tests leading to overflow but Gluten does not support ANSI mode thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1997864799
##########
backends-velox/src/test/scala/org/apache/spark/sql/catalyst/expressions/VeloxCastSuite.scala:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.catalyst.expressions
+
+import org.apache.gluten.execution.VeloxWholeStageTransformerSuite
+
+import org.apache.spark.sql.catalyst.util.DateTimeTestUtils.UTC_OPT
+import org.apache.spark.sql.types._
+
+import java.sql.Timestamp
+import java.util.TimeZone
+
+class VeloxCastSuite extends VeloxWholeStageTransformerSuite with ExpressionEvalHelper {
+ def cast(v: Any, targetType: DataType, timeZoneId: Option[String] = None): Cast = {
+ v match {
+ case lit: Expression =>
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ case _ =>
+ val lit = Literal(v)
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ }
+ }
+
+ test("cast from double to timestamp format") {
Review Comment:
This is slightly different, I have raised this to fix the "cast from timestamp II" issue thanks, https://github.com/apache/incubator-gluten/pull/9011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1997864438
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -242,6 +242,9 @@ bool SubstraitToVeloxPlanValidator::isAllowedCast(const TypePtr& fromType, const
// 3. Timestamp to most categories except few supported types is not allowed.
// 4. Certain complex types are not allowed.
+ auto fromKind = fromType->kind();
+ auto toKind = toType->kind();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2709885964
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1997826563
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -242,6 +242,9 @@ bool SubstraitToVeloxPlanValidator::isAllowedCast(const TypePtr& fromType, const
// 3. Timestamp to most categories except few supported types is not allowed.
// 4. Certain complex types are not allowed.
+ auto fromKind = fromType->kind();
+ auto toKind = toType->kind();
Review Comment:
Please remove these two lines if unused.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "jackylee-ch (via GitHub)" <gi...@apache.org>.
jackylee-ch commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1998113960
##########
gluten-ut/spark35/src/test/scala/org/apache/gluten/utils/velox/VeloxTestSettings.scala:
##########
@@ -100,6 +100,7 @@ class VeloxTestSettings extends BackendTestSettings {
.exclude(
"Process Infinity, -Infinity, NaN in case insensitive manner" // +inf not supported in folly.
)
+ .exclude("cast from timestamp II") // Rewrite test for Gluten
Review Comment:
We need a comment to explain why we exclude this test. `Rewrite test for Gluten` semms not a suitable reason. is it work defferently with velox?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE merged PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2627487544
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2710719763
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1997888658
##########
backends-velox/src/test/scala/org/apache/spark/sql/catalyst/expressions/VeloxCastSuite.scala:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.catalyst.expressions
+
+import org.apache.gluten.execution.VeloxWholeStageTransformerSuite
+
+import org.apache.spark.sql.catalyst.util.DateTimeTestUtils.UTC_OPT
+import org.apache.spark.sql.types._
+
+import java.sql.Timestamp
+import java.util.TimeZone
+
+class VeloxCastSuite extends VeloxWholeStageTransformerSuite with ExpressionEvalHelper {
+ def cast(v: Any, targetType: DataType, timeZoneId: Option[String] = None): Cast = {
+ v match {
+ case lit: Expression =>
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ case _ =>
+ val lit = Literal(v)
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ }
+ }
+
+ test("cast from double to timestamp format") {
+ val originalDefaultTz = TimeZone.getDefault
+ try {
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"))
Review Comment:
@ArnavBalyan, do we need this setting? I note Spark doesn't consider time zone in doing this cast.
See https://github.com/apache/spark/blob/branch-3.5/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L675
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1998042273
##########
backends-velox/src/test/scala/org/apache/spark/sql/catalyst/expressions/VeloxCastSuite.scala:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.catalyst.expressions
+
+import org.apache.gluten.execution.VeloxWholeStageTransformerSuite
+
+import org.apache.spark.sql.catalyst.util.DateTimeTestUtils.UTC_OPT
+import org.apache.spark.sql.types._
+
+import java.sql.Timestamp
+import java.util.TimeZone
+
+class VeloxCastSuite extends VeloxWholeStageTransformerSuite with ExpressionEvalHelper {
+ def cast(v: Any, targetType: DataType, timeZoneId: Option[String] = None): Cast = {
+ v match {
+ case lit: Expression =>
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ case _ =>
+ val lit = Literal(v)
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ }
+ }
+
+ test("cast from double to timestamp format") {
+ val originalDefaultTz = TimeZone.getDefault
+ try {
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"))
Review Comment:
We may need this, since Cast.scala returns the internal representation of timestamp in Long. However the timezone will be used for the timestamp generation from the epoch seconds, later in the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2727162146
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2727154696
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2631242881
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2690119808
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2693488953
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2695040971
Please update the PR description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "PHILO-HE (via GitHub)" <gi...@apache.org>.
PHILO-HE commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1978845245
##########
gluten-ut/spark33/src/test/scala/org/apache/spark/sql/catalyst/expressions/GlutenCastSuite.scala:
##########
@@ -152,4 +153,33 @@ class GlutenCastSuite extends CastSuite with GlutenTestsTrait {
checkEvaluation(cast(Literal.create(null, IntegerType), ShortType), null)
}
+
+ test("cast from double to timestamp format") {
Review Comment:
If this test is not a variant of vanilla spark test, we prefer moving it to some common places, not bound to one Spark version. E.g., `ScalarFunctoinsValidateSuite.scala` or one new dedicate test class under backends-velox.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2726912910
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2727160201
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1976988254
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -250,8 +250,8 @@ bool SubstraitToVeloxPlanValidator::validateCast(
const auto& toType = SubstraitParser::parseType(castExpr.type());
core::TypedExprPtr input = exprConverter_->toVeloxExpr(castExpr.input(), inputType);
- // Only support cast from date to timestamp
- if (toType->kind() == TypeKind::TIMESTAMP && !input->type()->isDate()) {
+ // Only support cast of certain types to timestamp (date and double)
+ if (toType->kind() == TypeKind::TIMESTAMP && !input->type()->isDate() && input->type()->kind() != TypeKind::DOUBLE) {
Review Comment:
updated thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2631240587
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2695060804
> Please update the PR description
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2712562902
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2627600273
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2694992445
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2711010981
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2695017078
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2695076892
Please refer to this one https://github.com/facebookincubator/velox/pull/12041 in the PR description, but it is not merged in oap-project/velox, please wait for https://github.com/apache/incubator-gluten/pull/8867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2726752582
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2703608478
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double/float to timestamp [incubator-gluten]
Posted by "ArnavBalyan (via GitHub)" <gi...@apache.org>.
ArnavBalyan commented on code in PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#discussion_r1998042273
##########
backends-velox/src/test/scala/org/apache/spark/sql/catalyst/expressions/VeloxCastSuite.scala:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.catalyst.expressions
+
+import org.apache.gluten.execution.VeloxWholeStageTransformerSuite
+
+import org.apache.spark.sql.catalyst.util.DateTimeTestUtils.UTC_OPT
+import org.apache.spark.sql.types._
+
+import java.sql.Timestamp
+import java.util.TimeZone
+
+class VeloxCastSuite extends VeloxWholeStageTransformerSuite with ExpressionEvalHelper {
+ def cast(v: Any, targetType: DataType, timeZoneId: Option[String] = None): Cast = {
+ v match {
+ case lit: Expression =>
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ case _ =>
+ val lit = Literal(v)
+ logDebug(s"Cast from: ${lit.dataType.typeName}, to: ${targetType.typeName}")
+ Cast(lit, targetType, timeZoneId)
+ }
+ }
+
+ test("cast from double to timestamp format") {
+ val originalDefaultTz = TimeZone.getDefault
+ try {
+ TimeZone.setDefault(TimeZone.getTimeZone("UTC"))
Review Comment:
We may need this, since Cast.scala returns the internal representation of timestamp in Long. However the timezone will be used for the timestamp generation from the epoch seconds
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-8639][VL] Support casting from double to timestamp [incubator-gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8640:
URL: https://github.com/apache/incubator-gluten/pull/8640#issuecomment-2629416226
Run Gluten ClickHouse CI on ARM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org