You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/11/10 09:07:36 UTC
[GitHub] [spark] LuciferYang opened a new pull request, #38598: [SPARK-41097][CORE][SQL] Remove redundant collection conversion base on Scala 2.13 code
LuciferYang opened a new pull request, #38598:
URL: https://github.com/apache/spark/pull/38598
### What changes were proposed in this pull request?
This pr aims clean up redundant collection conversion base on Scala 2.13 code
### Why are the changes needed?
Code clean up.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #38598:
URL: https://github.com/apache/spark/pull/38598#discussion_r1027893918
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala:
##########
@@ -940,7 +940,7 @@ case class ShowTablePropertiesCommand(
}
case None =>
properties.filterKeys(!_.startsWith(CatalogTable.VIEW_PREFIX))
- .toSeq.sortBy(_._1).map(p => Row(p._1, p._2)).toSeq
+ .toSeq.sortBy(_._1).map(p => Row(p._1, p._2))
Review Comment:
`toSeq` has been called previously
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #38598:
URL: https://github.com/apache/spark/pull/38598#discussion_r1027895822
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ShowTablePropertiesExec.scala:
##########
@@ -47,7 +47,7 @@ case class ShowTablePropertiesExec(
}
case None =>
properties.toSeq.sortBy(_._1).map(kv =>
- toCatalystRow(kv._1, kv._2)).toSeq
+ toCatalystRow(kv._1, kv._2))
Review Comment:
`toSeq` has been called previously
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38598: [SPARK-41097][CORE][SQL] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38598:
URL: https://github.com/apache/spark/pull/38598#issuecomment-1309979299
Will manually test Scala 2.13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #38598:
URL: https://github.com/apache/spark/pull/38598#discussion_r1027885751
##########
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala:
##########
@@ -568,7 +568,7 @@ private[spark] class ExecutorAllocationManager(
// We don't want to change our target number of executors, because we already did that
// when the task backlog decreased.
if (decommissionEnabled) {
- val executorIdsWithoutHostLoss = executorIdsToBeRemoved.toSeq.map(
Review Comment:
No need to `toSeq` here because `executorIdsToBeRemoved(ArrayBuffer).map` finally calls toArray
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38598:
URL: https://github.com/apache/spark/pull/38598#issuecomment-1317151904
Let me check again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38598:
URL: https://github.com/apache/spark/pull/38598#issuecomment-1321911349
Seems there is no API involved in lazy computing. I think it should be safe. Let me verify full UTs with 2.13 again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
srowen commented on PR #38598:
URL: https://github.com/apache/spark/pull/38598#issuecomment-1317141191
These are probably fine. Some may be hold-overs from earlier versions of Scala. I'm slightly worried that in some cases they cause a copy and we actually _rely_ on that, though this is a bad way to rely on it. SKimming it, I am not sure that's true in these cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #38598:
URL: https://github.com/apache/spark/pull/38598#discussion_r1027892237
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala:
##########
@@ -555,7 +555,7 @@ object StructType extends AbstractDataType {
def apply(fields: java.util.List[StructField]): StructType = {
import scala.collection.JavaConverters._
- StructType(fields.asScala.toSeq)
Review Comment:
This represents `fields.asScala.toSeq.toArray`, seems `toSeq` is redundant
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
srowen commented on PR #38598:
URL: https://github.com/apache/spark/pull/38598#issuecomment-1325129369
Let me know when it seems ready to merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
srowen commented on PR #38598:
URL: https://github.com/apache/spark/pull/38598#issuecomment-1326580994
Merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38598:
URL: https://github.com/apache/spark/pull/38598#issuecomment-1326049324
ready to merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #38598: [SPARK-41097][CORE][SQL][SS][PROTOBUF] Remove redundant collection conversion base on Scala 2.13 code
URL: https://github.com/apache/spark/pull/38598
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org