You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by sujith71955 <gi...@git.apache.org> on 2018/11/08 07:27:41 UTC
[GitHub] spark pull request #22721: [SPARK-25403][SQL] Refreshes the table after inse...
Github user sujith71955 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22721#discussion_r231785137
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala ---
@@ -183,13 +183,14 @@ case class InsertIntoHadoopFsRelationCommand(
refreshUpdatedPartitions(updatedPartitionPaths)
}
- // refresh cached files in FileIndex
- fileIndex.foreach(_.refresh())
- // refresh data cache if table is cached
- sparkSession.catalog.refreshByPath(outputPath.toString)
-
if (catalogTable.nonEmpty) {
+ sparkSession.sessionState.catalog.refreshTable(catalogTable.get.identifier)
--- End diff --
Already in CommandUtils.updateTableStats(sparkSession, catalogTable.get) flow we are invalidating table relation cache, then do we need to call invalidate here also? May i know the difference between these two statements Thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org