[jira] [Created] (SPARK-21338) AggregatedDialect doesn't override isCascadingTruncateTable() method

["Ostap Gonchar (JIRA)" <ji...@apache.org>]

Ostap Gonchar created SPARK-21338:
-------------------------------------

             Summary: AggregatedDialect doesn't override isCascadingTruncateTable() method
                 Key: SPARK-21338
                 URL: https://issues.apache.org/jira/browse/SPARK-21338
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.1.1, 2.1.0
            Reporter: Ostap Gonchar


org.apache.spark.sql.jdbc.JdbcDialect's method:
def isCascadingTruncateTable(): Option[Boolean] = None
is not overriden in org.apache.spark.sql.jdbc.AggregatedDialect class.
Because of this issue - *+when you add  more than one dialect Spark doesn't truncate table because isCascadingTruncateTable always returns default None for Aggregated Dialect+*.
Can be tracked when you write dataset in jdbc mode with SaveMode.Overwrite and extra JdbcDialect added via JdbcDialects.registerDialect(...); method.
Then inside of org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider on line 65 in createRelation method it will never get to line 67 even if isTruncate returns true.
It will only get inside if you have one JdbcDialect and AggregatedDialect is not used.
Fix would be to overrite this method in AggregatedDialect and also to reduce value (I guess with and clause same as in overriden getHandle method) e.g. 
override def isCascadingTruncateTable(): Option[Boolean]=
    Some(dialects.map(_.isCascadingTruncateTable()).reduce(_ && _))



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org