You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "FANNG1 (via GitHub)" <gi...@apache.org> on 2024/02/22 12:58:20 UTC
[I] [Bug] failed to insert data to hive partition table if using `INSERT INTO` for spark-hive-connector [kyuubi]
FANNG1 opened a new issue, #6078:
URL: https://github.com/apache/kyuubi/issues/6078
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
failed to insert data to hive partition table if using `INSERT INTO` for spark-hive-connector, could be reproduced by replace `INSERT INTO` with `INSERT OVERWRITE` in HiveQuerySuite.scala
```
private def readPartitionedTable(format: String, hiveTable: Boolean): Unit = {
withSparkSession() { spark =>
val table = "hive.default.employee"
withTempPartitionedTable(spark, table, format, hiveTable) {
spark.sql(
s"""
| INSERT INTO
| $table PARTITION(year = '2023')
| VALUES("zhao", "09")
|""".stripMargin)
checkQueryResult(s"select * from $table", spark, Array(Row.apply("zhao", "2023", "09")))
}
}
}
```
### Affects Version(s)
master
### Kyuubi Server Log Output
```logtalk
org.apache.kyuubi.spark.connector.hive.KyuubiHiveConnectorException: Dynamic partition strict mode requires at least one static partition column. To turn this off set hive.exec.dynamic.partition.mode=nonstrict
at org.apache.kyuubi.spark.connector.hive.write.HiveWrite.extractAndValidatePartitionCols(HiveWrite.scala:205)
at org.apache.kyuubi.spark.connector.hive.write.HiveWrite.toBatch(HiveWrite.scala:84)
at org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run(WriteToDataSourceV2Exec.scala:360)
at org.apache.spark.sql.execution.datasources.v2.V2ExistingTableWriteExec.run$(WriteToDataSourceV2Exec.scala:359)
```
```
### Kyuubi Engine Log Output
_No response_
### Kyuubi Server Configurations
_No response_
### Kyuubi Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- [ ] No. I cannot submit a PR at this time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Bug] failed to insert data to hive partition table if using `INSERT INTO` for spark-hive-connector [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed issue #6078: [Bug] failed to insert data to hive partition table if using `INSERT INTO` for spark-hive-connector
URL: https://github.com/apache/kyuubi/issues/6078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Bug] failed to insert data to hive partition table if using `INSERT INTO` for spark-hive-connector [kyuubi]
Posted by "Yikf (via GitHub)" <gi...@apache.org>.
Yikf commented on issue #6078:
URL: https://github.com/apache/kyuubi/issues/6078#issuecomment-1960828163
The process of writing with Apache Spark DataSourceV2 using dynamic partitioning to handle static partitions, This exception by the issue reported should be a bug in KSHC, and will take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Bug] failed to insert data to hive partition table if using `INSERT INTO` for spark-hive-connector [kyuubi]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #6078:
URL: https://github.com/apache/kyuubi/issues/6078#issuecomment-1959417512
@Yikf do u have time to take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org
Re: [I] [Bug] failed to insert data to hive partition table if using `INSERT INTO` for spark-hive-connector [kyuubi]
Posted by "FANNG1 (via GitHub)" <gi...@apache.org>.
FANNG1 commented on issue #6078:
URL: https://github.com/apache/kyuubi/issues/6078#issuecomment-1983300036
thanks @Yikf and @pan3793 for fixing the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org