You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jhchee (via GitHub)" <gi...@apache.org> on 2023/04/19 09:00:10 UTC

[GitHub] [hudi] jhchee opened a new issue, #8499: [SUPPORT] Support partial insert in merge into command

jhchee opened a new issue, #8499:
URL: https://github.com/apache/hudi/issues/8499

   **Describe the problem you faced**
   Currently, if the insert statement specifies less columns than the target table size, the following exception will be thrown:
   ```
   Exception in thread "main" java.lang.AssertionError: assertion failed: The number of insert assignments[3] must equal to the targetTable field size[4]
   ```
   For a wide table that has 1000 columns, the user is required to specifies the all the columns with default values null to avoid this exception. Can we support partial insert in merge (default to null if not specified) into command so the developer can maintain clean SQL statement.
   
   E.g. Delta merge command has already supported for this
   <img width="830" alt="image" src="https://user-images.githubusercontent.com/33974196/233024353-6c44effa-4f0a-4f3c-ad34-7504bab9f4ae.png">
   https://docs.databricks.com/sql/language-manual/delta-merge-into.html
   
   Delata
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   
   1. Using merge into command in spark.sql().
   2. Insert into command specifies only subset of the target table columns.
   
   **Expected behavior**
   Partial insert in merge into command and default to null if not specified. 
   
   **Environment Description**
   
   * Hudi version : 0.13.0
   
   * Spark version : 3.3.1
   
   * Hive version : Irrelevant
   
   * Hadoop version : Irrelevant
   
   * Storage (HDFS/S3/GCS..) : Irrelevant
   
   * Running on Docker? (yes/no) : Irrelevant
   
   
   **Additional context**
   
   Add any other context about the problem here.
   
   **Stacktrace**
   
   ```Add the stacktrace of the error.```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 closed issue #8499: [SUPPORT] Support partial insert in merge into command

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 closed issue #8499: [SUPPORT] Support partial insert in merge into command
URL: https://github.com/apache/hudi/issues/8499


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 commented on issue #8499: [SUPPORT] Support partial insert in merge into command

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #8499:
URL: https://github.com/apache/hudi/issues/8499#issuecomment-1516021215

   Agree, `partial upsert` is more accurate.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 commented on issue #8499: [SUPPORT] Support partial insert in merge into command

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #8499:
URL: https://github.com/apache/hudi/issues/8499#issuecomment-1515635501

   Reasonable, issue created: https://issues.apache.org/jira/browse/HUDI-6105


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] jhchee commented on issue #8499: [SUPPORT] Support partial insert in merge into command

Posted by "jhchee (via GitHub)" <gi...@apache.org>.
jhchee commented on issue #8499:
URL: https://github.com/apache/hudi/issues/8499#issuecomment-1515675847

   @danny0405 I think the ticket title is slightly misleading. It should be Partial insert for merge into.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 commented on issue #8499: [SUPPORT] Support partial insert in merge into command

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #8499:
URL: https://github.com/apache/hudi/issues/8499#issuecomment-1543290269

   Has been supported in https://github.com/apache/hudi/pull/8597, feel free to re-open it if you still got problems.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org