You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/12/08 07:35:49 UTC

[GitHub] [hudi] BruceKellan opened a new issue #4247: [SUPPORT] Unsupport operation exception occur when using flink+hudi in bulk_insert mode

BruceKellan opened a new issue #4247:
URL: https://github.com/apache/hudi/issues/4247


   **Describe the problem you faced**
   
   I am using flink+hudi to initial dataset from hive.
   but unsupport operation exception occur like this, it seems like doesn't support map type in bulk_insert mode but it's supported in upsert mode.
   
   ```java
   Caused by: java.lang.UnsupportedOperationException: Unsupported type: MAP<STRING, STRING>
     at org.apache.hudi.io.storage.row.parquet.ParquetSchemaConverter.convertToParquetType(ParquetSchemaConverter.java:620)
     at org.apache.hudi.io.storage.row.parquet.ParquetSchemaConverter.convertToParquetType(ParquetSchemaConverter.java:544)
     at org.apache.hudi.io.storage.row.parquet.ParquetSchemaConverter.convertToParquetMessageType(ParquetSchemaConverter.java:538)
     at org.apache.hudi.io.storage.row.RowDataParquetWriteSupport.<init>(RowDataParquetWriteSupport.java:45)
     at org.apache.hudi.io.storage.row.HoodieRowDataParquetWriteSupport.<init>(HoodieRowDataParquetWriteSupport.java:47)
     at org.apache.hudi.io.storage.row.HoodieRowDataFileWriterFactory.newParquetInternalRowFileWriter(HoodieRowDataFileWriterFactory.java:68)
     at org.apache.hudi.io.storage.row.HoodieRowDataFileWriterFactory.getRowDataFileWriter(HoodieRowDataFileWriterFactory.java:54)
     at org.apache.hudi.io.storage.row.HoodieRowDataCreateHandle.createNewFileWriter(HoodieRowDataCreateHandle.java:203)
     at org.apache.hudi.io.storage.row.HoodieRowDataCreateHandle.<init>(HoodieRowDataCreateHandle.java:100)
     at org.apache.hudi.sink.bulk.BulkInsertWriterHelper.getRowCreateHandle(BulkInsertWriterHelper.java:118)
     at org.apache.hudi.sink.bulk.BulkInsertWriterHelper.write(BulkInsertWriterHelper.java:97)
     at org.apache.hudi.sink.append.AppendWriteFunction.processElement(AppendWriteFunction.java:82)
     at org.apache.flink.streaming.api.operators.ProcessOperator.processElement(ProcessOperator.java:66)
     at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71)
     at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46)
     at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26)
     at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50)
     at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28)
     at StreamExecCalc$939.processElement(Unknown Source)
   ```
   
   
   **Expected behavior**
   
   * Hudi version : 0.10.0
   * Hive version : 2.3.7
   * Hadoop version : 2.7.3
   * Storage (HDFS/S3/GCS..) : HDFS
   * Running on Docker? (yes/no) : no
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] BruceKellan commented on issue #4247: [SUPPORT] Unsupport operation exception occur when using flink+hudi in bulk_insert mode

Posted by GitBox <gi...@apache.org>.
BruceKellan commented on issue #4247:
URL: https://github.com/apache/hudi/issues/4247#issuecomment-1000127817


   @danny0405 Will it be fixed in the next release version?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] BruceKellan closed issue #4247: [SUPPORT] Unsupport operation exception occur when using flink+hudi in bulk_insert mode

Posted by GitBox <gi...@apache.org>.
BruceKellan closed issue #4247:
URL: https://github.com/apache/hudi/issues/4247


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] BruceKellan commented on issue #4247: [SUPPORT] Unsupport operation exception occur when using flink+hudi in bulk_insert mode

Posted by GitBox <gi...@apache.org>.
BruceKellan commented on issue #4247:
URL: https://github.com/apache/hudi/issues/4247#issuecomment-1002858083


   Thanks. I have seen the PR.
   [Link](https://github.com/apache/hudi/pull/4470)
   I will close it. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org