[I] MOR seems not working as expected very well in some corner case [hudi]

["bithw1 (via GitHub)" <gi...@apache.org>]

bithw1 opened a new issue, #17515:
URL: https://github.com/apache/hudi/issues/17515

   ### Describe the problem you faced
   
   Hi，
   
   I have a simple MOR table, defined as follows:
   
   ```
   create table test_hudi_table_mor (
   a int,
   b int,
   c int
   ) using hudi
    partitioned by (c)
    options (
     primaryKey = 'a',
     preCombineField = 'b',
     type = 'mor'
    );
   ```
   Then, I do insert for twice,
   
   ```
   insert into test_hudi_table_mor  values(1,1,1);
   insert into test_hudi_table_mor  values(2,2,2);
   ```
   
   When I looked at the files created, I find two parquet files, no log files
   
   Then, I do one update
   
   ```
    update test_hudi_table_mor  set b = 11 where a =1;
   ```
   
   Then, I find a log file, so I would ask, why the two insert operations doesn't generate log files and only update generate, thanks.
   
   
   ### To Reproduce
   
   1.
   2.
   3.
   4.
   
   
   ### Expected behavior
   
   1
   
   ### Environment Description
   
   * Hudi version:
   * Spark version:
   * Flink version:
   * Hive version:
   * Hadoop version:
   * Storage (HDFS/S3/GCS..):
   * Running on Docker? (yes/no):
   
   
   ### Additional context
   
   1
   
   ### Stacktrace
   
   ```shell
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] MOR seems not working as expected very well in some corner case [hudi]

["bithw1 (via GitHub)" <gi...@apache.org>]

bithw1 commented on issue #17515:
URL: https://github.com/apache/hudi/issues/17515#issuecomment-3658620848

   Thanks @cshuo  for the helpful answer, I tried your approach and it works,thansk!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] MOR seems not working as expected very well in some corner case [hudi]

["bithw1 (via GitHub)" <gi...@apache.org>]

bithw1 closed issue #17515: MOR seems not working as expected very well in some corner case
URL: https://github.com/apache/hudi/issues/17515


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] MOR seems not working as expected very well in some corner case [hudi]

["cshuo (via GitHub)" <gi...@apache.org>]

cshuo commented on issue #17515:
URL: https://github.com/apache/hudi/issues/17515#issuecomment-3624368487

   @bithw1 it depends on which index you're using, spark use [SIMPLE](https://hudi.apache.org/docs/0.15.0/indexing/#index-types-in-hudi) index by default, which do not support file slice without base file, so base file will be generated for the first insert records. If you use index like `hoodie.index.type = 'INMEMORY'`, you can see log files generated for the first two inserts.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org