[GitHub] [incubator-pegasus] hycdong commented on issue #1140: bulkload failed when app with one replica fator and load small amount of data

[GitBox <gi...@apache.org>]

hycdong commented on issue #1140:
URL: https://github.com/apache/incubator-pegasus/issues/1140#issuecomment-1236776146

   Thanks for your bug report~
   
   > `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` ‘s order are not guaranteed.
   
   Function `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` are execute in thread pool `THREAD_POOL_META_STATE`, which is single thread in current pegasus design. As a result, the order can be guaranteed, `update_partition_metadata_on_remote_storage` will always execute before `update_partition_info_on_remote_storage`. 
   
   However, in current bulk load design, bulk load will be failed if there are some partitions don't have data. For example, if you have a table will 16 partitions, and partition[0] doesn't have data to be ingested, bulk load will be failed.
   
   I suggest:
   1. check your configuration whether threadpool THREAD_POOL_META_STATE is single thread
   2. check if all partitions have data to be ingested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org