You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2022/08/26 12:12:47 UTC
[GitHub] [incubator-pegasus] xihong08 opened a new issue, #1140: bulkload failed when app with one replica fator and load small amount of data
xihong08 opened a new issue, #1140:
URL: https://github.com/apache/incubator-pegasus/issues/1140
## Bug Report
I try to use bulkload for one replica fator's and 30 partitions's app, and the app just has small amount of data.
but this bulkload failed. I found the key information in the meta's log.
That's about it. 'partition doesn't have bulk load metadata, set bulk load failed'.
and then, i found the log for update bulk load metadata, and check zookkeeper's metadata for partion,
parts of 30 partitions occur this, doesn't have bulk load metadata, It's a matter of probability.
I suspect it was overwritten after the update. and check the code handle_app_downloading,
update_partition_metadata_on_remote_storage and update_partition_info_on_remote_storage‘s Order not guaranteed for asynchronous Multithreading
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] xihong08 commented on issue #1140: bulkload failed when app with one replica fator and load small amount of data
Posted by GitBox <gi...@apache.org>.
xihong08 commented on issue #1140:
URL: https://github.com/apache/incubator-pegasus/issues/1140#issuecomment-1240268272
> Thanks for your bug report~
>
> > `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` ‘s order are not guaranteed.
>
> Function `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` are execute in thread pool `THREAD_POOL_META_STATE`, which is single thread in current pegasus design. As a result, the order can be guaranteed, `update_partition_metadata_on_remote_storage` will always execute before `update_partition_info_on_remote_storage`.
>
> However, in current bulk load design, bulk load will be failed if there are some partitions don't have data. For example, if you have a table will 16 partitions, and partition[0] doesn't have data to be ingested, bulk load will be failed.
>
> I suggest:
>
> 1. check your configuration whether threadpool THREAD_POOL_META_STATE is single thread
> 2. check if all partitions have data to be ingested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] xihong08 closed issue #1140: bulkload failed when app with one replica fator and load small amount of data
Posted by GitBox <gi...@apache.org>.
xihong08 closed issue #1140: bulkload failed when app with one replica fator and load small amount of data
URL: https://github.com/apache/incubator-pegasus/issues/1140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] hycdong commented on issue #1140: bulkload failed when app with one replica fator and load small amount of data
Posted by GitBox <gi...@apache.org>.
hycdong commented on issue #1140:
URL: https://github.com/apache/incubator-pegasus/issues/1140#issuecomment-1236776146
Thanks for your bug report~
> `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` ‘s order are not guaranteed.
Function `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` are execute in thread pool `THREAD_POOL_META_STATE`, which is single thread in current pegasus design. As a result, the order can be guaranteed, `update_partition_metadata_on_remote_storage` will always execute before `update_partition_info_on_remote_storage`.
However, in current bulk load design, bulk load will be failed if there are some partitions don't have data. For example, if you have a table will 16 partitions, and partition[0] doesn't have data to be ingested, bulk load will be failed.
I suggest:
1. check your configuration whether threadpool THREAD_POOL_META_STATE is single thread
2. check if all partitions have data to be ingested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] xihong08 commented on issue #1140: bulkload failed when app with one replica fator and load small amount of data
Posted by GitBox <gi...@apache.org>.
xihong08 commented on issue #1140:
URL: https://github.com/apache/incubator-pegasus/issues/1140#issuecomment-1240268769
> > Thanks for your bug report~
> > > `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` ‘s order are not guaranteed.
> >
> >
> > Function `update_partition_metadata_on_remote_storage` and `update_partition_info_on_remote_storage` are execute in thread pool `THREAD_POOL_META_STATE`, which is single thread in current pegasus design. As a result, the order can be guaranteed, `update_partition_metadata_on_remote_storage` will always execute before `update_partition_info_on_remote_storage`.
> > However, in current bulk load design, bulk load will be failed if there are some partitions don't have data. For example, if you have a table will 16 partitions, and partition[0] doesn't have data to be ingested, bulk load will be failed.
> > I suggest:
> >
> > 1. check your configuration whether threadpool THREAD_POOL_META_STATE is single thread
> > 2. check if all partitions have data to be ingested.
THREAD_POOL_META_STATE has 2 threads
all partions have data need to ingested
update partition's metadata failed is my issue。like this, my partition's data has 30M, but when start ingest prompte 'partition doesn't have bulk load metadata, set bulk load failed'.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org