You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by GitBox <gi...@apache.org> on 2021/10/13 03:53:26 UTC
[GitHub] [incubator-pegasus] cauchy1988 opened a new issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
cauchy1988 opened a new issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824
After stopping inputting data to the storage, when using get_unordered_scanners to scan the database data, if a certain replicaserver is restarted, there will be a phenomenon that the data of some shards is not scanned completely and correspond scanner exits early;
The reasons for the problem are analyzed by viewing the code as follows:
In the scanning process, the client requests the server to be divided into two parts, on_get_scanner and on_scan. on_get_scanner: mainly establishes a context with the server; while on_scan: the client continuously scans data from the server
When restarting a replicaserver, some primary shards will be transferred to the new replicaserver, so the context determined by the client and server through on_get_scanner is lost; at this time, the client will have its own retry mechanism in on_get_scanner and the new replicaserver to determine the new Context, but at this time, the client will pass the last scanned key as start_key to on_get_scanner; the on_get_scanner function will mistakenly think that the intention of this scan is a fixed hashkey scan instead of a full scan, because start_key is a non-empty string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941916034
@Smityz @Shuo-Jia it is not the problem fixed in XiaoMi/pegasus-java-client#156 and XiaoMi/pegasus-go-client#86 ;
it's a new problem;
int this problem
in this problem, server side lost its context with the client and java client's logic will meet the block showing in the above picture;
int this case, java client will call 'on_get_scanners' again and restart the scan process, but at this time, the "start_key" field int the request struct will be filled with hashkey then cause the bug i have said above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] hycdong closed issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
hycdong closed issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941902884
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941894378
The latest version of java && go has fixed the bug:
https://github.com/XiaoMi/pegasus-java-client/pull/156
https://github.com/XiaoMi/pegasus-go-client/pull/86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 edited a comment on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
cauchy1988 edited a comment on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941916034
@Smityz @Shuo-Jia it is not the problem fixed in XiaoMi/pegasus-java-client#156 and XiaoMi/pegasus-go-client#86 ;
it's a new problem;
int this problem
in this problem, server side lost its context with the client and java client's logic will meet the block showing in the above picture;
int this case, java client will call 'on_get_scanners' again and restart the scan process, but at this time, the "start_key" field int the request struct will be filled with hashkey then cause the bug i have said above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Smityz commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
Smityz commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941905738
> i think the best way to fix this is in the server side : in this way, client scanner's process can resume from break-point
yes, but it's difficult to achieve. You can put an issue of your design.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941902884
i think the best way to fix this is in the server side : in this way, client scanner's process can resume from break-point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] cauchy1988 commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
cauchy1988 commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941916034
@Smityz @Shuo-Jia it is not the problem fixed in XiaoMi/pegasus-java-client#156 and XiaoMi/pegasus-go-client#86 ;
it's a new problem;
int this problem
in this problem, server side lost its context with the client and java client's logic will meet the red block showing in the above picture;
int this case, java client will call 'on_get_scanners' again and restart the scan process, but at this time, the "start_key" field int the request struct will be filled with hashkey then cause the bug i have said above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Smityz commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
Smityz commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941905738
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Shuo-Jia commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
Shuo-Jia commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-941894378
The latest version of java && go has fixed the bug:
https://github.com/XiaoMi/pegasus-java-client/pull/156
https://github.com/XiaoMi/pegasus-go-client/pull/86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
[GitHub] [incubator-pegasus] Smityz commented on issue #824: 'full scan' can't scan data completely when one replicaserver restart which causing some primary partition transferring
Posted by GitBox <gi...@apache.org>.
Smityz commented on issue #824:
URL: https://github.com/apache/incubator-pegasus/issues/824#issuecomment-942015866
okay，I see
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org