You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Denovo1998 (via GitHub)" <gi...@apache.org> on 2023/05/27 05:49:05 UTC

[GitHub] [pulsar] Denovo1998 opened a new issue, #20414: [improve][broker] Supplement schema ledger if schema ledger is lost

Denovo1998 opened a new issue, #20414:
URL: https://github.com/apache/pulsar/issues/20414

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Motivation
   
   #17221 describes an environment when multiple bookie copies are corrupted, or a Ledger has been deleted. The loss of schema ledger results in new producers and consumers not even being created and working properly.
   
   According to the solution of PR #18010, enable `autoSkipNonRecoverableData` and skip has gotten lost schema can lead to the schema information is not complete. And in the existing code, schema corruption will delete the metadata.
   https://github.com/apache/pulsar/blob/a953027aad38c9f54e952133949280ec2f4c04e8/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/SchemaRegistryServiceImpl.java#L564-L570
   If an error is not recoverable will be deleted, but PR #18010 and #19882 has been maked `NoSuchLedgerExistsOnMetadataServerException` also as recoverable exception.
   
   So we need a solution that does not just skip the schema with the missing ledger, but actually supplements the broken schema ledger.
   
   ### Solution
   
   A new method called `tryCompleteTheLostSchemaLedger`. When the schema ledger losted, if the new consumer subscription or a new producer created, when there is a "Failed to open gotten" such an error, call `tryCompleteTheLostSchemaLedger` method.
   ```java
   CompletableFuture<Long> tryCompleteTheLostSchemaLedger(String schemaId, SchemaVersion version, SchemaData schema);
   ```
   
   This method attempts to create a new ledger save schemaData and then update the new ledger id to the metadata.
   Connected producers and consumers will work even if scheme ledger is deleted. To get the SchemaData, we need to store the SchemaData and SchemaVersion information in the `org.apache.pulsar.broker.service.Producer` and `org.apache.pulsar.broker.service.Consumer` that are connected or subscribed to the topic on the broker side. When calling `tryCompleteTheLostSchemaLedger` incoming.
   
   ### Alternatives
   
   1. In the broker, `org.apache.pulsar.broker.service.Producer` and `org.apache.pulsar.broker.service.Consumer` do not save SchemaData and SchemaVersion, and only call `tryCompleteTheLostSchemaLedger` through the admin api.
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [improve][broker] Supplement schema ledger if schema ledger is lost [pulsar]

Posted by "Denovo1998 (via GitHub)" <gi...@apache.org>.
Denovo1998 commented on issue #20414:
URL: https://github.com/apache/pulsar/issues/20414#issuecomment-1879610702

   In the alternative, the implementation is updated. Needs to be discussed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] github-actions[bot] commented on issue #20414: [improve][broker] Supplement schema ledger if schema ledger is lost

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #20414:
URL: https://github.com/apache/pulsar/issues/20414#issuecomment-1622846478

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [improve][broker] Supplement schema ledger if schema ledger is lost [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #20414: [improve][broker] Supplement schema ledger if schema ledger is lost
URL: https://github.com/apache/pulsar/issues/20414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] github-actions[bot] commented on issue #20414: [improve][broker] Supplement schema ledger if schema ledger is lost

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #20414:
URL: https://github.com/apache/pulsar/issues/20414#issuecomment-1740206776

   The issue had no activity for 30 days, mark with Stale label.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] Denovo1998 commented on issue #20414: [improve][broker] Supplement schema ledger if schema ledger is lost

Posted by "Denovo1998 (via GitHub)" <gi...@apache.org>.
Denovo1998 commented on issue #20414:
URL: https://github.com/apache/pulsar/issues/20414#issuecomment-1697454696

   Waiting to discuss whether this plan is feasible. I will send an email to discuss it later.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [pulsar] Denovo1998 commented on issue #20414: [improve][broker] Supplement schema ledger if schema ledger is lost

Posted by "Denovo1998 (via GitHub)" <gi...@apache.org>.
Denovo1998 commented on issue #20414:
URL: https://github.com/apache/pulsar/issues/20414#issuecomment-1565232703

   @poorbarcode @codelipenghui @rdhabalia PTAL! 
   
   I came up with such a new idea to solve the problem of schema ledger loss. 
   
   Now I'm in `org.apache.pulsar.broker.service.schema.SchemaServiceTest#testSchemaLedgerLost` tested `tryCompleteTheLostSchemaLedger()` is no problem, new producers and consumers work. 
   
   But first we need to talk about **how we get `SchemaVersion` and `SchemaData` in the `SchemaRegistry`**. See if two of the `Solution` and `Alternatives` are feasible, or do you have any other good suggestions? 
   
   The code is in #20415 (some work is not done).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [improve][broker] Supplement schema ledger if schema ledger is lost [pulsar]

Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #20414:
URL: https://github.com/apache/pulsar/issues/20414#issuecomment-6388803774

   Thanks @Denovo1998. As noted when #20415 was closed, master already handles a lost schema ledger without blocking the topic, and PIP-327 added `schemaLedgerForceRecovery` for the remaining cases. Restoring the lost schema history is still a fair goal. It needs a new admin operation and a change to the schema storage interface, so it needs a PIP: https://github.com/apache/pulsar/blob/master/pip/README.md. The suggestion was an admin-only "repair schema version" operation that checks the stored hash and runs the compatibility check. Closing for now; a PIP is welcome.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org