You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/08/13 10:21:38 UTC
[GitHub] [pulsar] hnail opened a new issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
hnail opened a new issue #7811:
URL: https://github.com/apache/pulsar/issues/7811
## **Describe the bug**
Pulsar SQL can't query the InactiveTopics and un-managedLedger
## **Bug trace**
### Pulsar SQL [PulsarSplitManager.getSplitsForTopic()](https://github.com/apache/pulsar/blob/master/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarSplitManager.java#L261) split with this code:
``readOnlyCursor = managedLedgerFactory.openReadOnlyCursor(
topicNamePersistenceEncoding,
PositionImpl.earliest, new ManagedLedgerConfig());``
this will producing two unexpected result for me :
### 1) when produce expired with brokerDeleteInactiveTopicsMaxInactiveDurationSeconds ,the topic will log this on **broker** :
``08:31:43.073 [bookkeeper-..] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://aa/bb/cc-partition-0] Topic deleted successfully due to inactivity``
that will raise exception on **presto** :
``org.apache.bookkeeper.mledger.ManagedLedgerException$ManagedLedgerNotFoundException: org.apache.bookkeeper.mledger.ManagedLedgerException$MetadataNotFoundException: Managed ledger not found``
### 2) adjust brokerDeleteInactiveTopicsMaxInactiveDurationSeconds to infinite :
[PulsarSplitManager.getSplitsForTopic()] (https://github.com/apache/pulsar/blob/master/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarSplitManager.java#L261) only return the managedLedger (look like the Ledger with writing status, only one Ledger for one partiton ),that case lost read the history data。
### **Bug trace**
Pulsar SQL can read history or Inactive Topic data
### **Additional context**
TAG : `managed-ledger` `Pulsar SQL`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hnail commented on issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
hnail commented on issue #7811:
URL: https://github.com/apache/pulsar/issues/7811#issuecomment-673851935
> @hnail Can you disable `deleteInactiveTopics` and enable retention?
Thanks for your reply, I will try this and fed back to this issues today
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hnail commented on issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
hnail commented on issue #7811:
URL: https://github.com/apache/pulsar/issues/7811#issuecomment-673864951
> @hnail Can you disable `deleteInactiveTopics` and enable retention?
@sijie Thanks for your reply , set `retentionTimeInMinutes` `retentionSizeInMB` works for my test .
I think we should prompted `retention` and `deleteInactiveTopics` on [Pulsar SQL Overview doc](https://pulsar.apache.org/docs/en/sql-overview/)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Huanli-Meng commented on issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
Huanli-Meng commented on issue #7811:
URL: https://github.com/apache/pulsar/issues/7811#issuecomment-673868702
ok
> @hnail Agreed.
>
> @Huanli-Meng @Jennifer88huang Can you work with @hnail on adding documentation to the Pulsar SQL doc?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hnail removed a comment on issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
hnail removed a comment on issue #7811:
URL: https://github.com/apache/pulsar/issues/7811#issuecomment-673851935
> @hnail Can you disable `deleteInactiveTopics` and enable retention?
Thanks for your reply, I will try this and fed back to this issues today
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7811:
URL: https://github.com/apache/pulsar/issues/7811#issuecomment-673841713
@hnail Can you disable `deleteInactiveTopics` and enable retention?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #7811:
URL: https://github.com/apache/pulsar/issues/7811
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7811:
URL: https://github.com/apache/pulsar/issues/7811#issuecomment-673866319
@hnail Agreed.
@Huanli-Meng @Jennifer88huang Can you work with @hnail on adding documentation to the Pulsar SQL doc?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7811: Pulsar SQL can't query the InactiveTopics and un-managedLedger
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7811:
URL: https://github.com/apache/pulsar/issues/7811#issuecomment-673841948
We can improve the error message. /cc @codelipenghui in the conversation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org