You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/05/28 03:25:56 UTC
[GitHub] [pulsar] DGBStarSky opened a new issue #7076: Pulsar SQL cannot get the latest message
DGBStarSky opened a new issue #7076:
URL: https://github.com/apache/pulsar/issues/7076
**Describe the bug**
Pulsar SQL cannot get the latest message of topic.
**To Reproduce**
Steps to reproduce the behavior:
I use pulsar sql query messages, but cannot get the latest message of topic.I have configured the parameters in broker.conf, but no effect.
**Screenshots**
![Uploading image.png…]()
**Desktop (please complete the following information):**
ubuntu 18.04
**Additional context**
chart deployment
2.5.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] ErikJansenIRefact commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
ErikJansenIRefact commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-662999827
@sijie Any idea when the documentation is updated. We are in the middle of testing Presto with Pulsar.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] rec7y33 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
rec7y33 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-673841282
bump
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wen-qiang001 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
wen-qiang001 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-639338978
I set up bookkeeperExplicitLacIntervalInMills in broker.conf and standalone.conf, but still couldn't get the latest news
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] DGBStarSky closed issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
DGBStarSky closed issue #7076:
URL: https://github.com/apache/pulsar/issues/7076
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] javamrGithub removed a comment on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
javamrGithub removed a comment on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-639336113
I set up bookkeeperExplicitLacIntervalInMills in broker.conf and standalone.conf, but still couldn't get the latest news
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-657969573
@bleacha870721 the setting requires some configuration changes in the bookies' side (e.g. bump journal version). We need to update the documentation about the full process.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #7076:
URL: https://github.com/apache/pulsar/issues/7076
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wen-qiang001 removed a comment on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
wen-qiang001 removed a comment on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-639338978
I set up bookkeeperExplicitLacIntervalInMills in broker.conf and standalone.conf, but still couldn't get the latest news
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-734026650
Close via #8635.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobDickinson commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
RobDickinson commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-699323797
@sijie I'd pay a cash bounty for a solution -- doesn't need to be full documentation, but just pointing us in the right direction. We've been trying to figure this out and would really appreciate your help! And we'd be happy to contribute this back as a pull request to the docs as well.
Anybody else on this thread -- same offer applies, happy to pay you for your time if you can share the answer on this!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wen-qiang001 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
wen-qiang001 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-639345726
I set up bookkeeperExplicitLacIntervalInMills in broker.conf and standalone.conf, but still couldn't get the latest news。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] javamrGithub commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
javamrGithub commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-639336113
I set up bookkeeperExplicitLacIntervalInMills in broker.conf and standalone.conf, but still couldn't get the latest news
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-699638900
@RobDickinson I replied in the email.
for ExplicitLac to work, you need to configure the following settings in bookies:
journalFormatVersionToWrite=6
fileInfoFormatVersionToWrite=1
and then turn on the explicit lac interval on broker.
bookkeeperExplicitLacIntervalInMills=1000
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] opera33 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
opera33 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-635771948
Tested this in 2.5.2 as well in Standalone mode, not getting latest data from Pulsar SQL
Previously opened this #6884
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bleacha870721 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
bleacha870721 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-663387026
> @bleacha870721 the setting requires some configuration changes in the bookies' side (e.g. bump journal version). We need to update the documentation about the full process.
Any idea when the documentation is updated？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
zymap commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-714882862
It seems the bookkeeper client still uses the piggyback lac. We need the bookkeeper pr https://github.com/apache/bookkeeper/pull/1901, which can make pulsar read the explicit LAC from the bookkeeper. After this change merged, then update the pulsar to use that bookkeeper version. Then setting the `bookkeeperExplicitLacIntervalInMills` in the broker configuration file, pulsar SQL can query the last message.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Raven888888 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
Raven888888 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-927678755
>
>
> @RobDickinson I replied in the email.
>
> for ExplicitLac to work, you need to configure the following settings in bookies:
>
> journalFormatVersionToWrite=6
> fileInfoFormatVersionToWrite=1
>
> and then turn on the explicit lac interval on broker.
>
> bookkeeperExplicitLacIntervalInMills=1000
Tried the above for a pulsar cluster of 3 nodes. Still unable to get the latest row. Am using 2.7.0 pulsar release.
Anybody has any idea? Has this been verified on a cluster setup?
Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] javamrGithub removed a comment on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
javamrGithub removed a comment on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-639337069
I set up bookkeeperExplicitLacIntervalInMills in broker.conf and standalone.conf, but still couldn't get the latest news
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bleacha870721 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
bleacha870721 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-699909964
@sijie i have tried this on pulsar 2.5.2：
for ExplicitLac to work, you need to configure the following settings in bookies:
journalFormatVersionToWrite=6
fileInfoFormatVersionToWrite=1
and then turn on the explicit lac interval on broker.
bookkeeperExplicitLacIntervalInMills=1000
But still cannot get latest message by pulsar sql.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] javamrGithub commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
javamrGithub commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-639337069
I set up bookkeeperExplicitLacIntervalInMills in broker.conf and standalone.conf, but still couldn't get the latest news
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] bleacha870721 commented on issue #7076: Pulsar SQL cannot get the latest message
Posted by GitBox <gi...@apache.org>.
bleacha870721 commented on issue #7076:
URL: https://github.com/apache/pulsar/issues/7076#issuecomment-657966039
@sijie May I know the current progress of this issue? I still couldn't get the latest messages in 2.6.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org