You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/02/03 04:41:50 UTC
[GitHub] [pulsar] merajblueshift opened a new issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
merajblueshift opened a new issue #9441:
URL: https://github.com/apache/pulsar/issues/9441
**Describe the bug**
When creating a subscription using REST API on an existing topic having a good flow of messages, the cursor is set to the earliest position and if the same is done via pulsar-admin it sets the cursor correctly.
**To Reproduce**
Steps to reproduce the behavior:
1. Create a topic having messages flowing in continuously and create a consumer for it.
2. Let some messages get accumulated.
3. Create a new subscription via REST API using this REST API
```
curl -X PUT -L http://<pulsar_url>:8080/admin/v2/persistent/<tenant>/<namespace>/<topic>/subscription/<subscription_name>_rest_api -H 'Content-Type: application/json' -d '{
"messageId": "latest"
}'
```
4. Also create a subscription using Admin API
```
pulsar-admin topics create-subscription -m latest -s <subscription_name>_admin_api persistent://<tenant>/<namespace>/<topic>
```
**Expected behavior**
The subscription created via REST API should also have the cursor set on the latest.
**Screenshots**
Both subscription_adminApi and subscription_restApi were created at the same time. See the difference in backlog.
<img width="1637" alt="Screenshot 2021-02-03 at 10 08 33 AM" src="https://user-images.githubusercontent.com/58764022/106698934-da683a00-6607-11eb-985e-3ec1d10fefba.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #9441:
URL: https://github.com/apache/pulsar/issues/9441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap commented on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
zymap commented on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-772374199
Hi @merajblueshift , I checkout the pulsar admin request, it sends a HTTP request like this:
```
PUT /admin/v2/persistent/public/default/test/subscription/test HTTP/1.1
Accept: application/json
Content-Type: application/json
content-length: 82
host: localhost:8088
user-agent: Pulsar-Java-v2.7.0
{"ledgerId":9223372036854775807,"entryId":9223372036854775807,"partitionIndex":-1}
```
As you see, it doesn't send a message position like `{"messageId": "latest"}`.
In pulsar admin, we convert the string `latest` to the `MessageId.latest` https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/MessageId.java#L86
So actually, it passes a position ({"ledgerId":9223372036854775807,"entryId":9223372036854775807,"partitionIndex":-1}) to the broker with the REST API.
I think we should handle the position string like `earliest` and `latest` in the broker as well like how we did in the pulsar-admin client.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. When using the REST API, I think we have to passed a `MessageIdImpl` type parameter to broker.
@merajblueshift You can use REST API like this first.
for latest position
`-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'`
for earliest position
`-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, @zymap please feel free to open a new PR if you find a way to fix this problem from broker side.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. When using the REST API, I think we have to passed a `MessageIdImpl` type parameter to broker.
@merajblueshift You can use REST API like this first.
for latest position
`-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'`
for earliest position
`-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. When using the REST API, I think we have to passed a `MessageIdImpl` type parameter to broker, such as
`-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'`
or
`-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-966393158
I can work on this. @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. When using the REST API, I think we have to passed a `MessageIdImpl` type parameter to broker.
@merajblueshift You can use REST API like this first.
for latest position
`-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'`
for earliest position
`-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang commented on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. I think we have to passed a `MessageIdImpl` type parameter to broker, like `-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'` or `-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
zymap edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-772374199
Hi @merajblueshift , I checkout the pulsar admin request, it sends a HTTP request like this:
```
PUT /admin/v2/persistent/public/default/test/subscription/test HTTP/1.1
Accept: application/json
Content-Type: application/json
content-length: 82
host: localhost:8088
user-agent: Pulsar-Java-v2.7.0
{"ledgerId":9223372036854775807,"entryId":9223372036854775807,"partitionIndex":-1}
```
As you see, it doesn't send a message position like `{"messageId": "latest"}`.
In pulsar admin, we convert the string `latest` to the `MessageId.latest` https://github.com/apache/pulsar/blob/fa41d02bebfd841767846240f3ae574047f118f0/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/MessageId.java#L86
So actually, it passes a position ({"ledgerId":9223372036854775807,"entryId":9223372036854775807,"partitionIndex":-1}) to the broker with the REST API.
I think we should handle the position string like `earliest` and `latest` in the broker like how we did in the pulsar-admin client.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. When using the REST API, I think we have to passed a `MessageIdImpl` type parameter to broker.
@merajblueshift You can use REST API like this first.
for latest position
`-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'`
for earliest position
`-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, @zymap please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang commented on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. I think we have to passed a `MessageIdImpl` type parameter to broker, like `-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'` or `-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. I think we have to passed a `MessageIdImpl` type parameter to broker, like
`-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'`
or
`-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang edited a comment on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
aloyszhang edited a comment on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-773120451
@zymap Thanks for your comments.
As I comment in https://github.com/apache/pulsar/pull/9444. When using the REST API, I think we have to passed a `MessageIdImpl` type parameter to broker, like
`-d '{"ledgerId": 9223372036854775807, "entryId": 9223372036854775807, "partitionIndex": -1}'`
or
`-d '{"ledgerId": -1, "entryId": -1, "partitionIndex": -1}'`.
I removed the link of https://github.com/apache/pulsar/pull/9444 since that PR does not fix this issue, please feel free to open a new PR if you find a way to fix this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #9441: Creating subscription on a Topic via REST API sets the cursor to earliest position
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #9441:
URL: https://github.com/apache/pulsar/issues/9441#issuecomment-788944534
@zymap makes sense to me, we can add a new endpoint to handle this issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org