You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Brian Candler <b....@pobox.com> on 2019/11/03 08:51:27 UTC
Readers and partitioned topics
In the documentation
<https://pulsar.apache.org/docs/en/next/concepts-clients/> there is an
admonition:
Non-partitioned topics only
The reader interface for Pulsar cannot currently be used withpartitioned
topics
<https://pulsar.apache.org/docs/en/next/concepts-messaging#partitioned-topics>.
Is this still true? I notice in ReaderImpl.Java
* final int partitionIdx =
TopicName.getPartitionIndex(readerConfiguration.getTopicName());**
* consumer = new ConsumerImpl<>(client,
readerConfiguration.getTopicName(), consumerConfiguration,
listenerExecutor, *partitionIdx*, false,
consumerFuture, SubscriptionMode.NonDurable,
readerConfiguration.getStartMessageId(),
readerConfiguration.getStartMessageFromRollbackDurationInSec(),
schema, null, true /* createTopicIfDoesNotExist */);
Thanks,
Brian.
Re: Readers and partitioned topics
Posted by Brian Candler <b....@pobox.com>.
On 04/11/2019 16:34, Matteo Merli wrote:
> The documentation is correct in that a reader can only "read" a
> non-partitioned topic or one specific partition of a topic.
>
> To get a list of the partitions, you can use
> `PulsarClient.getPartitionsForTopic()` and then create a reader for
> each partitions.
Thanks. So I suggest replacing:
"The reader interface for Pulsar cannot currently be used with
partitioned topics."
with something like:
"When using the reader interface with a partitioned topic, a separate
reader instance is required for each partition"
Re: Readers and partitioned topics
Posted by Brian Candler <b....@pobox.com>.
On 04/11/2019 16:34, Matteo Merli wrote:
> The documentation is correct in that a reader can only "read" a
> non-partitioned topic or one specific partition of a topic.
>
> To get a list of the partitions, you can use
> `PulsarClient.getPartitionsForTopic()` and then create a reader for
> each partitions.
Thanks. So I suggest replacing:
"The reader interface for Pulsar cannot currently be used with
partitioned topics."
with something like:
"When using the reader interface with a partitioned topic, a separate
reader instance is required for each partition"
Re: Readers and partitioned topics
Posted by Matteo Merli <ma...@gmail.com>.
The documentation is correct in that a reader can only "read" a
non-partitioned topic or one specific partition of a topic.
To get a list of the partitions, you can use
`PulsarClient.getPartitionsForTopic()` and then create a reader for
each partitions.
--
Matteo Merli
<ma...@gmail.com>
On Mon, Nov 4, 2019 at 3:30 AM Jinfeng Huang <hj...@streamnative.io> wrote:
>
> From the code snippet, it seems that the description in the document is obsolete. If so, we can update the doc accordingly.
> Could any eng confirm the issue?
>
> Best Regards,
> Jennifer
>
>
> On Sun, Nov 3, 2019 at 4:51 PM Brian Candler <b....@pobox.com> wrote:
>>
>> In the documentation there is an admonition:
>>
>> Non-partitioned topics only
>>
>> The reader interface for Pulsar cannot currently be used with partitioned topics.
>>
>> Is this still true? I notice in ReaderImpl.Java
>>
>> final int partitionIdx = TopicName.getPartitionIndex(readerConfiguration.getTopicName());
>> consumer = new ConsumerImpl<>(client, readerConfiguration.getTopicName(), consumerConfiguration,
>> listenerExecutor, partitionIdx, false, consumerFuture, SubscriptionMode.NonDurable,
>> readerConfiguration.getStartMessageId(), readerConfiguration.getStartMessageFromRollbackDurationInSec(),
>> schema, null, true /* createTopicIfDoesNotExist */);
>>
>> Thanks,
>>
>> Brian.
Re: Readers and partitioned topics
Posted by Matteo Merli <ma...@gmail.com>.
The documentation is correct in that a reader can only "read" a
non-partitioned topic or one specific partition of a topic.
To get a list of the partitions, you can use
`PulsarClient.getPartitionsForTopic()` and then create a reader for
each partitions.
--
Matteo Merli
<ma...@gmail.com>
On Mon, Nov 4, 2019 at 3:30 AM Jinfeng Huang <hj...@streamnative.io> wrote:
>
> From the code snippet, it seems that the description in the document is obsolete. If so, we can update the doc accordingly.
> Could any eng confirm the issue?
>
> Best Regards,
> Jennifer
>
>
> On Sun, Nov 3, 2019 at 4:51 PM Brian Candler <b....@pobox.com> wrote:
>>
>> In the documentation there is an admonition:
>>
>> Non-partitioned topics only
>>
>> The reader interface for Pulsar cannot currently be used with partitioned topics.
>>
>> Is this still true? I notice in ReaderImpl.Java
>>
>> final int partitionIdx = TopicName.getPartitionIndex(readerConfiguration.getTopicName());
>> consumer = new ConsumerImpl<>(client, readerConfiguration.getTopicName(), consumerConfiguration,
>> listenerExecutor, partitionIdx, false, consumerFuture, SubscriptionMode.NonDurable,
>> readerConfiguration.getStartMessageId(), readerConfiguration.getStartMessageFromRollbackDurationInSec(),
>> schema, null, true /* createTopicIfDoesNotExist */);
>>
>> Thanks,
>>
>> Brian.
Re: Readers and partitioned topics
Posted by Jinfeng Huang <hj...@streamnative.io>.
From the code snippet, it seems that the description in the document is
obsolete. If so, we can update the doc accordingly.
Could any eng confirm the issue?
Best Regards,
Jennifer
On Sun, Nov 3, 2019 at 4:51 PM Brian Candler <b....@pobox.com> wrote:
> In the documentation
> <https://pulsar.apache.org/docs/en/next/concepts-clients/> there is an
> admonition:
> Non-partitioned topics only
>
> The reader interface for Pulsar cannot currently be used with partitioned
> topics
> <https://pulsar.apache.org/docs/en/next/concepts-messaging#partitioned-topics>
> .
>
> Is this still true? I notice in ReaderImpl.Java
>
> * final int partitionIdx =
> TopicName.getPartitionIndex(readerConfiguration.getTopicName());*
> consumer = new ConsumerImpl<>(client,
> readerConfiguration.getTopicName(), consumerConfiguration,
> listenerExecutor, *partitionIdx*, false, consumerFuture,
> SubscriptionMode.NonDurable,
> readerConfiguration.getStartMessageId(),
> readerConfiguration.getStartMessageFromRollbackDurationInSec(),
> schema, null, true /* createTopicIfDoesNotExist */);
>
> Thanks,
>
> Brian.
>
Re: Readers and partitioned topics
Posted by Jinfeng Huang <hj...@streamnative.io>.
From the code snippet, it seems that the description in the document is
obsolete. If so, we can update the doc accordingly.
Could any eng confirm the issue?
Best Regards,
Jennifer
On Sun, Nov 3, 2019 at 4:51 PM Brian Candler <b....@pobox.com> wrote:
> In the documentation
> <https://pulsar.apache.org/docs/en/next/concepts-clients/> there is an
> admonition:
> Non-partitioned topics only
>
> The reader interface for Pulsar cannot currently be used with partitioned
> topics
> <https://pulsar.apache.org/docs/en/next/concepts-messaging#partitioned-topics>
> .
>
> Is this still true? I notice in ReaderImpl.Java
>
> * final int partitionIdx =
> TopicName.getPartitionIndex(readerConfiguration.getTopicName());*
> consumer = new ConsumerImpl<>(client,
> readerConfiguration.getTopicName(), consumerConfiguration,
> listenerExecutor, *partitionIdx*, false, consumerFuture,
> SubscriptionMode.NonDurable,
> readerConfiguration.getStartMessageId(),
> readerConfiguration.getStartMessageFromRollbackDurationInSec(),
> schema, null, true /* createTopicIfDoesNotExist */);
>
> Thanks,
>
> Brian.
>