Azure event hub checkpointing with Qpid proton C++ API

[Arun Koshal <ak...@gmail.com>]

Hi,

I plan to use Apache qpid proton C++ API for receiving messages from Azure
Event hub. I am able to implement a client and fetch the events. The
problem is, whenever I restart my clients it reads all the events from the
event hub.

A quick search revealed that Event hub provides stream offset with the
events sent to the readers which the reader can remember. The reader can
start reading from a new offset which it hasn't read so far.

I did a lot of searching in Qpid proton code and the C++ API, but could not
find a way to get the following:
1. the offset value when a message is received from event hub
2. a way to specify the offset from which to start reading messages, while
opening a receiver

It would be a great help if someone can suggest a solution for this. Please
note that I can only use C or C++ API only.

Thanks in advance!
Arun

Re: Azure event hub checkpointing with Qpid proton C++ API

[Gordon Sim <gs...@redhat.com>]

On Wed, Sep 22, 2021 at 8:11 PM Adrian Florea <fl...@gmail.com> wrote:
> You can take a look at:
>
> https://azure.github.io/amqpnetlite/articles/azure_eventhubs.html
>
> Look at the "Filter" section.

For an example of setting a filter with proton c++:
https://github.com/apache/qpid-proton/blob/main/cpp/examples/selected_recv.cpp#L38

There is no example for setting message annotations, but it is similar
to setting properties,
https://github.com/apache/qpid-proton/blob/main/cpp/examples/colour_send.cpp#L64,
except instead of the properties() method you use
message_annotations() as per
https://qpid.apache.org/releases/qpid-proton-0.35.0/proton/cpp/api/classproton_1_1message.html#a5d96699458d8be84d821f5651a7b3717


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: Azure event hub checkpointing with Qpid proton C++ API

[Adrian Florea <fl...@gmail.com>]

Hi,

You can take a look at:

https://azure.github.io/amqpnetlite/articles/azure_eventhubs.html

Look at the "Filter" section.
Most likely, you'll be able to implement a similar pattern.

Hope this helps,
Adrian.


On Wed, Sep 22, 2021, 1:05 PM Gordon Sim <gs...@redhat.com> wrote:

> On Wed, Sep 22, 2021 at 6:54 PM Arun Koshal <ak...@gmail.com> wrote:
> > I plan to use Apache qpid proton C++ API for receiving messages from
> Azure
> > Event hub. I am able to implement a client and fetch the events. The
> > problem is, whenever I restart my clients it reads all the events from
> the
> > event hub.
> >
> > A quick search revealed that Event hub provides stream offset with the
> > events sent to the readers which the reader can remember. The reader can
> > start reading from a new offset which it hasn't read so far.
>
> What is the mechanism in AMQP terms by which that is done? E.g. is it
> a link property, a filter, ....?
>
> > I did a lot of searching in Qpid proton code and the C++ API, but could
> not
> > find a way to get the following:
> > 1. the offset value when a message is received from event hub
> > 2. a way to specify the offset from which to start reading messages,
> while
> > opening a receiver
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>

Re: Azure event hub checkpointing with Qpid proton C++ API

[Gordon Sim <gs...@redhat.com>]

On Wed, Sep 22, 2021 at 6:54 PM Arun Koshal <ak...@gmail.com> wrote:
> I plan to use Apache qpid proton C++ API for receiving messages from Azure
> Event hub. I am able to implement a client and fetch the events. The
> problem is, whenever I restart my clients it reads all the events from the
> event hub.
>
> A quick search revealed that Event hub provides stream offset with the
> events sent to the readers which the reader can remember. The reader can
> start reading from a new offset which it hasn't read so far.

What is the mechanism in AMQP terms by which that is done? E.g. is it
a link property, a filter, ....?

> I did a lot of searching in Qpid proton code and the C++ API, but could not
> find a way to get the following:
> 1. the offset value when a message is received from event hub
> 2. a way to specify the offset from which to start reading messages, while
> opening a receiver


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org