You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by SRIKANT MVS <sr...@gmail.com> on 2020/12/10 07:47:17 UTC
How to produce messages to Azure SQL DB in Camel
Team,
I have a usecase where I need to replicate data for a specific duration
from one Azure SQL DB (subscription1) to another Azure SQL DB
(supbscription2).
The source DB has live data with continuous inserts, updates and deletes
possibility. Henceforth, we will miss the updates while performing DB copy
operations.
For consuming the messages, I am considering to use Debizium SQL Server
Connector
<https://camel.apache.org/components/latest/debezium-sqlserver-component.html>,
but this component
*supports only the consumer functionality*
Which camel component can I use to produce the same event to the
destination DB ??
-Srikant
Re: How to produce messages to Azure SQL DB in Camel
Posted by SRIKANT MVS <sr...@gmail.com>.
Hi,
Does these components have HA by default or do I need to use the camel
clustering to achieve it ?
camel debezium sql server connector
camel-jdbc
camel-sql
-Regards
Srikant Mantha
On Mon, Dec 14, 2020 at 10:22 AM Omar Al-Safi <om...@oalsafi.com> wrote:
> Hi,
>
> As mentioned here
> <https://camel.apache.org/components/latest/sql-component.html>, both use
> JDBC driver meanwhile Camel SQL provides better support for SQL queries in
> the endpoint. Personally, it is more what you'd feel more comfortable with,
> try both and see what component that can fit your needs better.
>
> Regards,
> Omar
>
> On Fri, Dec 11, 2020 at 6:58 PM SRIKANT MVS <sr...@gmail.com> wrote:
>
> > Thanks for your reply, how better is it to use the Camel SQL compared to
> > Camel JDBC.
> >
> > -Regards
> > Srikant
> >
> > On Thu, 10 Dec, 2020, 10:53 Omar Al-Safi, <om...@oalsafi.com> wrote:
> >
> > > Hi,
> > >
> > > To write your events to your target Azure SQL server database, you can
> > > use Camel
> > > JDBC <https://camel.apache.org/components/latest/jdbc-component.html>
> > > component since Azure SQL supports JDBC uses JDBC driver to connect to
> > the
> > > database.
> > > Plus Camel Debezium SQL Server, I'd recommend using something like
> Azure
> > > Eventhub as event messaging medium in order to retain the events for
> some
> > > period of time in case of failures .. etc, hence your pipeline may look
> > > like this:
> > > Azure SQL DB (live) -> Camel Debezium SQL Server -> Azure Eventhub ->
> > Camel
> > > JDBC -> Azure SQL DB (replication).
> > >
> > > There are few examples that can help you to get started:
> > >
> > > - Debezium with AWS Kinesis and Cassandra Example
> > > <
> > >
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium
> > > >
> > > .
> > > - Debezium with Azure Eventhubs with Azure Storage Blob Example.
> > > <
> > >
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium-eventhubs-blob
> > > >
> > > - Camel JDBC Example
> > > <
> > >
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-jdbc
> > > >
> > > .
> > >
> > > I hope it helps!
> > >
> > > Regards,
> > > Omar
> > >
> > > On Thu, Dec 10, 2020 at 8:47 AM SRIKANT MVS <sr...@gmail.com>
> > wrote:
> > >
> > > > Team,
> > > > I have a usecase where I need to replicate data for a specific
> duration
> > > > from one Azure SQL DB (subscription1) to another Azure SQL DB
> > > > (supbscription2).
> > > >
> > > > The source DB has live data with continuous inserts, updates and
> > deletes
> > > > possibility. Henceforth, we will miss the updates while performing DB
> > > copy
> > > > operations.
> > > >
> > > > For consuming the messages, I am considering to use Debizium SQL
> Server
> > > > Connector
> > > > <
> > > >
> > >
> >
> https://camel.apache.org/components/latest/debezium-sqlserver-component.html
> > > > >,
> > > > but this component
> > > >
> > > > *supports only the consumer functionality*
> > > >
> > > > Which camel component can I use to produce the same event to the
> > > > destination DB ??
> > > >
> > > > -Srikant
> > > >
> > >
> >
>
Re: How to produce messages to Azure SQL DB in Camel
Posted by Omar Al-Safi <om...@oalsafi.com>.
Hi,
As mentioned here
<https://camel.apache.org/components/latest/sql-component.html>, both use
JDBC driver meanwhile Camel SQL provides better support for SQL queries in
the endpoint. Personally, it is more what you'd feel more comfortable with,
try both and see what component that can fit your needs better.
Regards,
Omar
On Fri, Dec 11, 2020 at 6:58 PM SRIKANT MVS <sr...@gmail.com> wrote:
> Thanks for your reply, how better is it to use the Camel SQL compared to
> Camel JDBC.
>
> -Regards
> Srikant
>
> On Thu, 10 Dec, 2020, 10:53 Omar Al-Safi, <om...@oalsafi.com> wrote:
>
> > Hi,
> >
> > To write your events to your target Azure SQL server database, you can
> > use Camel
> > JDBC <https://camel.apache.org/components/latest/jdbc-component.html>
> > component since Azure SQL supports JDBC uses JDBC driver to connect to
> the
> > database.
> > Plus Camel Debezium SQL Server, I'd recommend using something like Azure
> > Eventhub as event messaging medium in order to retain the events for some
> > period of time in case of failures .. etc, hence your pipeline may look
> > like this:
> > Azure SQL DB (live) -> Camel Debezium SQL Server -> Azure Eventhub ->
> Camel
> > JDBC -> Azure SQL DB (replication).
> >
> > There are few examples that can help you to get started:
> >
> > - Debezium with AWS Kinesis and Cassandra Example
> > <
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium
> > >
> > .
> > - Debezium with Azure Eventhubs with Azure Storage Blob Example.
> > <
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium-eventhubs-blob
> > >
> > - Camel JDBC Example
> > <
> >
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-jdbc
> > >
> > .
> >
> > I hope it helps!
> >
> > Regards,
> > Omar
> >
> > On Thu, Dec 10, 2020 at 8:47 AM SRIKANT MVS <sr...@gmail.com>
> wrote:
> >
> > > Team,
> > > I have a usecase where I need to replicate data for a specific duration
> > > from one Azure SQL DB (subscription1) to another Azure SQL DB
> > > (supbscription2).
> > >
> > > The source DB has live data with continuous inserts, updates and
> deletes
> > > possibility. Henceforth, we will miss the updates while performing DB
> > copy
> > > operations.
> > >
> > > For consuming the messages, I am considering to use Debizium SQL Server
> > > Connector
> > > <
> > >
> >
> https://camel.apache.org/components/latest/debezium-sqlserver-component.html
> > > >,
> > > but this component
> > >
> > > *supports only the consumer functionality*
> > >
> > > Which camel component can I use to produce the same event to the
> > > destination DB ??
> > >
> > > -Srikant
> > >
> >
>
Re: How to produce messages to Azure SQL DB in Camel
Posted by SRIKANT MVS <sr...@gmail.com>.
Thanks for your reply, how better is it to use the Camel SQL compared to
Camel JDBC.
-Regards
Srikant
On Thu, 10 Dec, 2020, 10:53 Omar Al-Safi, <om...@oalsafi.com> wrote:
> Hi,
>
> To write your events to your target Azure SQL server database, you can
> use Camel
> JDBC <https://camel.apache.org/components/latest/jdbc-component.html>
> component since Azure SQL supports JDBC uses JDBC driver to connect to the
> database.
> Plus Camel Debezium SQL Server, I'd recommend using something like Azure
> Eventhub as event messaging medium in order to retain the events for some
> period of time in case of failures .. etc, hence your pipeline may look
> like this:
> Azure SQL DB (live) -> Camel Debezium SQL Server -> Azure Eventhub -> Camel
> JDBC -> Azure SQL DB (replication).
>
> There are few examples that can help you to get started:
>
> - Debezium with AWS Kinesis and Cassandra Example
> <
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium
> >
> .
> - Debezium with Azure Eventhubs with Azure Storage Blob Example.
> <
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium-eventhubs-blob
> >
> - Camel JDBC Example
> <
> https://github.com/apache/camel-examples/tree/master/examples/camel-example-jdbc
> >
> .
>
> I hope it helps!
>
> Regards,
> Omar
>
> On Thu, Dec 10, 2020 at 8:47 AM SRIKANT MVS <sr...@gmail.com> wrote:
>
> > Team,
> > I have a usecase where I need to replicate data for a specific duration
> > from one Azure SQL DB (subscription1) to another Azure SQL DB
> > (supbscription2).
> >
> > The source DB has live data with continuous inserts, updates and deletes
> > possibility. Henceforth, we will miss the updates while performing DB
> copy
> > operations.
> >
> > For consuming the messages, I am considering to use Debizium SQL Server
> > Connector
> > <
> >
> https://camel.apache.org/components/latest/debezium-sqlserver-component.html
> > >,
> > but this component
> >
> > *supports only the consumer functionality*
> >
> > Which camel component can I use to produce the same event to the
> > destination DB ??
> >
> > -Srikant
> >
>
Re: How to produce messages to Azure SQL DB in Camel
Posted by Omar Al-Safi <om...@oalsafi.com>.
Hi,
To write your events to your target Azure SQL server database, you can
use Camel
JDBC <https://camel.apache.org/components/latest/jdbc-component.html>
component since Azure SQL supports JDBC uses JDBC driver to connect to the
database.
Plus Camel Debezium SQL Server, I'd recommend using something like Azure
Eventhub as event messaging medium in order to retain the events for some
period of time in case of failures .. etc, hence your pipeline may look
like this:
Azure SQL DB (live) -> Camel Debezium SQL Server -> Azure Eventhub -> Camel
JDBC -> Azure SQL DB (replication).
There are few examples that can help you to get started:
- Debezium with AWS Kinesis and Cassandra Example
<https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium>
.
- Debezium with Azure Eventhubs with Azure Storage Blob Example.
<https://github.com/apache/camel-examples/tree/master/examples/camel-example-debezium-eventhubs-blob>
- Camel JDBC Example
<https://github.com/apache/camel-examples/tree/master/examples/camel-example-jdbc>
.
I hope it helps!
Regards,
Omar
On Thu, Dec 10, 2020 at 8:47 AM SRIKANT MVS <sr...@gmail.com> wrote:
> Team,
> I have a usecase where I need to replicate data for a specific duration
> from one Azure SQL DB (subscription1) to another Azure SQL DB
> (supbscription2).
>
> The source DB has live data with continuous inserts, updates and deletes
> possibility. Henceforth, we will miss the updates while performing DB copy
> operations.
>
> For consuming the messages, I am considering to use Debizium SQL Server
> Connector
> <
> https://camel.apache.org/components/latest/debezium-sqlserver-component.html
> >,
> but this component
>
> *supports only the consumer functionality*
>
> Which camel component can I use to produce the same event to the
> destination DB ??
>
> -Srikant
>