RE: onCompletion issue on SQL Endpoint. [I]

[Sunil Keyal <su...@db.com>]

Classification: For internal use only

Thanks for responding. I have issues with both 2.20.1 and 2.20.2

I narrowed down the issue to marshal()..... These are few scenarios.

1. My SELECT_QUERY_ENDPOINT takes more than a minute to return the resultset from oracle, keep marshal(csv) on the route,
         onCompletion is not triggered.
2. My SELECT_QUERY_ENDPOINT takes more than a minute to return the resultset from oracle, remove marshal(csv) on the route
        onCompletion is triggered
3. My SELECT_QUERY_ENDPOINT takes less than a minute to return the resultset from oracle, keep or remove marshal(csv) on the route
        onCompletion is triggered.


Even if the onCompletion is not triggered, the final generated CSV file does have all the records from the SQL and does send the file to sftp endpoint. It is just that on scenario number 1 above, the onCompletion is not triggered.

This is the simplified version of my route
        CsvDataFormat csv = new CsvDataFormat();

        from("direct:my_route")
                .routeId("my_route")
                .log("Starting Job my route")
                .onCompletion()
                        .log("************** Inside onComplete")
                .end()
                .to(SELECT_ENDPOINT)
                .marshal(csv)
                .log("my job has completed successfully")
                . to(sftp:xxxxxxxxx)
                .end();



Kind regards,
Sunil Keyal

-----Original Message-----
From: Claus Ibsen [mailto:claus.ibsen@gmail.com]
Sent: Sunday, February 18, 2018 8:25 AM
To: users@camel.apache.org
Subject: Re: onCompletion issue on SQL Endpoint.

Hi

What version of Camel do you use. And maybe try adding a

.log("After SQL")

or turn on tracing
http://camel.apache.org/tracer.html

To see if the SQL endpoint is finished after that 1 minute or what happens.
Also if a SQL takes longer than a minute it sounds as if either the SQL is complex and still executing on the database, or the SQL returns a huge dataset and that SQL/Camel is reading in too much data into the JVM so the JVM is under stress / low on memory.

You can try to execute that SQL in some SQL tool to see how many rows it would return.
And if its huge number, then you should work on limiting the number of rows returned or use the streamList return type, and then use a splitter or something to work on the list in streaming on-demand mode, but that is another conversation.

The on completion ought to be triggered no matter what when the exchange is done.


On Fri, Feb 16, 2018 at 8:44 PM, Sunil Keyal <su...@db.com> wrote:
> Classification: Public
>
> It seems it is bindy...
>
> If I remove   .marshal(csvBindingForSsiExtractReport) from the route, then even after a minute of execution, I see onCompletion being called.
>
> Does anyone know how to fix this?
>
>
> Kind regards,
> Sunil Keyal
>
> -----Original Message-----
> From: Sunil Keyal [mailto:sunil.keyal+external@db.com]
> Sent: Friday, February 16, 2018 2:25 PM
> To: users@camel.apache.org
> Subject: onCompletion issue on SQL Endpoint.
>
> Classification: Public
> I have a simple route which gets data from database using SQL endpoint and which supposed to do something onCompletion. If the SQL takes more than a minute, the onCompletion is never called, I see it working if SQL takes less than a minute. I am using HikariCP as JDBC connection pool.
>
> What is wrong? Is there any default setting of a minute for onCompletion, somewhere?
>
> Your help will be appreciated.
>
> ***************************
>
>
> private static final String SELECT_ROUTE_ENDPOINT =
>         "sql:classpath:sql/select.sql" +
>         "?dataSource=fesrDataSource&outputType=SelectList&" +
>         "outputClass=com.test.MyReport";
>
>
> from("direct:myroute")
>         .onCompletion()
>             .log("************** Inside onComplete")
>         .end()
>         .log("Starting Job ")
>         .to(SELECT_SSI_EXTRACT_REPORT_ROUTE_ENDPOINT)
>         .marshal(csvBindingForSsiExtractReport)
>         .log("Route has completed successfully")
>         .end();
>
>
> Kind regards,
> Sunil
>
>
>
>
> ---
> This communication may contain confidential and/or privileged information. If you are not the intended recipient (or have received this communication in error) please notify the sender immediately and destroy this communication. Any unauthorized copying, disclosure or distribution of the material in this communication is strictly forbidden.
>
> Please refer to https://db.com/disclosures for additional EU corporate and regulatory disclosures.
>
> Deutsche Bank does not render legal or tax advice, and the information contained in this communication should not be regarded as such.
>
>
> ---
> This communication may contain confidential and/or privileged information. If you are not the intended recipient (or have received this communication in error) please notify the sender immediately and destroy this communication. Any unauthorized copying, disclosure or distribution of the material in this communication is strictly forbidden.
>
> Please refer to https://db.com/disclosures for additional EU corporate and regulatory disclosures.
>
> Deutsche Bank does not render legal or tax advice, and the information contained in this communication should not be regarded as such.



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2


---
This communication may contain confidential and/or privileged information. If you are not the intended recipient (or have received this communication in error) please notify the sender immediately and destroy this communication. Any unauthorized copying, disclosure or distribution of the material in this communication is strictly forbidden.

Please refer to https://db.com/disclosures for additional EU corporate and regulatory disclosures.

Deutsche Bank does not render legal or tax advice, and the information contained in this communication should not be regarded as such.