You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Edoardo Causarano <ed...@gmail.com> on 2016/10/27 12:03:08 UTC
Losing mi sanity on pipeline testing
Hi all,
I’m trying to do something relatively simple, yet it’s been eluding me for hours already!
I’d like to have a timer trigger a broadcast to a couple pipelines to invoke a remote cloud component, transform the responses to a common type and merge the results in one homogenous list to further process.
It’s been a nightmare so far and even testing is proving to be nigh impossible, even this simple setup is failing with a :
@Override
protected RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
@Override
public void configure() throws Exception {
from("direct:start").pipeline()
.to("dataset:source")
.to("dropBoxTranslator")
.end()
.to("mock:result");
}
};
}
Seems that the dataset endpoint tries to assert that its’ own output equal the dummy string I sent from the direct:start producer to execute an exchange (in org.apache.camel.component.dataset.DataSetSupport#assertMessageExpected). What on earth is going on?!
Best,
Edoardo
Re: Losing mi sanity on pipeline testing
Posted by Claus Ibsen <cl...@gmail.com>.
I suggest to read the testing page and start tests with only mock
endpoints first.
Look at some of the examples and how they may have unit tests
And consider buy one of the Camel books that covers Camel in much better details
On Thu, Oct 27, 2016 at 2:03 PM, Edoardo Causarano
<ed...@gmail.com> wrote:
> Hi all,
>
> I’m trying to do something relatively simple, yet it’s been eluding me for hours already!
>
> I’d like to have a timer trigger a broadcast to a couple pipelines to invoke a remote cloud component, transform the responses to a common type and merge the results in one homogenous list to further process.
>
> It’s been a nightmare so far and even testing is proving to be nigh impossible, even this simple setup is failing with a :
>
> @Override
> protected RouteBuilder createRouteBuilder() {
> return new RouteBuilder() {
> @Override
> public void configure() throws Exception {
> from("direct:start").pipeline()
> .to("dataset:source")
> .to("dropBoxTranslator")
> .end()
> .to("mock:result");
> }
> };
> }
>
> Seems that the dataset endpoint tries to assert that its’ own output equal the dummy string I sent from the direct:start producer to execute an exchange (in org.apache.camel.component.dataset.DataSetSupport#assertMessageExpected). What on earth is going on?!
>
>
> Best,
> Edoardo
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Losing mi sanity on pipeline testing
Posted by Quinn Stevenson <qu...@pronoia-solutions.com>.
I don’t think DataSet is going to do what you’re after.
You could try a mock endpoint, and use returnReplyBody(…) to specify the response - that may do what you’re after
> On Oct 28, 2016, at 1:18 AM, Edoardo Causarano <ed...@gmail.com> wrote:
>
> Hi Quinn,
>
> Thanks that's correct, but really I was looking for a way to test the
> construct timer+broadcast+pipelines+merged_processor.
>
> Shoving the datasets to fake the remotes inside the pipelines made sense.
>
> Anyway, I'll see if I have time to get back to this setup sometime later.
> Right now I'm working along with a different kludge :/
>
>
> Edoardo
Re: Losing mi sanity on pipeline testing
Posted by Edoardo Causarano <ed...@gmail.com>.
Hi Quinn,
Thanks that's correct, but really I was looking for a way to test the
construct timer+broadcast+pipelines+merged_processor.
Shoving the datasets to fake the remotes inside the pipelines made sense.
Anyway, I'll see if I have time to get back to this setup sometime later.
Right now I'm working along with a different kludge :/
Edoardo
Re: Losing mi sanity on pipeline testing
Posted by Quinn Stevenson <qu...@pronoia-solutions.com>.
I’m assuming you’re just trying to get the DataSet to work for testing - correct? (DataSet isn’t really meant for anything other than testing).
If that’s the case, use the DataSet as the Consumer - i.e. from(“dataset://source”) and you should be OK.
> On Oct 27, 2016, at 7:15 AM, Edoardo Causarano <ed...@gmail.com> wrote:
>
> Hi,
>
> answers inline:
>
>> On 27 Oct 2016, at 14:50, Brad Johnson <br...@mediadriver.com> wrote:
>>
>> Try putting in .log(${body}) in between each of the lines and see what you
>> get. It appears you are using the SimpleDataSet which takes a single
>> String. What behavior are you expecting from ti? Is it throwing an
>> exception? Have you tried sending in a List<String> with a ListDataSet to
>> see what behavior it gives you?
>
> I’m expecting the SimpleDataSet to provide 1 message body consisting of a List<DbxEntry> (a list of items from a DropBox query.)
> It’s throwing when trying to assert that the body correspond to the data provided by the direct producer template (I only figured that out after step debugging, is this the intended behavior?)
>
>> Is there a specific reason you are using a .pipeline() call explicitly? I
>> believe that's the default behavior. The only reason I can think of
>> explicitly using it is when you just want to use it as a short hand for
>> routing as shown in the Camel documentation. You don't have to chain a
>> bunch of to() method calls then.
>
> Because I’d like to broadcast a timer trigger to a couple pipelines consisting of remote API getters and transformers. Otherwise all stages - including the transformers - would be directly wired to the timer and that would make no sense.
>
> timer --> pipeline(dropbox, transformer) --> doStuff
> |-> pipeline(google, transformer) --|
>
>
> Best,
> Edoardo
>
>> from("direct:a").pipeline("direct:x", "direct:y", "direct:z",
>> "mock:result");
>>
>> On Thu, Oct 27, 2016 at 7:03 AM, Edoardo Causarano <
>> edoardo.causarano@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I’m trying to do something relatively simple, yet it’s been eluding me for
>>> hours already!
>>>
>>> I’d like to have a timer trigger a broadcast to a couple pipelines to
>>> invoke a remote cloud component, transform the responses to a common type
>>> and merge the results in one homogenous list to further process.
>>>
>>> It’s been a nightmare so far and even testing is proving to be nigh
>>> impossible, even this simple setup is failing with a :
>>>
>>> @Override
>>> protected RouteBuilder createRouteBuilder() {
>>> return new RouteBuilder() {
>>> @Override
>>> public void configure() throws Exception {
>>> from("direct:start").pipeline()
>>> .to("dataset:source")
>>> .to("dropBoxTranslator")
>>> .end()
>>> .to("mock:result");
>>> }
>>> };
>>> }
>>>
>>> Seems that the dataset endpoint tries to assert that its’ own output equal
>>> the dummy string I sent from the direct:start producer to execute an
>>> exchange (in org.apache.camel.component.dataset.DataSetSupport#assertMessageExpected).
>>> What on earth is going on?!
>>>
>>>
>>> Best,
>>> Edoardo
>
Re: Losing mi sanity on pipeline testing
Posted by Edoardo Causarano <ed...@gmail.com>.
Hi,
answers inline:
> On 27 Oct 2016, at 14:50, Brad Johnson <br...@mediadriver.com> wrote:
>
> Try putting in .log(${body}) in between each of the lines and see what you
> get. It appears you are using the SimpleDataSet which takes a single
> String. What behavior are you expecting from ti? Is it throwing an
> exception? Have you tried sending in a List<String> with a ListDataSet to
> see what behavior it gives you?
I’m expecting the SimpleDataSet to provide 1 message body consisting of a List<DbxEntry> (a list of items from a DropBox query.)
It’s throwing when trying to assert that the body correspond to the data provided by the direct producer template (I only figured that out after step debugging, is this the intended behavior?)
> Is there a specific reason you are using a .pipeline() call explicitly? I
> believe that's the default behavior. The only reason I can think of
> explicitly using it is when you just want to use it as a short hand for
> routing as shown in the Camel documentation. You don't have to chain a
> bunch of to() method calls then.
Because I’d like to broadcast a timer trigger to a couple pipelines consisting of remote API getters and transformers. Otherwise all stages - including the transformers - would be directly wired to the timer and that would make no sense.
timer --> pipeline(dropbox, transformer) --> doStuff
|-> pipeline(google, transformer) --|
Best,
Edoardo
> from("direct:a").pipeline("direct:x", "direct:y", "direct:z",
> "mock:result");
>
> On Thu, Oct 27, 2016 at 7:03 AM, Edoardo Causarano <
> edoardo.causarano@gmail.com> wrote:
>
>> Hi all,
>>
>> I’m trying to do something relatively simple, yet it’s been eluding me for
>> hours already!
>>
>> I’d like to have a timer trigger a broadcast to a couple pipelines to
>> invoke a remote cloud component, transform the responses to a common type
>> and merge the results in one homogenous list to further process.
>>
>> It’s been a nightmare so far and even testing is proving to be nigh
>> impossible, even this simple setup is failing with a :
>>
>> @Override
>> protected RouteBuilder createRouteBuilder() {
>> return new RouteBuilder() {
>> @Override
>> public void configure() throws Exception {
>> from("direct:start").pipeline()
>> .to("dataset:source")
>> .to("dropBoxTranslator")
>> .end()
>> .to("mock:result");
>> }
>> };
>> }
>>
>> Seems that the dataset endpoint tries to assert that its’ own output equal
>> the dummy string I sent from the direct:start producer to execute an
>> exchange (in org.apache.camel.component.dataset.DataSetSupport#assertMessageExpected).
>> What on earth is going on?!
>>
>>
>> Best,
>> Edoardo
Re: Losing mi sanity on pipeline testing
Posted by Brad Johnson <br...@mediadriver.com>.
Try putting in .log(${body}) in between each of the lines and see what you
get. It appears you are using the SimpleDataSet which takes a single
String. What behavior are you expecting from ti? Is it throwing an
exception? Have you tried sending in a List<String> with a ListDataSet to
see what behavior it gives you?
Is there a specific reason you are using a .pipeline() call explicitly? I
believe that's the default behavior. The only reason I can think of
explicitly using it is when you just want to use it as a short hand for
routing as shown in the Camel documentation. You don't have to chain a
bunch of to() method calls then.
from("direct:a").pipeline("direct:x", "direct:y", "direct:z",
"mock:result");
On Thu, Oct 27, 2016 at 7:03 AM, Edoardo Causarano <
edoardo.causarano@gmail.com> wrote:
> Hi all,
>
> I’m trying to do something relatively simple, yet it’s been eluding me for
> hours already!
>
> I’d like to have a timer trigger a broadcast to a couple pipelines to
> invoke a remote cloud component, transform the responses to a common type
> and merge the results in one homogenous list to further process.
>
> It’s been a nightmare so far and even testing is proving to be nigh
> impossible, even this simple setup is failing with a :
>
> @Override
> protected RouteBuilder createRouteBuilder() {
> return new RouteBuilder() {
> @Override
> public void configure() throws Exception {
> from("direct:start").pipeline()
> .to("dataset:source")
> .to("dropBoxTranslator")
> .end()
> .to("mock:result");
> }
> };
> }
>
> Seems that the dataset endpoint tries to assert that its’ own output equal
> the dummy string I sent from the direct:start producer to execute an
> exchange (in org.apache.camel.component.dataset.DataSetSupport#assertMessageExpected).
> What on earth is going on?!
>
>
> Best,
> Edoardo