You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Oliver Geisser <ol...@gmail.com> on 2012/11/29 23:27:34 UTC
Splitter and combining messages / exchanges
Hello,
I am using a splitter to handle a big txt file (streaming mode). I need to
combine the first line
with all the other lines and store each result into a database.
What is a good Camel solution for this?
Example input file:
Hello
A
B
C
...
This needs to be combined into:
Hello A
Hello B
Hello C
...
Every result is stored into a database.
It's important to use streaming because of the size of the input file.
My first idea was to use a Splitter and then store the first line into an
Exchange property.
But this does not work because the Splitter creates new Exchanges for every
line.
So how do I combine the first line with all the other lines?
Thanks
Oliver
--
og
Re: Splitter and combining messages / exchanges
Posted by Jason Chaffee <jc...@shopzilla.com>.
What about using the stream component instead of a splitter? You could
then route to a bean and keep the first line in ThreadLocal or something
like that, and do the aggregation in the bean. You could not doing any
asynchronous processing in this case do to using the ThreadLocal.
This might not be the optimal solution. Claus might have some other
solutions using the EIPs, but I think this would work nonetheless.
Jason
On 11/30/12 3:13 PM, "Oliver Geisser" <ol...@gmail.com> wrote:
>Hi Jason,
>
>that was my first idea, too. But this does not work.
>Because I need the streaming mode (big file) I need to "release"
>every line from from the Aggregator immediately, e.g. the "completion
>predicate" will be true for all lines but the first.
>
>And when an aggregated Exchange is "completed" it is "gone"
>and with it I will also loose the header.
>
>Do you have another idea?
>
>Greetings
>Oliver
>
>
>
>2012/11/30 Jason Chaffee <jc...@shopzilla.com>
>
>> I believe could write a custom aggregator. On the first call, you put
>>the
>> "Hello" into a header on the Exchange and return that Exchange. On the
>> rest of the calls, you can read the header and aggregate it with the
>> current body.
>>
>>
>> On 11/30/12 2:33 PM, "Oliver Geisser" <ol...@gmail.com> wrote:
>>
>> >Hi Claus,
>> >
>> >thanks for your help. I've checked the Composed Message Processor EIP.
>> >
>> >Maybe I'm missing something but as far as I can see the EIP will not
>>work
>> >for my use case.
>> >
>> >Let's recap my problem with the input file
>> >
>> >Hello
>> >A
>> >B
>> >C
>> >...
>> >
>> >Because it's a very large file it will be splitted in streaming mode.
>> >Therfor
>> >every line becomes a separate Message.
>> >
>> >Using the Aggregator the "Hello" Message will be stored in the
>> >AggregationRepository.
>> >The the second message ("A") will be aggregated with "Hello" to "Hello
>>A".
>> >And now the important point: the "completion predicate" is now true and
>> >the
>> >meesage is released from the Aggregator, e.g.
>> >AggregateProcessor.onCompletion will
>> >be called and the "Hello" Exchange is removed from the
>> >AggregationRepository.
>> >
>> >The Exchange needs to be released (completed) at this point because it
>>is
>> >a
>> >big file and
>> >we need to operate in streaming mode.
>> >
>> >Now the third message ("B") comes in. Because the "Hello" Exchange was
>> >removed
>> >we can not aggregate the Exchanges to "Hello B" anymore.
>> >
>> >So the question remains: what is the best way to solve the given
>>problem
>> >(see below) with Camel?
>> >
>> >One idea I had was to mimic the Aggregate Repository but without
>>removing
>> >the "Hello" Exchange
>> >after combining it with the "A", "B", "C", etc. messages.
>> >
>> >But is this really the best solution for my problem?
>> >
>> >What I need is access to a storage location which is scoped to the
>> >"Splitter input Exchange"
>> >during processing the "splitted" Exchanges. For example: if it would be
>> >possible to access
>> >the "input" Exchange during processing the "splitted" Exchanges I would
>> >set
>> >a property
>> >on the "input" Exchange during processing the first line and reading it
>> >afterwards during
>> >processing the remaining lines.
>> >
>> >Is there any other storage location besides an Exchange property which
>>is
>> >scoped to the
>> >Exchange and is accessible from the "splitted" Exchanges?
>> >
>> >Greetings
>> >Oliver
>> >
>> >
>> >
>> >2012/11/30 Claus Ibsen <cl...@gmail.com>
>> >
>> >> Hi
>> >>
>> >> See this EIP
>> >> http://camel.apache.org/composed-message-processor.html
>> >>
>> >> On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
>> >> <ol...@gmail.com> wrote:
>> >> > Hello,
>> >> >
>> >> > I am using a splitter to handle a big txt file (streaming mode). I
>> >>need
>> >> to
>> >> > combine the first line
>> >> > with all the other lines and store each result into a database.
>> >> >
>> >> > What is a good Camel solution for this?
>> >> >
>> >> > Example input file:
>> >> >
>> >> > Hello
>> >> > A
>> >> > B
>> >> > C
>> >> > ...
>> >> >
>> >> >
>> >> > This needs to be combined into:
>> >> >
>> >> > Hello A
>> >> > Hello B
>> >> > Hello C
>> >> > ...
>> >> >
>> >> > Every result is stored into a database.
>> >> > It's important to use streaming because of the size of the input
>>file.
>> >> >
>> >> >
>> >> > My first idea was to use a Splitter and then store the first line
>> >>into an
>> >> > Exchange property.
>> >> > But this does not work because the Splitter creates new Exchanges
>>for
>> >> every
>> >> > line.
>> >> >
>> >> > So how do I combine the first line with all the other lines?
>> >> >
>> >> > Thanks
>> >> > Oliver
>> >> >
>> >> > --
>> >> > og
>> >>
>> >>
>> >>
>> >> --
>> >> Claus Ibsen
>> >> -----------------
>> >> Red Hat, Inc.
>> >> FuseSource is now part of Red Hat
>> >> Email: cibsen@redhat.com
>> >> Web: http://fusesource.com
>> >> Twitter: davsclaus
>> >> Blog: http://davsclaus.com
>> >> Author of Camel in Action: http://www.manning.com/ibsen
>> >>
>> >
>> >
>> >
>> >--
>> >og
>>
>>
>
>
>--
>og
Re: Splitter and combining messages / exchanges
Posted by Oliver Geisser <ol...@gmail.com>.
Hi Jason,
that was my first idea, too. But this does not work.
Because I need the streaming mode (big file) I need to "release"
every line from from the Aggregator immediately, e.g. the "completion
predicate" will be true for all lines but the first.
And when an aggregated Exchange is "completed" it is "gone"
and with it I will also loose the header.
Do you have another idea?
Greetings
Oliver
2012/11/30 Jason Chaffee <jc...@shopzilla.com>
> I believe could write a custom aggregator. On the first call, you put the
> "Hello" into a header on the Exchange and return that Exchange. On the
> rest of the calls, you can read the header and aggregate it with the
> current body.
>
>
> On 11/30/12 2:33 PM, "Oliver Geisser" <ol...@gmail.com> wrote:
>
> >Hi Claus,
> >
> >thanks for your help. I've checked the Composed Message Processor EIP.
> >
> >Maybe I'm missing something but as far as I can see the EIP will not work
> >for my use case.
> >
> >Let's recap my problem with the input file
> >
> >Hello
> >A
> >B
> >C
> >...
> >
> >Because it's a very large file it will be splitted in streaming mode.
> >Therfor
> >every line becomes a separate Message.
> >
> >Using the Aggregator the "Hello" Message will be stored in the
> >AggregationRepository.
> >The the second message ("A") will be aggregated with "Hello" to "Hello A".
> >And now the important point: the "completion predicate" is now true and
> >the
> >meesage is released from the Aggregator, e.g.
> >AggregateProcessor.onCompletion will
> >be called and the "Hello" Exchange is removed from the
> >AggregationRepository.
> >
> >The Exchange needs to be released (completed) at this point because it is
> >a
> >big file and
> >we need to operate in streaming mode.
> >
> >Now the third message ("B") comes in. Because the "Hello" Exchange was
> >removed
> >we can not aggregate the Exchanges to "Hello B" anymore.
> >
> >So the question remains: what is the best way to solve the given problem
> >(see below) with Camel?
> >
> >One idea I had was to mimic the Aggregate Repository but without removing
> >the "Hello" Exchange
> >after combining it with the "A", "B", "C", etc. messages.
> >
> >But is this really the best solution for my problem?
> >
> >What I need is access to a storage location which is scoped to the
> >"Splitter input Exchange"
> >during processing the "splitted" Exchanges. For example: if it would be
> >possible to access
> >the "input" Exchange during processing the "splitted" Exchanges I would
> >set
> >a property
> >on the "input" Exchange during processing the first line and reading it
> >afterwards during
> >processing the remaining lines.
> >
> >Is there any other storage location besides an Exchange property which is
> >scoped to the
> >Exchange and is accessible from the "splitted" Exchanges?
> >
> >Greetings
> >Oliver
> >
> >
> >
> >2012/11/30 Claus Ibsen <cl...@gmail.com>
> >
> >> Hi
> >>
> >> See this EIP
> >> http://camel.apache.org/composed-message-processor.html
> >>
> >> On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
> >> <ol...@gmail.com> wrote:
> >> > Hello,
> >> >
> >> > I am using a splitter to handle a big txt file (streaming mode). I
> >>need
> >> to
> >> > combine the first line
> >> > with all the other lines and store each result into a database.
> >> >
> >> > What is a good Camel solution for this?
> >> >
> >> > Example input file:
> >> >
> >> > Hello
> >> > A
> >> > B
> >> > C
> >> > ...
> >> >
> >> >
> >> > This needs to be combined into:
> >> >
> >> > Hello A
> >> > Hello B
> >> > Hello C
> >> > ...
> >> >
> >> > Every result is stored into a database.
> >> > It's important to use streaming because of the size of the input file.
> >> >
> >> >
> >> > My first idea was to use a Splitter and then store the first line
> >>into an
> >> > Exchange property.
> >> > But this does not work because the Splitter creates new Exchanges for
> >> every
> >> > line.
> >> >
> >> > So how do I combine the first line with all the other lines?
> >> >
> >> > Thanks
> >> > Oliver
> >> >
> >> > --
> >> > og
> >>
> >>
> >>
> >> --
> >> Claus Ibsen
> >> -----------------
> >> Red Hat, Inc.
> >> FuseSource is now part of Red Hat
> >> Email: cibsen@redhat.com
> >> Web: http://fusesource.com
> >> Twitter: davsclaus
> >> Blog: http://davsclaus.com
> >> Author of Camel in Action: http://www.manning.com/ibsen
> >>
> >
> >
> >
> >--
> >og
>
>
--
og
Re: Splitter and combining messages / exchanges
Posted by Oliver Geisser <ol...@gmail.com>.
Hi Raul,
thanks a lot. I think this will work best.
It solves the problem where to store the first line in the best way
(property)
and it also supports streaming mode.
I was misguided in my assumption that I can not access the message
before the splitter because of the size. But of course I can use plain old
Java to read just the first line.
Thanks again.
Oliver
2012/12/1 Raul Kripalani <ra...@evosent.com>
> I would suggest the following pipeline:
>
> - Before the splitter, read the first line and store it in a property
> - Run the splitter, and for every element concatenate the above property
> with the incoming line
> - Discard the first Exchange, by using a filter EIP to only allow messages
> whose header CamelSplitIndex is greater than 1 or 2 (can't remember if the
> index starts at 0 or 1).
>
> Does this approach help? Just to clarify, you wouldn't be using the
> aggregator EIP at all.
>
> Regards,
> Raúl.
>
> Sent from a mobile device
> On 30 Nov 2012 22:33, "Oliver Geisser" <ol...@gmail.com> wrote:
>
> > Hi Claus,
> >
> > thanks for your help. I've checked the Composed Message Processor EIP.
> >
> > Maybe I'm missing something but as far as I can see the EIP will not work
> > for my use case.
> >
> > Let's recap my problem with the input file
> >
> > Hello
> > A
> > B
> > C
> > ...
> >
> > Because it's a very large file it will be splitted in streaming mode.
> > Therfor
> > every line becomes a separate Message.
> >
> > Using the Aggregator the "Hello" Message will be stored in the
> > AggregationRepository.
> > The the second message ("A") will be aggregated with "Hello" to "Hello
> A".
> > And now the important point: the "completion predicate" is now true and
> the
> > meesage is released from the Aggregator, e.g.
> > AggregateProcessor.onCompletion will
> > be called and the "Hello" Exchange is removed from the
> > AggregationRepository.
> >
> > The Exchange needs to be released (completed) at this point because it
> is a
> > big file and
> > we need to operate in streaming mode.
> >
> > Now the third message ("B") comes in. Because the "Hello" Exchange was
> > removed
> > we can not aggregate the Exchanges to "Hello B" anymore.
> >
> > So the question remains: what is the best way to solve the given problem
> > (see below) with Camel?
> >
> > One idea I had was to mimic the Aggregate Repository but without removing
> > the "Hello" Exchange
> > after combining it with the "A", "B", "C", etc. messages.
> >
> > But is this really the best solution for my problem?
> >
> > What I need is access to a storage location which is scoped to the
> > "Splitter input Exchange"
> > during processing the "splitted" Exchanges. For example: if it would be
> > possible to access
> > the "input" Exchange during processing the "splitted" Exchanges I would
> set
> > a property
> > on the "input" Exchange during processing the first line and reading it
> > afterwards during
> > processing the remaining lines.
> >
> > Is there any other storage location besides an Exchange property which is
> > scoped to the
> > Exchange and is accessible from the "splitted" Exchanges?
> >
> > Greetings
> > Oliver
> >
> >
> >
> > 2012/11/30 Claus Ibsen <cl...@gmail.com>
> >
> > > Hi
> > >
> > > See this EIP
> > > http://camel.apache.org/composed-message-processor.html
> > >
> > > On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
> > > <ol...@gmail.com> wrote:
> > > > Hello,
> > > >
> > > > I am using a splitter to handle a big txt file (streaming mode). I
> need
> > > to
> > > > combine the first line
> > > > with all the other lines and store each result into a database.
> > > >
> > > > What is a good Camel solution for this?
> > > >
> > > > Example input file:
> > > >
> > > > Hello
> > > > A
> > > > B
> > > > C
> > > > ...
> > > >
> > > >
> > > > This needs to be combined into:
> > > >
> > > > Hello A
> > > > Hello B
> > > > Hello C
> > > > ...
> > > >
> > > > Every result is stored into a database.
> > > > It's important to use streaming because of the size of the input
> file.
> > > >
> > > >
> > > > My first idea was to use a Splitter and then store the first line
> into
> > an
> > > > Exchange property.
> > > > But this does not work because the Splitter creates new Exchanges for
> > > every
> > > > line.
> > > >
> > > > So how do I combine the first line with all the other lines?
> > > >
> > > > Thanks
> > > > Oliver
> > > >
> > > > --
> > > > og
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -----------------
> > > Red Hat, Inc.
> > > FuseSource is now part of Red Hat
> > > Email: cibsen@redhat.com
> > > Web: http://fusesource.com
> > > Twitter: davsclaus
> > > Blog: http://davsclaus.com
> > > Author of Camel in Action: http://www.manning.com/ibsen
> > >
> >
> >
> >
> > --
> > og
> >
>
--
og
Re: Splitter and combining messages / exchanges
Posted by Raul Kripalani <ra...@evosent.com>.
I would suggest the following pipeline:
- Before the splitter, read the first line and store it in a property
- Run the splitter, and for every element concatenate the above property
with the incoming line
- Discard the first Exchange, by using a filter EIP to only allow messages
whose header CamelSplitIndex is greater than 1 or 2 (can't remember if the
index starts at 0 or 1).
Does this approach help? Just to clarify, you wouldn't be using the
aggregator EIP at all.
Regards,
Raúl.
Sent from a mobile device
On 30 Nov 2012 22:33, "Oliver Geisser" <ol...@gmail.com> wrote:
> Hi Claus,
>
> thanks for your help. I've checked the Composed Message Processor EIP.
>
> Maybe I'm missing something but as far as I can see the EIP will not work
> for my use case.
>
> Let's recap my problem with the input file
>
> Hello
> A
> B
> C
> ...
>
> Because it's a very large file it will be splitted in streaming mode.
> Therfor
> every line becomes a separate Message.
>
> Using the Aggregator the "Hello" Message will be stored in the
> AggregationRepository.
> The the second message ("A") will be aggregated with "Hello" to "Hello A".
> And now the important point: the "completion predicate" is now true and the
> meesage is released from the Aggregator, e.g.
> AggregateProcessor.onCompletion will
> be called and the "Hello" Exchange is removed from the
> AggregationRepository.
>
> The Exchange needs to be released (completed) at this point because it is a
> big file and
> we need to operate in streaming mode.
>
> Now the third message ("B") comes in. Because the "Hello" Exchange was
> removed
> we can not aggregate the Exchanges to "Hello B" anymore.
>
> So the question remains: what is the best way to solve the given problem
> (see below) with Camel?
>
> One idea I had was to mimic the Aggregate Repository but without removing
> the "Hello" Exchange
> after combining it with the "A", "B", "C", etc. messages.
>
> But is this really the best solution for my problem?
>
> What I need is access to a storage location which is scoped to the
> "Splitter input Exchange"
> during processing the "splitted" Exchanges. For example: if it would be
> possible to access
> the "input" Exchange during processing the "splitted" Exchanges I would set
> a property
> on the "input" Exchange during processing the first line and reading it
> afterwards during
> processing the remaining lines.
>
> Is there any other storage location besides an Exchange property which is
> scoped to the
> Exchange and is accessible from the "splitted" Exchanges?
>
> Greetings
> Oliver
>
>
>
> 2012/11/30 Claus Ibsen <cl...@gmail.com>
>
> > Hi
> >
> > See this EIP
> > http://camel.apache.org/composed-message-processor.html
> >
> > On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
> > <ol...@gmail.com> wrote:
> > > Hello,
> > >
> > > I am using a splitter to handle a big txt file (streaming mode). I need
> > to
> > > combine the first line
> > > with all the other lines and store each result into a database.
> > >
> > > What is a good Camel solution for this?
> > >
> > > Example input file:
> > >
> > > Hello
> > > A
> > > B
> > > C
> > > ...
> > >
> > >
> > > This needs to be combined into:
> > >
> > > Hello A
> > > Hello B
> > > Hello C
> > > ...
> > >
> > > Every result is stored into a database.
> > > It's important to use streaming because of the size of the input file.
> > >
> > >
> > > My first idea was to use a Splitter and then store the first line into
> an
> > > Exchange property.
> > > But this does not work because the Splitter creates new Exchanges for
> > every
> > > line.
> > >
> > > So how do I combine the first line with all the other lines?
> > >
> > > Thanks
> > > Oliver
> > >
> > > --
> > > og
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > Red Hat, Inc.
> > FuseSource is now part of Red Hat
> > Email: cibsen@redhat.com
> > Web: http://fusesource.com
> > Twitter: davsclaus
> > Blog: http://davsclaus.com
> > Author of Camel in Action: http://www.manning.com/ibsen
> >
>
>
>
> --
> og
>
Re: Splitter and combining messages / exchanges
Posted by Jason Chaffee <jc...@shopzilla.com>.
I believe could write a custom aggregator. On the first call, you put the
"Hello" into a header on the Exchange and return that Exchange. On the
rest of the calls, you can read the header and aggregate it with the
current body.
On 11/30/12 2:33 PM, "Oliver Geisser" <ol...@gmail.com> wrote:
>Hi Claus,
>
>thanks for your help. I've checked the Composed Message Processor EIP.
>
>Maybe I'm missing something but as far as I can see the EIP will not work
>for my use case.
>
>Let's recap my problem with the input file
>
>Hello
>A
>B
>C
>...
>
>Because it's a very large file it will be splitted in streaming mode.
>Therfor
>every line becomes a separate Message.
>
>Using the Aggregator the "Hello" Message will be stored in the
>AggregationRepository.
>The the second message ("A") will be aggregated with "Hello" to "Hello A".
>And now the important point: the "completion predicate" is now true and
>the
>meesage is released from the Aggregator, e.g.
>AggregateProcessor.onCompletion will
>be called and the "Hello" Exchange is removed from the
>AggregationRepository.
>
>The Exchange needs to be released (completed) at this point because it is
>a
>big file and
>we need to operate in streaming mode.
>
>Now the third message ("B") comes in. Because the "Hello" Exchange was
>removed
>we can not aggregate the Exchanges to "Hello B" anymore.
>
>So the question remains: what is the best way to solve the given problem
>(see below) with Camel?
>
>One idea I had was to mimic the Aggregate Repository but without removing
>the "Hello" Exchange
>after combining it with the "A", "B", "C", etc. messages.
>
>But is this really the best solution for my problem?
>
>What I need is access to a storage location which is scoped to the
>"Splitter input Exchange"
>during processing the "splitted" Exchanges. For example: if it would be
>possible to access
>the "input" Exchange during processing the "splitted" Exchanges I would
>set
>a property
>on the "input" Exchange during processing the first line and reading it
>afterwards during
>processing the remaining lines.
>
>Is there any other storage location besides an Exchange property which is
>scoped to the
>Exchange and is accessible from the "splitted" Exchanges?
>
>Greetings
>Oliver
>
>
>
>2012/11/30 Claus Ibsen <cl...@gmail.com>
>
>> Hi
>>
>> See this EIP
>> http://camel.apache.org/composed-message-processor.html
>>
>> On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
>> <ol...@gmail.com> wrote:
>> > Hello,
>> >
>> > I am using a splitter to handle a big txt file (streaming mode). I
>>need
>> to
>> > combine the first line
>> > with all the other lines and store each result into a database.
>> >
>> > What is a good Camel solution for this?
>> >
>> > Example input file:
>> >
>> > Hello
>> > A
>> > B
>> > C
>> > ...
>> >
>> >
>> > This needs to be combined into:
>> >
>> > Hello A
>> > Hello B
>> > Hello C
>> > ...
>> >
>> > Every result is stored into a database.
>> > It's important to use streaming because of the size of the input file.
>> >
>> >
>> > My first idea was to use a Splitter and then store the first line
>>into an
>> > Exchange property.
>> > But this does not work because the Splitter creates new Exchanges for
>> every
>> > line.
>> >
>> > So how do I combine the first line with all the other lines?
>> >
>> > Thanks
>> > Oliver
>> >
>> > --
>> > og
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> Red Hat, Inc.
>> FuseSource is now part of Red Hat
>> Email: cibsen@redhat.com
>> Web: http://fusesource.com
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
>>
>
>
>
>--
>og
Re: Splitter and combining messages / exchanges
Posted by Oliver Geisser <ol...@gmail.com>.
Hi Claus,
thanks for your help. I've checked the Composed Message Processor EIP.
Maybe I'm missing something but as far as I can see the EIP will not work
for my use case.
Let's recap my problem with the input file
Hello
A
B
C
...
Because it's a very large file it will be splitted in streaming mode.
Therfor
every line becomes a separate Message.
Using the Aggregator the "Hello" Message will be stored in the
AggregationRepository.
The the second message ("A") will be aggregated with "Hello" to "Hello A".
And now the important point: the "completion predicate" is now true and the
meesage is released from the Aggregator, e.g.
AggregateProcessor.onCompletion will
be called and the "Hello" Exchange is removed from the
AggregationRepository.
The Exchange needs to be released (completed) at this point because it is a
big file and
we need to operate in streaming mode.
Now the third message ("B") comes in. Because the "Hello" Exchange was
removed
we can not aggregate the Exchanges to "Hello B" anymore.
So the question remains: what is the best way to solve the given problem
(see below) with Camel?
One idea I had was to mimic the Aggregate Repository but without removing
the "Hello" Exchange
after combining it with the "A", "B", "C", etc. messages.
But is this really the best solution for my problem?
What I need is access to a storage location which is scoped to the
"Splitter input Exchange"
during processing the "splitted" Exchanges. For example: if it would be
possible to access
the "input" Exchange during processing the "splitted" Exchanges I would set
a property
on the "input" Exchange during processing the first line and reading it
afterwards during
processing the remaining lines.
Is there any other storage location besides an Exchange property which is
scoped to the
Exchange and is accessible from the "splitted" Exchanges?
Greetings
Oliver
2012/11/30 Claus Ibsen <cl...@gmail.com>
> Hi
>
> See this EIP
> http://camel.apache.org/composed-message-processor.html
>
> On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
> <ol...@gmail.com> wrote:
> > Hello,
> >
> > I am using a splitter to handle a big txt file (streaming mode). I need
> to
> > combine the first line
> > with all the other lines and store each result into a database.
> >
> > What is a good Camel solution for this?
> >
> > Example input file:
> >
> > Hello
> > A
> > B
> > C
> > ...
> >
> >
> > This needs to be combined into:
> >
> > Hello A
> > Hello B
> > Hello C
> > ...
> >
> > Every result is stored into a database.
> > It's important to use streaming because of the size of the input file.
> >
> >
> > My first idea was to use a Splitter and then store the first line into an
> > Exchange property.
> > But this does not work because the Splitter creates new Exchanges for
> every
> > line.
> >
> > So how do I combine the first line with all the other lines?
> >
> > Thanks
> > Oliver
> >
> > --
> > og
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: cibsen@redhat.com
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
>
--
og
Re: Splitter and combining messages / exchanges
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
See this EIP
http://camel.apache.org/composed-message-processor.html
On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
<ol...@gmail.com> wrote:
> Hello,
>
> I am using a splitter to handle a big txt file (streaming mode). I need to
> combine the first line
> with all the other lines and store each result into a database.
>
> What is a good Camel solution for this?
>
> Example input file:
>
> Hello
> A
> B
> C
> ...
>
>
> This needs to be combined into:
>
> Hello A
> Hello B
> Hello C
> ...
>
> Every result is stored into a database.
> It's important to use streaming because of the size of the input file.
>
>
> My first idea was to use a Splitter and then store the first line into an
> Exchange property.
> But this does not work because the Splitter creates new Exchanges for every
> line.
>
> So how do I combine the first line with all the other lines?
>
> Thanks
> Oliver
>
> --
> og
--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Splitter and combining messages / exchanges
Posted by Christian Müller <ch...@gmail.com>.
We did this in one of our projects by providing a bean to the splitter
which returns an iterator. In this iterator, we first read the first line
and store it in this iterator as instance variable. For each next() call,
we read the next line and and "prefix" it with the first line.
Best,
Christian
Sent from a mobile device
Am 29.11.2012 23:28 schrieb "Oliver Geisser" <ol...@gmail.com>:
> Hello,
>
> I am using a splitter to handle a big txt file (streaming mode). I need to
> combine the first line
> with all the other lines and store each result into a database.
>
> What is a good Camel solution for this?
>
> Example input file:
>
> Hello
> A
> B
> C
> ...
>
>
> This needs to be combined into:
>
> Hello A
> Hello B
> Hello C
> ...
>
> Every result is stored into a database.
> It's important to use streaming because of the size of the input file.
>
>
> My first idea was to use a Splitter and then store the first line into an
> Exchange property.
> But this does not work because the Splitter creates new Exchanges for every
> line.
>
> So how do I combine the first line with all the other lines?
>
> Thanks
> Oliver
>
> --
> og
>