You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by cmoulliard <cm...@gmail.com> on 2008/07/14 16:04:20 UTC

Component to read FixedLength file ?

Hi,

Is there a component expected to read fixedlength files in Camel like
flatpack (http://flatpack.sourceforge.net/) allow to do it ?

Regards,

Charles Moulliard

-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18444614.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
Can we also create a fixed-length file (using Flatpack) through Camel ? If
yes, How ?


James.Strachan wrote:
> 
> Its probably gonna miss the 1.4 release but here you go...
> http://cwiki.apache.org/CAMEL/flatpack.html
> 
> 2008/7/14 James Strachan <ja...@gmail.com>:
>> BTW trunk has an early spike of a flatpack component. Needs more work,
>> but there's a basic fixed-width reader example there for the curious
>>
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> I've raised a JIRA to track this...
>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>
>>>> It should be pretty easy to write a flatpack component.
>>>>
>>>> there's a commercial tool called Artix Data Services which comes with
>>>> a nice design UI that does this with Camel...
>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>
>>>> but it'd be nice to have an OSS option :)
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18466977.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Jesus James two new components in 2-3 days.
You are really on a fire here.

Now go fix CAMEL-616, it's the dataformat side as well ;)

Use-case:
The files could for instance be provided with FTP so we should be able to process them as well. But we could of course store the FTP files as local files and use the new component to read them again. 

Anyone else see a valid use case for a dataformat? Flatpack kinda files are usually bound to files.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
-----Original Message-----
From: James Strachan [mailto:james.strachan@gmail.com] 
Sent: 14. juli 2008 20:05
To: camel-user@activemq.apache.org
Subject: Re: Component to read FixedLength file ?

Its probably gonna miss the 1.4 release but here you go...
http://cwiki.apache.org/CAMEL/flatpack.html

2008/7/14 James Strachan <ja...@gmail.com>:
> BTW trunk has an early spike of a flatpack component. Needs more work,
> but there's a basic fixed-width reader example there for the curious
>
> 2008/7/14 James Strachan <ja...@gmail.com>:
>> I've raised a JIRA to track this...
>> https://issues.apache.org/activemq/browse/CAMEL-717
>>
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>
>>>> Hi,
>>>>
>>>> Is there a component expected to read fixedlength files in Camel like
>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>
>>> It should be pretty easy to write a flatpack component.
>>>
>>> there's a commercial tool called Artix Data Services which comes with
>>> a nice design UI that does this with Camel...
>>> http://activemq.apache.org/camel/artix-data-services.html
>>>
>>> but it'd be nice to have an OSS option :)
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
Its probably gonna miss the 1.4 release but here you go...
http://cwiki.apache.org/CAMEL/flatpack.html

2008/7/14 James Strachan <ja...@gmail.com>:
> BTW trunk has an early spike of a flatpack component. Needs more work,
> but there's a basic fixed-width reader example there for the curious
>
> 2008/7/14 James Strachan <ja...@gmail.com>:
>> I've raised a JIRA to track this...
>> https://issues.apache.org/activemq/browse/CAMEL-717
>>
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>
>>>> Hi,
>>>>
>>>> Is there a component expected to read fixedlength files in Camel like
>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>
>>> It should be pretty easy to write a flatpack component.
>>>
>>> there's a commercial tool called Artix Data Services which comes with
>>> a nice design UI that does this with Camel...
>>> http://activemq.apache.org/camel/artix-data-services.html
>>>
>>> but it'd be nice to have an OSS option :)
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Flatpatch (was: RE: Component to read FixedLength file ?)

Posted by cmoulliard <cm...@gmail.com>.
What if you need only 1 exchange for the entire flatpack file? 
>> That's my idea because the content of the file must be processed as a
>> whole

Header
row1
row2
row3
...
row n
Trailer

and the information of the header/trailer are precious. Moreover, the
definition of the file's structure (defining in a XML file) has no sense if
the component create a message/line because in this case, flatpack must
process globally the file.

So, is it possible to adapt the behavior of the component to process all the
content of the file ?


Claus Ibsen wrote:
> 
> Hi
> 
> I see the current behaviour of the flatpack component is to create an
> exchange for each line in the flatpack file.
> 
> What if you need only 1 exchange for the entire flatpack file? 
> Before you say, you can use the aggregator, then yes but then I am not
> sure when the total count so I can actually not aggregate them all. 
> 
> What if there was an option to specify if you want:
> Option a) 1 exchange only with a List of Map with the data
> Option b) as now 1 exchange pr. row with a Map as the data
> 
> The atom component has a similar feature where you can configure either
> option A or B.
> 
> With this option it's easier for end-users to get what they want. Also if
> possible we should have a total row count on the message as a header
> option if possible. So you will now the current sequence and the total
> number of rows, in the exchange you get.
> 
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: James Strachan [mailto:james.strachan@gmail.com] 
> Sent: 15. juli 2008 20:52
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
> to use. The downside is the data format kinda has to turn the DataSet
> into a List of Maps to make it easier to work with; but at least the
> routes are a bit simpler. e.g. this is from the
> DelimitedWithUnmarshalTest (the convertBodyTo is not required
> currently)
> 
>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>     <route>
>       <from uri="file://src/test/data/delim?noop=true"/>
>       <unmarshal ref="delimitedFormat"/>
>       <splitter>
>         <simple>in.body</simple>
>         <convertBodyTo type="java.util.Map"/>
>         <to uri="mock:results"/>
>       </splitter>
>     </route>
>   </camelContext>
> 
>   <bean id="delimitedFormat"
> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>   </bean>
> 
> 2008/7/15 James Strachan <ja...@gmail.com>:
>> Yeah - I did ponder if it should be a dataformat or a component; or
>> maybe even both. The bit I wasn't sure about was flatpack really just
>> provides an iterator over rows of the file; I wasn't sure if this
>> properly matched the dataformat model of marshalling or unmarshalling
>> a payload (which currently assumes that unmarshalling generates a
>> single result).
>>
>> Maybe we need to enhance the dataformat concept to work with this use
>> case too?
>>
>> So maybe if we want to process each row of a file we do something like...
>>
>> from("file://whatnot").
>> 
>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>  split().body().
>>  to("whatever");
>>
>> where the split().body() would do the iteration over each DataSet?
>>
>>
>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>> Hi
>>>
>>> Yes I thought also at first it was a dataformat ;) But how James created
>>> it it works as if it is a dataformat. You just have to do it in to..from
>>> from..to steps.
>>>
>>>
>>> from("file://c:/test").to("flatpack:foo:
>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>
>>> from("flatpack:foo").to("whatever you want");
>>>
>>> As a component it gets more attention, as people can find it on the
>>> component list. The dataformats are kinda hidden in the wiki
>>> documentation.
>>> Maybe they need a "dataformat" top menu as well ;)
>>>
>>> Med venlig hilsen
>>>
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>>
>>> -----Original Message-----
>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>> Sent: 15. juli 2008 09:32
>>> To: camel-user@activemq.apache.org
>>> Subject: Re: Component to read FixedLength file ?
>>>
>>>
>>> Many thanks.
>>>
>>> I will test the integration of this component within Camel.
>>>
>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>> The
>>> project has been integrated like a component but from my point of view,
>>> it
>>> should be integrated (for consistency reason) like a Dataformat. Because
>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>>> into another format (Collection of Java objects)
>>>
>>> E.g.
>>>
>>> DataFormat flatpack = new
>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>
>>> from("file:///c:/test")
>>> .unmarshall(flatpack)
>>> .to("whatever you want")
>>>
>>> Regards,
>>>
>>> Charles
>>>
>>>
>>>
>>> James.Strachan wrote:
>>>>
>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>> but there's a basic fixed-width reader example there for the curious
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> I've raised a JIRA to track this...
>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>> like
>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>
>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>
>>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>>> a nice design UI that does this with Camel...
>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>
>>>>>> but it'd be nice to have an OSS option :)
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>>
>>>
>>>
>>> -----
>>> Enterprise Architect
>>>
>>> Xpectis
>>> 12, route d'Esch
>>> L-1470 Luxembourg
>>>
>>> Phone +352 25 10 70 470
>>> Mobile +352 621 45 36 22
>>>
>>> e-mail : cmoulliard@xpectis.com
>>> web site : www.xpectis.com
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18488887.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Flatpatch (was: RE: Component to read FixedLength file ?)

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Hi

I see the current behaviour of the flatpack component is to create an exchange for each line in the flatpack file.

What if you need only 1 exchange for the entire flatpack file? 
Before you say, you can use the aggregator, then yes but then I am not sure when the total count so I can actually not aggregate them all. 

What if there was an option to specify if you want:
Option a) 1 exchange only with a List of Map with the data
Option b) as now 1 exchange pr. row with a Map as the data

The atom component has a similar feature where you can configure either option A or B.

With this option it's easier for end-users to get what they want. Also if possible we should have a total row count on the message as a header option if possible. So you will now the current sequence and the total number of rows, in the exchange you get.



Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: James Strachan [mailto:james.strachan@gmail.com] 
Sent: 15. juli 2008 20:52
To: camel-user@activemq.apache.org
Subject: Re: Component to read FixedLength file ?

FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
to use. The downside is the data format kinda has to turn the DataSet
into a List of Maps to make it easier to work with; but at least the
routes are a bit simpler. e.g. this is from the
DelimitedWithUnmarshalTest (the convertBodyTo is not required
currently)

  <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
    <route>
      <from uri="file://src/test/data/delim?noop=true"/>
      <unmarshal ref="delimitedFormat"/>
      <splitter>
        <simple>in.body</simple>
        <convertBodyTo type="java.util.Map"/>
        <to uri="mock:results"/>
      </splitter>
    </route>
  </camelContext>

  <bean id="delimitedFormat"
class="org.apache.camel.component.flatpack.FlatpackDataFormat">
    <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
  </bean>

2008/7/15 James Strachan <ja...@gmail.com>:
> Yeah - I did ponder if it should be a dataformat or a component; or
> maybe even both. The bit I wasn't sure about was flatpack really just
> provides an iterator over rows of the file; I wasn't sure if this
> properly matched the dataformat model of marshalling or unmarshalling
> a payload (which currently assumes that unmarshalling generates a
> single result).
>
> Maybe we need to enhance the dataformat concept to work with this use case too?
>
> So maybe if we want to process each row of a file we do something like...
>
> from("file://whatnot").
>  unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>  split().body().
>  to("whatever");
>
> where the split().body() would do the iteration over each DataSet?
>
>
> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>> Hi
>>
>> Yes I thought also at first it was a dataformat ;) But how James created it it works as if it is a dataformat. You just have to do it in to..from from..to steps.
>>
>>
>> from("file://c:/test").to("flatpack:foo: locationOftheXMLConfigurationfile.pzmap.xml");
>>
>> from("flatpack:foo").to("whatever you want");
>>
>> As a component it gets more attention, as people can find it on the component list. The dataformats are kinda hidden in the wiki documentation.
>> Maybe they need a "dataformat" top menu as well ;)
>>
>> Med venlig hilsen
>>
>> Claus Ibsen
>> ......................................
>> Silverbullet
>> Skovsgårdsvænget 21
>> 8362 Hørning
>> Tlf. +45 2962 7576
>> Web: www.silverbullet.dk
>>
>> -----Original Message-----
>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>> Sent: 15. juli 2008 09:32
>> To: camel-user@activemq.apache.org
>> Subject: Re: Component to read FixedLength file ?
>>
>>
>> Many thanks.
>>
>> I will test the integration of this component within Camel.
>>
>> Remark : I'm a little confused about how Flatpack has been integrated. The
>> project has been integrated like a component but from my point of view, it
>> should be integrated (for consistency reason) like a Dataformat. Because
>> like the other framework/technology implemented in Camel (JAXB, Artix,
>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>> into another format (Collection of Java objects)
>>
>> E.g.
>>
>> DataFormat flatpack = new
>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>
>> from("file:///c:/test")
>> .unmarshall(flatpack)
>> .to("whatever you want")
>>
>> Regards,
>>
>> Charles
>>
>>
>>
>> James.Strachan wrote:
>>>
>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>> but there's a basic fixed-width reader example there for the curious
>>>
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> I've raised a JIRA to track this...
>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>
>>>>> It should be pretty easy to write a flatpack component.
>>>>>
>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>> a nice design UI that does this with Camel...
>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>
>>>>> but it'd be nice to have an OSS option :)
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>>
>>
>>
>> -----
>> Enterprise Architect
>>
>> Xpectis
>> 12, route d'Esch
>> L-1470 Luxembourg
>>
>> Phone +352 25 10 70 470
>> Mobile +352 621 45 36 22
>>
>> e-mail : cmoulliard@xpectis.com
>> web site : www.xpectis.com
>> --
>> View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
2008/7/17 cmoulliard <cm...@gmail.com>:
>
> Hi Claus,
>
> If I remove from the route the .convertBody(Map.class) and that I create a
> processor to display the result. I can see all the rows (including also the
> trailer) except the first which is the trailer when I parse a
> fixedlengthwithheaderandtrailer file. From my point of there is a bug.
>
> The flag that you mention is only used when you process a delimited file
> (=csv). In my case, this one is not used because we are working with a
> fixedlength file including a header + trailer.
>
> To clarify the situation, it should be interesting that we can use a flag to
> inform the flatpack component that we want to generate one message/row
> defined in the incoming file. Otherwise, the user will be dependent of the
> component behavior. In certain cases, it makes sense to generate one message
> / row but in another situation not at all because the client wants that we
> process in one thread its file. Moreover, when the file contains a
> header/trailer flatpack will map them too (using the record trailer/header)
> defined in the xxxx.pzm.xml configuration file.
>
> I will to try to prepare a unit test. How can I provide it to the project ?

Here's how to contribute...
http://activemq.apache.org/camel/contributing.html

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Hi Charles

Better later han newer. I have improved the camel-flatpack component. It should handle now:
- header/trailer
- being able to send all the in one exchange (splitRows=false)

See the updated wiki:
http://activemq.apache.org/camel/flatpack.html

You must use the 1.5-SNAPSHOT as the changes are in the next release.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: cmoulliard [mailto:cmoulliard@gmail.com] 
Sent: 17. juli 2008 14:34
To: camel-user@activemq.apache.org
Subject: RE: Component to read FixedLength file ?


Hi Claus,

If I remove from the route the .convertBody(Map.class) and that I create a
processor to display the result. I can see all the rows (including also the
trailer) except the first which is the trailer when I parse a
fixedlengthwithheaderandtrailer file. From my point of there is a bug.

The flag that you mention is only used when you process a delimited file
(=csv). In my case, this one is not used because we are working with a
fixedlength file including a header + trailer.

To clarify the situation, it should be interesting that we can use a flag to
inform the flatpack component that we want to generate one message/row
defined in the incoming file. Otherwise, the user will be dependent of the
component behavior. In certain cases, it makes sense to generate one message
/ row but in another situation not at all because the client wants that we
process in one thread its file. Moreover, when the file contains a
header/trailer flatpack will map them too (using the record trailer/header)
defined in the xxxx.pzm.xml configuration file.
 
I will to try to prepare a unit test. How can I provide it to the project ?

Regards,

Charles Moulliard

Claus Ibsen wrote:
> 
> Hi Charles
> 
> And the "trailer" is something that flatpack support out-of-the-box?
> I am by no means an expert on flatpack but are you sure its something it
> understands?
> 
> In the camel-flatpack code there is an option to skip the first record,
> but none for the last record. So I am just digging here.
> 
> Feel free to put together a small junit test that we can add to the test
> in camel-flatpack where the input file has both a header and a trailer
> row. Then we can use that as a base for a bugfix.
> 
> You can use some of the existing unit tests as inspiration how to add your
> unit test.
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 16. juli 2008 10:21
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> 
> I have a new error when I use the following routes. 
> 
> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>                
> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>                
>                
>        
> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>                 .convertBodyTo(Map.class)
>                 .to("activemq:queue:cutas.in.queue");
> 
> and xml file
> 
> <?xml version='1.0'?>
> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
> 
> <PZMAP>
> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="HEADERDATA" length="26" />
> 	</RECORD>
> 	<COLUMN name="RECORDTYPE" length="1" />
> 	<COLUMN name="SYSTEMID" length="3" />
> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
> 	<COLUMN name="FUND_CODE" length="5" />
> 	<COLUMN name="PRODUCT_CODE" length="5" />
> 	<COLUMN name="PRODUCT_TYPE" length="4" />
> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
> 	<COLUMN name="FUND_TYPE" length="1" />
> 	<COLUMN name="UNIT_SIGN" length="1" />
> 	<COLUMN name="DATA" length="412" />
> 	
> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="TRAILERDATA" length="26" />
> 	</RECORD>
> 
> </PZMAP> 
> 
> Remark : In my case, we have a header / Trailer in the message.
> 
> Error reported :
> 
> 
> 43org.apache.camel.RuntimeCamelException:
> java.util.NoSuchElementException:
> Column [recordtype] does not exist, check case/spelling. key:trailer
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
> 	at
> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
> 	at
> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
> 	at
> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
> 	at
> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
> 	at
> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
> 	at
> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
> 	at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
> 	at
> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
> 	at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: java.util.NoSuchElementException: Column [recordtype] does not
> exist, check case/spelling. key:trailer
> 	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
> 	at net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 	at java.lang.reflect.Method.invoke(Unknown Source)
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
> 	... 55 more
> 
> Content of the message :
> 
> 4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-0 -> to7
> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
> InOnly Properties:{CamelCauseException=null,
> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
> exclusive valid],
> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
> Body:HBT00032700000327190620071714
> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
> +0000000010000000000009076000000138137A000090760EUR  
> GBPSS808800714636FIEU
> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
> 00001522000 000000000000000O                  0000000000000000                                        
> 20070907155200                                                                             
> X
> C00111a30571b170007B000005        APAA          200805271554  
> +0000000001835910000003941700000039417A000332147USD  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000B                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571a600006B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000X                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571c2a0009B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000                  
> 0000000000000000                                         20070619101000                                                                             
> X
> FBT00032700000001190620071714
> 
> Info displayed on the console :
> 
> 4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
> null]]), sendTo(Endpoint[seda:errors]),
> RedeliveryPolicy[maximumRedeliveries=1]],
> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
> sendTo(Endpoint[seda:errors]), RedeliveryPolicy[maximumRedeliveries=1]]])]
> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
> Rows:6
> Position:0
> Conversion
> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
> , Name:SYSTEMID Length:3 Start:0 End:0
> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
> , Name:FUND_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
> , Name:FUND_TYPE Length:1 Start:0 End:0
> , Name:UNIT_SIGN Length:1 Start:0 End:0
> , Name:DATA Length:412 Start:0 End:0
> ]
> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10, systemid=1,
> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0, data=12,
> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
> 
> Regards,
> 
> Charles
> 
> 
> 
> James.Strachan wrote:
>> 
>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>> to use. The downside is the data format kinda has to turn the DataSet
>> into a List of Maps to make it easier to work with; but at least the
>> routes are a bit simpler. e.g. this is from the
>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>> currently)
>> 
>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>     <route>
>>       <from uri="file://src/test/data/delim?noop=true"/>
>>       <unmarshal ref="delimitedFormat"/>
>>       <splitter>
>>         <simple>in.body</simple>
>>         <convertBodyTo type="java.util.Map"/>
>>         <to uri="mock:results"/>
>>       </splitter>
>>     </route>
>>   </camelContext>
>> 
>>   <bean id="delimitedFormat"
>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>   </bean>
>> 
>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>> provides an iterator over rows of the file; I wasn't sure if this
>>> properly matched the dataformat model of marshalling or unmarshalling
>>> a payload (which currently assumes that unmarshalling generates a
>>> single result).
>>>
>>> Maybe we need to enhance the dataformat concept to work with this use
>>> case too?
>>>
>>> So maybe if we want to process each row of a file we do something
>>> like...
>>>
>>> from("file://whatnot").
>>> 
>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>  split().body().
>>>  to("whatever");
>>>
>>> where the split().body() would do the iteration over each DataSet?
>>>
>>>
>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>> Hi
>>>>
>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>> created
>>>> it it works as if it is a dataformat. You just have to do it in
>>>> to..from
>>>> from..to steps.
>>>>
>>>>
>>>> from("file://c:/test").to("flatpack:foo:
>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>
>>>> from("flatpack:foo").to("whatever you want");
>>>>
>>>> As a component it gets more attention, as people can find it on the
>>>> component list. The dataformats are kinda hidden in the wiki
>>>> documentation.
>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>
>>>> Med venlig hilsen
>>>>
>>>> Claus Ibsen
>>>> ......................................
>>>> Silverbullet
>>>> Skovsgårdsvænget 21
>>>> 8362 Hørning
>>>> Tlf. +45 2962 7576
>>>> Web: www.silverbullet.dk
>>>>
>>>> -----Original Message-----
>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>> Sent: 15. juli 2008 09:32
>>>> To: camel-user@activemq.apache.org
>>>> Subject: Re: Component to read FixedLength file ?
>>>>
>>>>
>>>> Many thanks.
>>>>
>>>> I will test the integration of this component within Camel.
>>>>
>>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>>> The
>>>> project has been integrated like a component but from my point of view,
>>>> it
>>>> should be integrated (for consistency reason) like a Dataformat.
>>>> Because
>>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>> ...)
>>>> into another format (Collection of Java objects)
>>>>
>>>> E.g.
>>>>
>>>> DataFormat flatpack = new
>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>
>>>> from("file:///c:/test")
>>>> .unmarshall(flatpack)
>>>> .to("whatever you want")
>>>>
>>>> Regards,
>>>>
>>>> Charles
>>>>
>>>>
>>>>
>>>> James.Strachan wrote:
>>>>>
>>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> I've raised a JIRA to track this...
>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>
>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>> like
>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>
>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>
>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>> with
>>>>>>> a nice design UI that does this with Camel...
>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>
>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>>
>>>>
>>>>
>>>> -----
>>>> Enterprise Architect
>>>>
>>>> Xpectis
>>>> 12, route d'Esch
>>>> L-1470 Luxembourg
>>>>
>>>> Phone +352 25 10 70 470
>>>> Mobile +352 621 45 36 22
>>>>
>>>> e-mail : cmoulliard@xpectis.com
>>>> web site : www.xpectis.com
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>> 
>> 
>> 
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>> 
>> Open Source Integration
>> http://open.iona.com
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18482019.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18507000.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
Hi Claus,

If I remove from the route the .convertBody(Map.class) and that I create a
processor to display the result. I can see all the rows (including also the
trailer) except the first which is the trailer when I parse a
fixedlengthwithheaderandtrailer file. From my point of there is a bug.

The flag that you mention is only used when you process a delimited file
(=csv). In my case, this one is not used because we are working with a
fixedlength file including a header + trailer.

To clarify the situation, it should be interesting that we can use a flag to
inform the flatpack component that we want to generate one message/row
defined in the incoming file. Otherwise, the user will be dependent of the
component behavior. In certain cases, it makes sense to generate one message
/ row but in another situation not at all because the client wants that we
process in one thread its file. Moreover, when the file contains a
header/trailer flatpack will map them too (using the record trailer/header)
defined in the xxxx.pzm.xml configuration file.
 
I will to try to prepare a unit test. How can I provide it to the project ?

Regards,

Charles Moulliard

Claus Ibsen wrote:
> 
> Hi Charles
> 
> And the "trailer" is something that flatpack support out-of-the-box?
> I am by no means an expert on flatpack but are you sure its something it
> understands?
> 
> In the camel-flatpack code there is an option to skip the first record,
> but none for the last record. So I am just digging here.
> 
> Feel free to put together a small junit test that we can add to the test
> in camel-flatpack where the input file has both a header and a trailer
> row. Then we can use that as a base for a bugfix.
> 
> You can use some of the existing unit tests as inspiration how to add your
> unit test.
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 16. juli 2008 10:21
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> 
> I have a new error when I use the following routes. 
> 
> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>                
> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>                
>                
>        
> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>                 .convertBodyTo(Map.class)
>                 .to("activemq:queue:cutas.in.queue");
> 
> and xml file
> 
> <?xml version='1.0'?>
> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
> 
> <PZMAP>
> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="HEADERDATA" length="26" />
> 	</RECORD>
> 	<COLUMN name="RECORDTYPE" length="1" />
> 	<COLUMN name="SYSTEMID" length="3" />
> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
> 	<COLUMN name="FUND_CODE" length="5" />
> 	<COLUMN name="PRODUCT_CODE" length="5" />
> 	<COLUMN name="PRODUCT_TYPE" length="4" />
> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
> 	<COLUMN name="FUND_TYPE" length="1" />
> 	<COLUMN name="UNIT_SIGN" length="1" />
> 	<COLUMN name="DATA" length="412" />
> 	
> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="TRAILERDATA" length="26" />
> 	</RECORD>
> 
> </PZMAP> 
> 
> Remark : In my case, we have a header / Trailer in the message.
> 
> Error reported :
> 
> 
> 43org.apache.camel.RuntimeCamelException:
> java.util.NoSuchElementException:
> Column [recordtype] does not exist, check case/spelling. key:trailer
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
> 	at
> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
> 	at
> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
> 	at
> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
> 	at
> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
> 	at
> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
> 	at
> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
> 	at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
> 	at
> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
> 	at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: java.util.NoSuchElementException: Column [recordtype] does not
> exist, check case/spelling. key:trailer
> 	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
> 	at net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 	at java.lang.reflect.Method.invoke(Unknown Source)
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
> 	... 55 more
> 
> Content of the message :
> 
> 4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-0 -> to7
> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
> InOnly Properties:{CamelCauseException=null,
> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
> exclusive valid],
> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
> Body:HBT00032700000327190620071714
> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
> +0000000010000000000009076000000138137A000090760EUR  
> GBPSS808800714636FIEU
> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
> 00001522000 000000000000000O                  0000000000000000                                        
> 20070907155200                                                                             
> X
> C00111a30571b170007B000005        APAA          200805271554  
> +0000000001835910000003941700000039417A000332147USD  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000B                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571a600006B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000X                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571c2a0009B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000                  
> 0000000000000000                                         20070619101000                                                                             
> X
> FBT00032700000001190620071714
> 
> Info displayed on the console :
> 
> 4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
> null]]), sendTo(Endpoint[seda:errors]),
> RedeliveryPolicy[maximumRedeliveries=1]],
> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
> sendTo(Endpoint[seda:errors]), RedeliveryPolicy[maximumRedeliveries=1]]])]
> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
> Rows:6
> Position:0
> Conversion
> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
> , Name:SYSTEMID Length:3 Start:0 End:0
> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
> , Name:FUND_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
> , Name:FUND_TYPE Length:1 Start:0 End:0
> , Name:UNIT_SIGN Length:1 Start:0 End:0
> , Name:DATA Length:412 Start:0 End:0
> ]
> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10, systemid=1,
> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0, data=12,
> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
> 
> Regards,
> 
> Charles
> 
> 
> 
> James.Strachan wrote:
>> 
>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>> to use. The downside is the data format kinda has to turn the DataSet
>> into a List of Maps to make it easier to work with; but at least the
>> routes are a bit simpler. e.g. this is from the
>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>> currently)
>> 
>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>     <route>
>>       <from uri="file://src/test/data/delim?noop=true"/>
>>       <unmarshal ref="delimitedFormat"/>
>>       <splitter>
>>         <simple>in.body</simple>
>>         <convertBodyTo type="java.util.Map"/>
>>         <to uri="mock:results"/>
>>       </splitter>
>>     </route>
>>   </camelContext>
>> 
>>   <bean id="delimitedFormat"
>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>   </bean>
>> 
>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>> provides an iterator over rows of the file; I wasn't sure if this
>>> properly matched the dataformat model of marshalling or unmarshalling
>>> a payload (which currently assumes that unmarshalling generates a
>>> single result).
>>>
>>> Maybe we need to enhance the dataformat concept to work with this use
>>> case too?
>>>
>>> So maybe if we want to process each row of a file we do something
>>> like...
>>>
>>> from("file://whatnot").
>>> 
>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>  split().body().
>>>  to("whatever");
>>>
>>> where the split().body() would do the iteration over each DataSet?
>>>
>>>
>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>> Hi
>>>>
>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>> created
>>>> it it works as if it is a dataformat. You just have to do it in
>>>> to..from
>>>> from..to steps.
>>>>
>>>>
>>>> from("file://c:/test").to("flatpack:foo:
>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>
>>>> from("flatpack:foo").to("whatever you want");
>>>>
>>>> As a component it gets more attention, as people can find it on the
>>>> component list. The dataformats are kinda hidden in the wiki
>>>> documentation.
>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>
>>>> Med venlig hilsen
>>>>
>>>> Claus Ibsen
>>>> ......................................
>>>> Silverbullet
>>>> Skovsgårdsvænget 21
>>>> 8362 Hørning
>>>> Tlf. +45 2962 7576
>>>> Web: www.silverbullet.dk
>>>>
>>>> -----Original Message-----
>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>> Sent: 15. juli 2008 09:32
>>>> To: camel-user@activemq.apache.org
>>>> Subject: Re: Component to read FixedLength file ?
>>>>
>>>>
>>>> Many thanks.
>>>>
>>>> I will test the integration of this component within Camel.
>>>>
>>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>>> The
>>>> project has been integrated like a component but from my point of view,
>>>> it
>>>> should be integrated (for consistency reason) like a Dataformat.
>>>> Because
>>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>> ...)
>>>> into another format (Collection of Java objects)
>>>>
>>>> E.g.
>>>>
>>>> DataFormat flatpack = new
>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>
>>>> from("file:///c:/test")
>>>> .unmarshall(flatpack)
>>>> .to("whatever you want")
>>>>
>>>> Regards,
>>>>
>>>> Charles
>>>>
>>>>
>>>>
>>>> James.Strachan wrote:
>>>>>
>>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> I've raised a JIRA to track this...
>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>
>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>> like
>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>
>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>
>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>> with
>>>>>>> a nice design UI that does this with Camel...
>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>
>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>>
>>>>
>>>>
>>>> -----
>>>> Enterprise Architect
>>>>
>>>> Xpectis
>>>> 12, route d'Esch
>>>> L-1470 Luxembourg
>>>>
>>>> Phone +352 25 10 70 470
>>>> Mobile +352 621 45 36 22
>>>>
>>>> e-mail : cmoulliard@xpectis.com
>>>> web site : www.xpectis.com
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>> 
>> 
>> 
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>> 
>> Open Source Integration
>> http://open.iona.com
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18482019.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18507000.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Hi Charles

And the "trailer" is something that flatpack support out-of-the-box?
I am by no means an expert on flatpack but are you sure its something it understands?

In the camel-flatpack code there is an option to skip the first record, but none for the last record. So I am just digging here.

Feel free to put together a small junit test that we can add to the test in camel-flatpack where the input file has both a header and a trailer row. Then we can use that as a base for a bugfix.

You can use some of the existing unit tests as inspiration how to add your unit test.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
-----Original Message-----
From: cmoulliard [mailto:cmoulliard@gmail.com] 
Sent: 16. juli 2008 10:21
To: camel-user@activemq.apache.org
Subject: Re: Component to read FixedLength file ?


I have a new error when I use the following routes. 

from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
               
.to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
               
               
       
from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
                .convertBodyTo(Map.class)
                .to("activemq:queue:cutas.in.queue");

and xml file

<?xml version='1.0'?>
<!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >

<PZMAP>
	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="HEADERDATA" length="26" />
	</RECORD>
	<COLUMN name="RECORDTYPE" length="1" />
	<COLUMN name="SYSTEMID" length="3" />
	<COLUMN name="THIRD_PARTY_REF" length="15" />
	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
	<COLUMN name="FUND_CODE" length="5" />
	<COLUMN name="PRODUCT_CODE" length="5" />
	<COLUMN name="PRODUCT_TYPE" length="4" />
	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
	<COLUMN name="TRANSACTION_TYPE" length="2" />
	<COLUMN name="FUND_TYPE" length="1" />
	<COLUMN name="UNIT_SIGN" length="1" />
	<COLUMN name="DATA" length="412" />
	
	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="TRAILERDATA" length="26" />
	</RECORD>

</PZMAP> 

Remark : In my case, we have a header / Trailer in the message.

Error reported :


43org.apache.camel.RuntimeCamelException: java.util.NoSuchElementException:
Column [recordtype] does not exist, check case/spelling. key:trailer
	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
	at
org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
	at
org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
	at
org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
	at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
	at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
	at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
	at
org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
	at
org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
	at
org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
	at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
	at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
	at
org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
	at org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
	at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
Source)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
Source)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.util.NoSuchElementException: Column [recordtype] does not
exist, check case/spelling. key:trailer
	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
	at net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
	at
org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
	at
org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
	... 55 more

Content of the message :

4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/1446-1216195452000/2-0 -> to7
To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
InOnly Properties:{CamelCauseException=null,
org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
exclusive valid],
org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
Body:HBT00032700000327190620071714
C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
+0000000010000000000009076000000138137A000090760EUR   GBPSS808800714636FIEU
/0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
00001522000 000000000000000O                  0000000000000000                                        
20070907155200                                                                             
X
C00111a30571b170007B000005        APAA          200805271554  
+0000000001835910000003941700000039417A000332147USD  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000B                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571a600006B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000X                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571c2a0009B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000                  
0000000000000000                                         20070619101000                                                                             
X
FBT00032700000001190620071714

Info displayed on the console :

4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
null]]), sendTo(Endpoint[seda:errors]),
RedeliveryPolicy[maximumRedeliveries=1]],
DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
sendTo(Endpoint[seda:errors]), RedeliveryPolicy[maximumRedeliveries=1]]])]
InOnly Properties:{} Headers:{camelFlatpackCounter=0}
BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
Rows:6
Position:0
Conversion Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
, Name:SYSTEMID Length:3 Start:0 End:0
, Name:THIRD_PARTY_REF Length:15 Start:0 End:0
, Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
, Name:FUND_CODE Length:5 Start:0 End:0
, Name:PRODUCT_CODE Length:5 Start:0 End:0
, Name:PRODUCT_TYPE Length:4 Start:0 End:0
, Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
, Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
, Name:TRANSACTION_TYPE Length:2 Start:0 End:0
, Name:FUND_TYPE Length:1 Start:0 End:0
, Name:UNIT_SIGN Length:1 Start:0 End:0
, Name:DATA Length:412 Start:0 End:0
]
Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10, systemid=1,
cutas_account_nr=3, product_code=5, product_type=6, recordtype=0, data=12,
cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}

Regards,

Charles



James.Strachan wrote:
> 
> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
> to use. The downside is the data format kinda has to turn the DataSet
> into a List of Maps to make it easier to work with; but at least the
> routes are a bit simpler. e.g. this is from the
> DelimitedWithUnmarshalTest (the convertBodyTo is not required
> currently)
> 
>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>     <route>
>       <from uri="file://src/test/data/delim?noop=true"/>
>       <unmarshal ref="delimitedFormat"/>
>       <splitter>
>         <simple>in.body</simple>
>         <convertBodyTo type="java.util.Map"/>
>         <to uri="mock:results"/>
>       </splitter>
>     </route>
>   </camelContext>
> 
>   <bean id="delimitedFormat"
> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>   </bean>
> 
> 2008/7/15 James Strachan <ja...@gmail.com>:
>> Yeah - I did ponder if it should be a dataformat or a component; or
>> maybe even both. The bit I wasn't sure about was flatpack really just
>> provides an iterator over rows of the file; I wasn't sure if this
>> properly matched the dataformat model of marshalling or unmarshalling
>> a payload (which currently assumes that unmarshalling generates a
>> single result).
>>
>> Maybe we need to enhance the dataformat concept to work with this use
>> case too?
>>
>> So maybe if we want to process each row of a file we do something like...
>>
>> from("file://whatnot").
>> 
>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>  split().body().
>>  to("whatever");
>>
>> where the split().body() would do the iteration over each DataSet?
>>
>>
>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>> Hi
>>>
>>> Yes I thought also at first it was a dataformat ;) But how James created
>>> it it works as if it is a dataformat. You just have to do it in to..from
>>> from..to steps.
>>>
>>>
>>> from("file://c:/test").to("flatpack:foo:
>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>
>>> from("flatpack:foo").to("whatever you want");
>>>
>>> As a component it gets more attention, as people can find it on the
>>> component list. The dataformats are kinda hidden in the wiki
>>> documentation.
>>> Maybe they need a "dataformat" top menu as well ;)
>>>
>>> Med venlig hilsen
>>>
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>>
>>> -----Original Message-----
>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>> Sent: 15. juli 2008 09:32
>>> To: camel-user@activemq.apache.org
>>> Subject: Re: Component to read FixedLength file ?
>>>
>>>
>>> Many thanks.
>>>
>>> I will test the integration of this component within Camel.
>>>
>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>> The
>>> project has been integrated like a component but from my point of view,
>>> it
>>> should be integrated (for consistency reason) like a Dataformat. Because
>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>>> into another format (Collection of Java objects)
>>>
>>> E.g.
>>>
>>> DataFormat flatpack = new
>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>
>>> from("file:///c:/test")
>>> .unmarshall(flatpack)
>>> .to("whatever you want")
>>>
>>> Regards,
>>>
>>> Charles
>>>
>>>
>>>
>>> James.Strachan wrote:
>>>>
>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>> but there's a basic fixed-width reader example there for the curious
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> I've raised a JIRA to track this...
>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>> like
>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>
>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>
>>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>>> a nice design UI that does this with Camel...
>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>
>>>>>> but it'd be nice to have an OSS option :)
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>>
>>>
>>>
>>> -----
>>> Enterprise Architect
>>>
>>> Xpectis
>>> 12, route d'Esch
>>> L-1470 Luxembourg
>>>
>>> Phone +352 25 10 70 470
>>> Mobile +352 621 45 36 22
>>>
>>> e-mail : cmoulliard@xpectis.com
>>> web site : www.xpectis.com
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18482019.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
This is more and more strange and curious. I see that the content of the
fixedlength file is send in one message and that the trailer is there but
not the header

4320438 [Thread: 1 org.apache.camel.component.file.FileComponent@97f621]
INFO  com.bravura.babel.server.common.Convert  - Trailer
4320438 [Thread: 1 org.apache.camel.component.file.FileComponent@97f621]
INFO  com.server.common.Convert  - COLUMN NAME: TRAILERDATA VALUE:
00032700000001190620071714
4320438 [Thread: 1 org.apache.camel.component.file.FileComponent@97f621]
INFO  com.server.common.Convert  -
===========================================================================
4320453 [Thread: 1 org.apache.camel.component.file.FileComponent@97f621]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/4014-1216223898765/2-1 -> to9
To[activemq:queue:cutas.in.queue] InOnly
Properties:{CamelCauseException=null, org.apache.camel.bean.BeanHolder=bean:
convert} Headers:{camelFlatpackCounter=0, org.apache.camel.MethodName=null}
BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0

When I test the same file + xml in a separate unit test, they work
perfectly. Is it possible that there is a bug in the existing component ?


cmoulliard wrote:
> 
> It is not required to make a mistake because Flatpack can process the
> content without header/trailer in the message exchanged. This is not
> blocking.
> 
> Nevertheless, something which is strange is that in the log we can see the
> following header/trailer objects   :
> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
> 
> I don't know if the objects contain something if they are reachable but
> .... ?
> 
> Regards,
> 
> Charles
> 
> 
> Claus Ibsen wrote:
>> 
>> Hi
>> 
>> To make a mistake in this definition, so flatpack will skip the line
>> 
>> From:
>> <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>>>> 		<COLUMN name="INDICATOR" length="3" />
>>>> 		<COLUMN name="HEADERDATA" length="26" />
>>>> 	</RECORD>
>> 
>> To:
>> <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>>>> 		<COLUMN name="INDICATOR" length="3" />
>>>> 		<COLUMN name="HEADERDATA" length="26" />
>>>> 		<COLUMN name="DUMMY " length="99" />
>>>> 	</RECORD>
>> 
>> Then maybe you can continue. The same for the trail as well.
>> 
>> I don't think there is time left in Camel 1.4 to be able to improve the
>> flatpack component to be able to handle header/trails currently. 
>> 
>> 
>> Med venlig hilsen
>>  
>> Claus Ibsen
>> ......................................
>> Silverbullet
>> Skovsgårdsvænget 21
>> 8362 Hørning
>> Tlf. +45 2962 7576
>> Web: www.silverbullet.dk
>> 
>> -----Original Message-----
>> From: cmoulliard [mailto:cmoulliard@gmail.com] 
>> Sent: 16. juli 2008 17:23
>> To: camel-user@activemq.apache.org
>> Subject: RE: Component to read FixedLength file ?
>> 
>> 
>> Hi,
>> 
>> I don't understand what you propose.
>> 
>> Regards,
>> 
>> Charles Moulliard
>> 
>> 
>> 
>> Claus Ibsen wrote:
>>> 
>>> Hi Charles
>>> 
>>> A workaround for your current problem is to make a mistake in the
>>> header/trailer definition and flatpack will skip it, at least in my
>>> current unit test.
>>> 
>>> Med venlig hilsen
>>>  
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>> 
>>> -----Original Message-----
>>> From: cmoulliard [mailto:cmoulliard@gmail.com] 
>>> Sent: 16. juli 2008 11:11
>>> To: camel-user@activemq.apache.org
>>> Subject: Re: Component to read FixedLength file ?
>>> 
>>> 
>>> The error comes from the fact that the convertBody(Map.class) does not
>>> take
>>> into account that the first record and the last one are the
>>> header/trailer
>>> of the message. 
>>> 
>>> In this case, three different map have to be created :
>>> - Header,
>>> - Body 
>>> - Trailer
>>> 
>>> The property of the message and attribute of the DSL/URI must be adapted
>>> to
>>> use a new keywork -> flatpack:fixedHeader instead of flatpack:fixed to
>>> inform the Body converter How to transform the
>>> net.sf.flatpack.DefaultDataSet into their three Maps
>>> 
>>> Regards,
>>> 
>>> Charles
>>> 
>>>  
>>> 
>>> cmoulliard wrote:
>>>> 
>>>> I have a new error when I use the following routes. 
>>>> 
>>>> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>>>>                
>>>> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>>>>                
>>>>                
>>>>        
>>>> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>>>>                 .convertBodyTo(Map.class)
>>>>                 .to("activemq:queue:cutas.in.queue");
>>>> 
>>>> and xml file
>>>> 
>>>> <?xml version='1.0'?>
>>>> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
>>>> 
>>>> <PZMAP>
>>>> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>>>> 		<COLUMN name="INDICATOR" length="3" />
>>>> 		<COLUMN name="HEADERDATA" length="26" />
>>>> 	</RECORD>
>>>> 	<COLUMN name="RECORDTYPE" length="1" />
>>>> 	<COLUMN name="SYSTEMID" length="3" />
>>>> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
>>>> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
>>>> 	<COLUMN name="FUND_CODE" length="5" />
>>>> 	<COLUMN name="PRODUCT_CODE" length="5" />
>>>> 	<COLUMN name="PRODUCT_TYPE" length="4" />
>>>> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
>>>> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
>>>> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
>>>> 	<COLUMN name="FUND_TYPE" length="1" />
>>>> 	<COLUMN name="UNIT_SIGN" length="1" />
>>>> 	<COLUMN name="DATA" length="412" />
>>>> 	
>>>> 	<RECORD id="trailer" startPosition="1" endPosition="3"
>>>> indicator="FBT">
>>>> 		<COLUMN name="INDICATOR" length="3" />
>>>> 		<COLUMN name="TRAILERDATA" length="26" />
>>>> 	</RECORD>
>>>> 
>>>> </PZMAP> 
>>>> 
>>>> Remark : In my case, we have a header / Trailer in the message.
>>>> 
>>>> Error reported :
>>>> 
>>>> 
>>>> 43org.apache.camel.RuntimeCamelException:
>>>> java.util.NoSuchElementException: Column [recordtype] does not exist,
>>>> check case/spelling. key:trailer
>>>> 	at
>>>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
>>>> 	at
>>>> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
>>>> 	at
>>>> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
>>>> 	at
>>>> org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
>>>> 	at
>>>> org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
>>>> 	at
>>>> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>>> 	at
>>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>>> 	at
>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>>>> 	at
>>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>>>> 	at
>>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>>>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>>>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
>>>> 	at
>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>>>> 	at
>>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>>>> 	at
>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>>> 	at
>>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>>> 	at
>>>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>>>> 	at
>>>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>>>> 	at
>>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>>>> 	at
>>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
>>>> 	at
>>>> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
>>>> 	at
>>>> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
>>>> 	at
>>>> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
>>>> 	at
>>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>>> 	at
>>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>>> 	at
>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>>>> 	at
>>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>>>> 	at
>>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>>>> 	at
>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>>>> 	at
>>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>>>> 	at
>>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>>> 	at
>>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>>> 	at
>>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>>> 	at
>>>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>>>> 	at
>>>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>>>> 	at
>>>> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
>>>> 	at
>>>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
>>>> 	at
>>>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
>>>> 	at
>>>> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
>>>> 	at
>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
>>>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>>>> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown
>>>> Source)
>>>> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>>>> 	at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
>>>> Source)
>>>> 	at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
>>>> Source)
>>>> 	at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>>>> Source)
>>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>>> Source)
>>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>>> 	at java.lang.Thread.run(Unknown Source)
>>>> Caused by: java.util.NoSuchElementException: Column [recordtype] does
>>>> not
>>>> exist, check case/spelling. key:trailer
>>>> 	at
>>>> net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
>>>> 	at
>>>> net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
>>>> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
>>>> 	at
>>>> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
>>>> 	at
>>>> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
>>>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>>> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>>>> 	at java.lang.reflect.Method.invoke(Unknown Source)
>>>> 	at
>>>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
>>>> 	... 55 more
>>>> 
>>>> Content of the message :
>>>> 
>>>> 4349422 [Thread: 1
>>>> org.apache.camel.component.file.FileComponent@1b78953]
>>>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>>>> ID-dell-charles/1446-1216195452000/2-0 -> to7
>>>> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
>>>> InOnly Properties:{CamelCauseException=null,
>>>> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
>>>> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
>>>> exclusive valid],
>>>> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
>>>> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
>>>> Body:HBT00032700000327190620071714
>>>> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
>>>> +0000000010000000000009076000000138137A000090760EUR  
>>>> GBPSS808800714636FIEU
>>>> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
>>>> 00001522000 000000000000000O                  0000000000000000                                        
>>>> 20070907155200                                                                             
>>>> X
>>>> C00111a30571b170007B000005        APAA          200805271554  
>>>> +0000000001835910000003941700000039417A000332147USD  
>>>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>>>> 000000000000000000000000000 00000000000 000000000000000B                 
>>>> 0000000000000000                                         20070619101000                                                                             
>>>> X
>>>> C00111a30571a600006B000005        APAA          200805271554  
>>>> +0000000001855550000003941700000039417A000052147GBP  
>>>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>>>> 000000000000000000000000000 00000000000 000000000000000X                 
>>>> 0000000000000000                                         20070619101000                                                                             
>>>> X
>>>> C00111a30571c2a0009B000005        APAA          200805271554  
>>>> +0000000001855550000003941700000039417A000052147GBP  
>>>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>>>> 000000000000000000000000000 00000000000 000000000000000                  
>>>> 0000000000000000                                         20070619101000                                                                             
>>>> X
>>>> FBT00032700000001190620071714
>>>> 
>>>> Info displayed on the console :
>>>> 
>>>> 4349609 [Thread: 1
>>>> org.apache.camel.component.file.FileComponent@1b78953]
>>>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>>>> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
>>>> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
>>>> null]]), sendTo(Endpoint[seda:errors]),
>>>> RedeliveryPolicy[maximumRedeliveries=1]],
>>>> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
>>>> sendTo(Endpoint[seda:errors]),
>>>> RedeliveryPolicy[maximumRedeliveries=1]]])]
>>>> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
>>>> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
>>>> Rows:6
>>>> Position:0
>>>> Conversion
>>>> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
>>>> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
>>>> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
>>>> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
>>>> , Name:SYSTEMID Length:3 Start:0 End:0
>>>> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
>>>> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
>>>> , Name:FUND_CODE Length:5 Start:0 End:0
>>>> , Name:PRODUCT_CODE Length:5 Start:0 End:0
>>>> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
>>>> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
>>>> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
>>>> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
>>>> , Name:FUND_TYPE Length:1 Start:0 End:0
>>>> , Name:UNIT_SIGN Length:1 Start:0 End:0
>>>> , Name:DATA Length:412 Start:0 End:0
>>>> ]
>>>> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10,
>>>> systemid=1,
>>>> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0,
>>>> data=12,
>>>> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
>>>> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
>>>> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
>>>> 
>>>> Regards,
>>>> 
>>>> Charles
>>>> 
>>>> 
>>>> 
>>>> James.Strachan wrote:
>>>>> 
>>>>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>>>>> to use. The downside is the data format kinda has to turn the DataSet
>>>>> into a List of Maps to make it easier to work with; but at least the
>>>>> routes are a bit simpler. e.g. this is from the
>>>>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>>>>> currently)
>>>>> 
>>>>>   <camelContext
>>>>> xmlns="http://activemq.apache.org/camel/schema/spring">
>>>>>     <route>
>>>>>       <from uri="file://src/test/data/delim?noop=true"/>
>>>>>       <unmarshal ref="delimitedFormat"/>
>>>>>       <splitter>
>>>>>         <simple>in.body</simple>
>>>>>         <convertBodyTo type="java.util.Map"/>
>>>>>         <to uri="mock:results"/>
>>>>>       </splitter>
>>>>>     </route>
>>>>>   </camelContext>
>>>>> 
>>>>>   <bean id="delimitedFormat"
>>>>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>>>>     <property name="definition"
>>>>> value="INVENTORY-Delimited.pzmap.xml"/>
>>>>>   </bean>
>>>>> 
>>>>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>>>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>>>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>>>>> provides an iterator over rows of the file; I wasn't sure if this
>>>>>> properly matched the dataformat model of marshalling or unmarshalling
>>>>>> a payload (which currently assumes that unmarshalling generates a
>>>>>> single result).
>>>>>>
>>>>>> Maybe we need to enhance the dataformat concept to work with this use
>>>>>> case too?
>>>>>>
>>>>>> So maybe if we want to process each row of a file we do something
>>>>>> like...
>>>>>>
>>>>>> from("file://whatnot").
>>>>>> 
>>>>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>>>>  split().body().
>>>>>>  to("whatever");
>>>>>>
>>>>>> where the split().body() would do the iteration over each DataSet?
>>>>>>
>>>>>>
>>>>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>>>>> Hi
>>>>>>>
>>>>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>>>>> created it it works as if it is a dataformat. You just have to do it
>>>>>>> in
>>>>>>> to..from from..to steps.
>>>>>>>
>>>>>>>
>>>>>>> from("file://c:/test").to("flatpack:foo:
>>>>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>>>>
>>>>>>> from("flatpack:foo").to("whatever you want");
>>>>>>>
>>>>>>> As a component it gets more attention, as people can find it on the
>>>>>>> component list. The dataformats are kinda hidden in the wiki
>>>>>>> documentation.
>>>>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>>>>
>>>>>>> Med venlig hilsen
>>>>>>>
>>>>>>> Claus Ibsen
>>>>>>> ......................................
>>>>>>> Silverbullet
>>>>>>> Skovsgårdsvænget 21
>>>>>>> 8362 Hørning
>>>>>>> Tlf. +45 2962 7576
>>>>>>> Web: www.silverbullet.dk
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>>>>> Sent: 15. juli 2008 09:32
>>>>>>> To: camel-user@activemq.apache.org
>>>>>>> Subject: Re: Component to read FixedLength file ?
>>>>>>>
>>>>>>>
>>>>>>> Many thanks.
>>>>>>>
>>>>>>> I will test the integration of this component within Camel.
>>>>>>>
>>>>>>> Remark : I'm a little confused about how Flatpack has been
>>>>>>> integrated.
>>>>>>> The
>>>>>>> project has been integrated like a component but from my point of
>>>>>>> view,
>>>>>>> it
>>>>>>> should be integrated (for consistency reason) like a Dataformat.
>>>>>>> Because
>>>>>>> like the other framework/technology implemented in Camel (JAXB,
>>>>>>> Artix,
>>>>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>>>>> ...)
>>>>>>> into another format (Collection of Java objects)
>>>>>>>
>>>>>>> E.g.
>>>>>>>
>>>>>>> DataFormat flatpack = new
>>>>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>>>>
>>>>>>> from("file:///c:/test")
>>>>>>> .unmarshall(flatpack)
>>>>>>> .to("whatever you want")
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Charles
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> James.Strachan wrote:
>>>>>>>>
>>>>>>>> BTW trunk has an early spike of a flatpack component. Needs more
>>>>>>>> work,
>>>>>>>> but there's a basic fixed-width reader example there for the
>>>>>>>> curious
>>>>>>>>
>>>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>>>> I've raised a JIRA to track this...
>>>>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>>>>
>>>>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>>>>> like
>>>>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>>>>
>>>>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>>>>
>>>>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>>>>> with
>>>>>>>>>> a nice design UI that does this with Camel...
>>>>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>>>>
>>>>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> James
>>>>>>>>>> -------
>>>>>>>>>> http://macstrac.blogspot.com/
>>>>>>>>>>
>>>>>>>>>> Open Source Integration
>>>>>>>>>> http://open.iona.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> James
>>>>>>>>> -------
>>>>>>>>> http://macstrac.blogspot.com/
>>>>>>>>>
>>>>>>>>> Open Source Integration
>>>>>>>>> http://open.iona.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> James
>>>>>>>> -------
>>>>>>>> http://macstrac.blogspot.com/
>>>>>>>>
>>>>>>>> Open Source Integration
>>>>>>>> http://open.iona.com
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -----
>>>>>>> Enterprise Architect
>>>>>>>
>>>>>>> Xpectis
>>>>>>> 12, route d'Esch
>>>>>>> L-1470 Luxembourg
>>>>>>>
>>>>>>> Phone +352 25 10 70 470
>>>>>>> Mobile +352 621 45 36 22
>>>>>>>
>>>>>>> e-mail : cmoulliard@xpectis.com
>>>>>>> web site : www.xpectis.com
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>> 
>>>>> 
>>>>> 
>>>>> -- 
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>> 
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> -----
>>> Enterprise Architect
>>> 
>>> Xpectis
>>> 12, route d'Esch
>>> L-1470 Luxembourg
>>> 
>>> Phone +352 25 10 70 470
>>> Mobile +352 621 45 36 22
>>> 
>>> e-mail : cmoulliard@xpectis.com
>>> web site : www.xpectis.com 
>>> -- 
>>> View this message in context:
>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18483512.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>> 
>>> 
>>> 
>> 
>> 
>> -----
>> Enterprise Architect
>> 
>> Xpectis
>> 12, route d'Esch
>> L-1470 Luxembourg
>> 
>> Phone +352 25 10 70 470
>> Mobile +352 621 45 36 22
>> 
>> e-mail : cmoulliard@xpectis.com
>> web site : www.xpectis.com 
>> -- 
>> View this message in context:
>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18489977.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>> 
>> 
>> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18490857.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
It is not required to make a mistake because Flatpack can process the content
without header/trailer in the message exchanged. This is not blocking.

Nevertheless, something which is strange is that in the log we can see the
following header/trailer objects   :
XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}

I don't know if the objects contain something if they are reachable but ....
?

Regards,

Charles


Claus Ibsen wrote:
> 
> Hi
> 
> To make a mistake in this definition, so flatpack will skip the line
> 
> From:
> <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>>> 		<COLUMN name="INDICATOR" length="3" />
>>> 		<COLUMN name="HEADERDATA" length="26" />
>>> 	</RECORD>
> 
> To:
> <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>>> 		<COLUMN name="INDICATOR" length="3" />
>>> 		<COLUMN name="HEADERDATA" length="26" />
>>> 		<COLUMN name="DUMMY " length="99" />
>>> 	</RECORD>
> 
> Then maybe you can continue. The same for the trail as well.
> 
> I don't think there is time left in Camel 1.4 to be able to improve the
> flatpack component to be able to handle header/trails currently. 
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 16. juli 2008 17:23
> To: camel-user@activemq.apache.org
> Subject: RE: Component to read FixedLength file ?
> 
> 
> Hi,
> 
> I don't understand what you propose.
> 
> Regards,
> 
> Charles Moulliard
> 
> 
> 
> Claus Ibsen wrote:
>> 
>> Hi Charles
>> 
>> A workaround for your current problem is to make a mistake in the
>> header/trailer definition and flatpack will skip it, at least in my
>> current unit test.
>> 
>> Med venlig hilsen
>>  
>> Claus Ibsen
>> ......................................
>> Silverbullet
>> Skovsgårdsvænget 21
>> 8362 Hørning
>> Tlf. +45 2962 7576
>> Web: www.silverbullet.dk
>> 
>> -----Original Message-----
>> From: cmoulliard [mailto:cmoulliard@gmail.com] 
>> Sent: 16. juli 2008 11:11
>> To: camel-user@activemq.apache.org
>> Subject: Re: Component to read FixedLength file ?
>> 
>> 
>> The error comes from the fact that the convertBody(Map.class) does not
>> take
>> into account that the first record and the last one are the
>> header/trailer
>> of the message. 
>> 
>> In this case, three different map have to be created :
>> - Header,
>> - Body 
>> - Trailer
>> 
>> The property of the message and attribute of the DSL/URI must be adapted
>> to
>> use a new keywork -> flatpack:fixedHeader instead of flatpack:fixed to
>> inform the Body converter How to transform the
>> net.sf.flatpack.DefaultDataSet into their three Maps
>> 
>> Regards,
>> 
>> Charles
>> 
>>  
>> 
>> cmoulliard wrote:
>>> 
>>> I have a new error when I use the following routes. 
>>> 
>>> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>>>                
>>> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>>>                
>>>                
>>>        
>>> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>>>                 .convertBodyTo(Map.class)
>>>                 .to("activemq:queue:cutas.in.queue");
>>> 
>>> and xml file
>>> 
>>> <?xml version='1.0'?>
>>> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
>>> 
>>> <PZMAP>
>>> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>>> 		<COLUMN name="INDICATOR" length="3" />
>>> 		<COLUMN name="HEADERDATA" length="26" />
>>> 	</RECORD>
>>> 	<COLUMN name="RECORDTYPE" length="1" />
>>> 	<COLUMN name="SYSTEMID" length="3" />
>>> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
>>> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
>>> 	<COLUMN name="FUND_CODE" length="5" />
>>> 	<COLUMN name="PRODUCT_CODE" length="5" />
>>> 	<COLUMN name="PRODUCT_TYPE" length="4" />
>>> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
>>> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
>>> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
>>> 	<COLUMN name="FUND_TYPE" length="1" />
>>> 	<COLUMN name="UNIT_SIGN" length="1" />
>>> 	<COLUMN name="DATA" length="412" />
>>> 	
>>> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
>>> 		<COLUMN name="INDICATOR" length="3" />
>>> 		<COLUMN name="TRAILERDATA" length="26" />
>>> 	</RECORD>
>>> 
>>> </PZMAP> 
>>> 
>>> Remark : In my case, we have a header / Trailer in the message.
>>> 
>>> Error reported :
>>> 
>>> 
>>> 43org.apache.camel.RuntimeCamelException:
>>> java.util.NoSuchElementException: Column [recordtype] does not exist,
>>> check case/spelling. key:trailer
>>> 	at
>>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
>>> 	at
>>> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
>>> 	at
>>> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
>>> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
>>> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
>>> 	at
>>> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>> 	at
>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>> 	at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>>> 	at
>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>>> 	at
>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
>>> 	at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>>> 	at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>>> 	at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>> 	at
>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>> 	at
>>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>>> 	at
>>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>>> 	at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>>> 	at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
>>> 	at
>>> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
>>> 	at
>>> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
>>> 	at
>>> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
>>> 	at
>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>> 	at
>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>> 	at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>>> 	at
>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>>> 	at
>>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>>> 	at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>>> 	at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>>> 	at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>>> 	at
>>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>>> 	at
>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>>> 	at
>>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>>> 	at
>>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>>> 	at
>>> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
>>> 	at
>>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
>>> 	at
>>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
>>> 	at
>>> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
>>> 	at
>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
>>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>>> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown
>>> Source)
>>> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>>> 	at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
>>> Source)
>>> 	at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
>>> Source)
>>> 	at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>>> Source)
>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>>> Source)
>>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>> 	at java.lang.Thread.run(Unknown Source)
>>> Caused by: java.util.NoSuchElementException: Column [recordtype] does
>>> not
>>> exist, check case/spelling. key:trailer
>>> 	at
>>> net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
>>> 	at
>>> net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
>>> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
>>> 	at
>>> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
>>> 	at
>>> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
>>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>>> 	at java.lang.reflect.Method.invoke(Unknown Source)
>>> 	at
>>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
>>> 	... 55 more
>>> 
>>> Content of the message :
>>> 
>>> 4349422 [Thread: 1
>>> org.apache.camel.component.file.FileComponent@1b78953]
>>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>>> ID-dell-charles/1446-1216195452000/2-0 -> to7
>>> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
>>> InOnly Properties:{CamelCauseException=null,
>>> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
>>> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
>>> exclusive valid],
>>> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
>>> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
>>> Body:HBT00032700000327190620071714
>>> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
>>> +0000000010000000000009076000000138137A000090760EUR  
>>> GBPSS808800714636FIEU
>>> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
>>> 00001522000 000000000000000O                  0000000000000000                                        
>>> 20070907155200                                                                             
>>> X
>>> C00111a30571b170007B000005        APAA          200805271554  
>>> +0000000001835910000003941700000039417A000332147USD  
>>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>>> 000000000000000000000000000 00000000000 000000000000000B                 
>>> 0000000000000000                                         20070619101000                                                                             
>>> X
>>> C00111a30571a600006B000005        APAA          200805271554  
>>> +0000000001855550000003941700000039417A000052147GBP  
>>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>>> 000000000000000000000000000 00000000000 000000000000000X                 
>>> 0000000000000000                                         20070619101000                                                                             
>>> X
>>> C00111a30571c2a0009B000005        APAA          200805271554  
>>> +0000000001855550000003941700000039417A000052147GBP  
>>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>>> 000000000000000000000000000 00000000000 000000000000000                  
>>> 0000000000000000                                         20070619101000                                                                             
>>> X
>>> FBT00032700000001190620071714
>>> 
>>> Info displayed on the console :
>>> 
>>> 4349609 [Thread: 1
>>> org.apache.camel.component.file.FileComponent@1b78953]
>>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>>> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
>>> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
>>> null]]), sendTo(Endpoint[seda:errors]),
>>> RedeliveryPolicy[maximumRedeliveries=1]],
>>> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
>>> sendTo(Endpoint[seda:errors]),
>>> RedeliveryPolicy[maximumRedeliveries=1]]])]
>>> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
>>> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
>>> Rows:6
>>> Position:0
>>> Conversion
>>> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
>>> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
>>> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
>>> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
>>> , Name:SYSTEMID Length:3 Start:0 End:0
>>> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
>>> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
>>> , Name:FUND_CODE Length:5 Start:0 End:0
>>> , Name:PRODUCT_CODE Length:5 Start:0 End:0
>>> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
>>> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
>>> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
>>> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
>>> , Name:FUND_TYPE Length:1 Start:0 End:0
>>> , Name:UNIT_SIGN Length:1 Start:0 End:0
>>> , Name:DATA Length:412 Start:0 End:0
>>> ]
>>> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10,
>>> systemid=1,
>>> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0,
>>> data=12,
>>> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
>>> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
>>> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
>>> 
>>> Regards,
>>> 
>>> Charles
>>> 
>>> 
>>> 
>>> James.Strachan wrote:
>>>> 
>>>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>>>> to use. The downside is the data format kinda has to turn the DataSet
>>>> into a List of Maps to make it easier to work with; but at least the
>>>> routes are a bit simpler. e.g. this is from the
>>>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>>>> currently)
>>>> 
>>>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>>>     <route>
>>>>       <from uri="file://src/test/data/delim?noop=true"/>
>>>>       <unmarshal ref="delimitedFormat"/>
>>>>       <splitter>
>>>>         <simple>in.body</simple>
>>>>         <convertBodyTo type="java.util.Map"/>
>>>>         <to uri="mock:results"/>
>>>>       </splitter>
>>>>     </route>
>>>>   </camelContext>
>>>> 
>>>>   <bean id="delimitedFormat"
>>>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>>>   </bean>
>>>> 
>>>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>>>> provides an iterator over rows of the file; I wasn't sure if this
>>>>> properly matched the dataformat model of marshalling or unmarshalling
>>>>> a payload (which currently assumes that unmarshalling generates a
>>>>> single result).
>>>>>
>>>>> Maybe we need to enhance the dataformat concept to work with this use
>>>>> case too?
>>>>>
>>>>> So maybe if we want to process each row of a file we do something
>>>>> like...
>>>>>
>>>>> from("file://whatnot").
>>>>> 
>>>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>>>  split().body().
>>>>>  to("whatever");
>>>>>
>>>>> where the split().body() would do the iteration over each DataSet?
>>>>>
>>>>>
>>>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>>>> Hi
>>>>>>
>>>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>>>> created it it works as if it is a dataformat. You just have to do it
>>>>>> in
>>>>>> to..from from..to steps.
>>>>>>
>>>>>>
>>>>>> from("file://c:/test").to("flatpack:foo:
>>>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>>>
>>>>>> from("flatpack:foo").to("whatever you want");
>>>>>>
>>>>>> As a component it gets more attention, as people can find it on the
>>>>>> component list. The dataformats are kinda hidden in the wiki
>>>>>> documentation.
>>>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>>>
>>>>>> Med venlig hilsen
>>>>>>
>>>>>> Claus Ibsen
>>>>>> ......................................
>>>>>> Silverbullet
>>>>>> Skovsgårdsvænget 21
>>>>>> 8362 Hørning
>>>>>> Tlf. +45 2962 7576
>>>>>> Web: www.silverbullet.dk
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>>>> Sent: 15. juli 2008 09:32
>>>>>> To: camel-user@activemq.apache.org
>>>>>> Subject: Re: Component to read FixedLength file ?
>>>>>>
>>>>>>
>>>>>> Many thanks.
>>>>>>
>>>>>> I will test the integration of this component within Camel.
>>>>>>
>>>>>> Remark : I'm a little confused about how Flatpack has been
>>>>>> integrated.
>>>>>> The
>>>>>> project has been integrated like a component but from my point of
>>>>>> view,
>>>>>> it
>>>>>> should be integrated (for consistency reason) like a Dataformat.
>>>>>> Because
>>>>>> like the other framework/technology implemented in Camel (JAXB,
>>>>>> Artix,
>>>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>>>> ...)
>>>>>> into another format (Collection of Java objects)
>>>>>>
>>>>>> E.g.
>>>>>>
>>>>>> DataFormat flatpack = new
>>>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>>>
>>>>>> from("file:///c:/test")
>>>>>> .unmarshall(flatpack)
>>>>>> .to("whatever you want")
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Charles
>>>>>>
>>>>>>
>>>>>>
>>>>>> James.Strachan wrote:
>>>>>>>
>>>>>>> BTW trunk has an early spike of a flatpack component. Needs more
>>>>>>> work,
>>>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>>>
>>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>>> I've raised a JIRA to track this...
>>>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>>>
>>>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>>>> like
>>>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>>>
>>>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>>>
>>>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>>>> with
>>>>>>>>> a nice design UI that does this with Camel...
>>>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>>>
>>>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> James
>>>>>>>>> -------
>>>>>>>>> http://macstrac.blogspot.com/
>>>>>>>>>
>>>>>>>>> Open Source Integration
>>>>>>>>> http://open.iona.com
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> James
>>>>>>>> -------
>>>>>>>> http://macstrac.blogspot.com/
>>>>>>>>
>>>>>>>> Open Source Integration
>>>>>>>> http://open.iona.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> -----
>>>>>> Enterprise Architect
>>>>>>
>>>>>> Xpectis
>>>>>> 12, route d'Esch
>>>>>> L-1470 Luxembourg
>>>>>>
>>>>>> Phone +352 25 10 70 470
>>>>>> Mobile +352 621 45 36 22
>>>>>>
>>>>>> e-mail : cmoulliard@xpectis.com
>>>>>> web site : www.xpectis.com
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>> 
>>>> Open Source Integration
>>>> http://open.iona.com
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> -----
>> Enterprise Architect
>> 
>> Xpectis
>> 12, route d'Esch
>> L-1470 Luxembourg
>> 
>> Phone +352 25 10 70 470
>> Mobile +352 621 45 36 22
>> 
>> e-mail : cmoulliard@xpectis.com
>> web site : www.xpectis.com 
>> -- 
>> View this message in context:
>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18483512.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>> 
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18489977.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18490512.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Hi

To make a mistake in this definition, so flatpack will skip the line

From:
<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>> 		<COLUMN name="INDICATOR" length="3" />
>> 		<COLUMN name="HEADERDATA" length="26" />
>> 	</RECORD>

To:
<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>> 		<COLUMN name="INDICATOR" length="3" />
>> 		<COLUMN name="HEADERDATA" length="26" />
>> 		<COLUMN name="DUMMY " length="99" />
>> 	</RECORD>

Then maybe you can continue. The same for the trail as well.

I don't think there is time left in Camel 1.4 to be able to improve the flatpack component to be able to handle header/trails currently. 


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: cmoulliard [mailto:cmoulliard@gmail.com] 
Sent: 16. juli 2008 17:23
To: camel-user@activemq.apache.org
Subject: RE: Component to read FixedLength file ?


Hi,

I don't understand what you propose.

Regards,

Charles Moulliard



Claus Ibsen wrote:
> 
> Hi Charles
> 
> A workaround for your current problem is to make a mistake in the
> header/trailer definition and flatpack will skip it, at least in my
> current unit test.
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 16. juli 2008 11:11
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> 
> The error comes from the fact that the convertBody(Map.class) does not
> take
> into account that the first record and the last one are the header/trailer
> of the message. 
> 
> In this case, three different map have to be created :
> - Header,
> - Body 
> - Trailer
> 
> The property of the message and attribute of the DSL/URI must be adapted
> to
> use a new keywork -> flatpack:fixedHeader instead of flatpack:fixed to
> inform the Body converter How to transform the
> net.sf.flatpack.DefaultDataSet into their three Maps
> 
> Regards,
> 
> Charles
> 
>  
> 
> cmoulliard wrote:
>> 
>> I have a new error when I use the following routes. 
>> 
>> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>>                
>> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>>                
>>                
>>        
>> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>>                 .convertBodyTo(Map.class)
>>                 .to("activemq:queue:cutas.in.queue");
>> 
>> and xml file
>> 
>> <?xml version='1.0'?>
>> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
>> 
>> <PZMAP>
>> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>> 		<COLUMN name="INDICATOR" length="3" />
>> 		<COLUMN name="HEADERDATA" length="26" />
>> 	</RECORD>
>> 	<COLUMN name="RECORDTYPE" length="1" />
>> 	<COLUMN name="SYSTEMID" length="3" />
>> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
>> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
>> 	<COLUMN name="FUND_CODE" length="5" />
>> 	<COLUMN name="PRODUCT_CODE" length="5" />
>> 	<COLUMN name="PRODUCT_TYPE" length="4" />
>> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
>> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
>> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
>> 	<COLUMN name="FUND_TYPE" length="1" />
>> 	<COLUMN name="UNIT_SIGN" length="1" />
>> 	<COLUMN name="DATA" length="412" />
>> 	
>> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
>> 		<COLUMN name="INDICATOR" length="3" />
>> 		<COLUMN name="TRAILERDATA" length="26" />
>> 	</RECORD>
>> 
>> </PZMAP> 
>> 
>> Remark : In my case, we have a header / Trailer in the message.
>> 
>> Error reported :
>> 
>> 
>> 43org.apache.camel.RuntimeCamelException:
>> java.util.NoSuchElementException: Column [recordtype] does not exist,
>> check case/spelling. key:trailer
>> 	at
>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
>> 	at
>> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
>> 	at
>> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
>> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
>> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
>> 	at
>> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>> 	at
>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>> 	at
>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>> 	at
>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>> 	at
>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
>> 	at
>> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
>> 	at
>> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
>> 	at
>> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
>> 	at
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>> 	at
>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>> 	at
>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>> 	at
>> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
>> 	at
>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
>> 	at
>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
>> 	at
>> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
>> 	at
>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
>> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
>> Source)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
>> Source)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>> Source)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> Source)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> 	at java.lang.Thread.run(Unknown Source)
>> Caused by: java.util.NoSuchElementException: Column [recordtype] does not
>> exist, check case/spelling. key:trailer
>> 	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
>> 	at
>> net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
>> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
>> 	at
>> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
>> 	at
>> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>> 	at java.lang.reflect.Method.invoke(Unknown Source)
>> 	at
>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
>> 	... 55 more
>> 
>> Content of the message :
>> 
>> 4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>> ID-dell-charles/1446-1216195452000/2-0 -> to7
>> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
>> InOnly Properties:{CamelCauseException=null,
>> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
>> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
>> exclusive valid],
>> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
>> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
>> Body:HBT00032700000327190620071714
>> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
>> +0000000010000000000009076000000138137A000090760EUR  
>> GBPSS808800714636FIEU
>> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
>> 00001522000 000000000000000O                  0000000000000000                                        
>> 20070907155200                                                                             
>> X
>> C00111a30571b170007B000005        APAA          200805271554  
>> +0000000001835910000003941700000039417A000332147USD  
>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>> 000000000000000000000000000 00000000000 000000000000000B                 
>> 0000000000000000                                         20070619101000                                                                             
>> X
>> C00111a30571a600006B000005        APAA          200805271554  
>> +0000000001855550000003941700000039417A000052147GBP  
>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>> 000000000000000000000000000 00000000000 000000000000000X                 
>> 0000000000000000                                         20070619101000                                                                             
>> X
>> C00111a30571c2a0009B000005        APAA          200805271554  
>> +0000000001855550000003941700000039417A000052147GBP  
>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>> 000000000000000000000000000 00000000000 000000000000000                  
>> 0000000000000000                                         20070619101000                                                                             
>> X
>> FBT00032700000001190620071714
>> 
>> Info displayed on the console :
>> 
>> 4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
>> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
>> null]]), sendTo(Endpoint[seda:errors]),
>> RedeliveryPolicy[maximumRedeliveries=1]],
>> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
>> sendTo(Endpoint[seda:errors]),
>> RedeliveryPolicy[maximumRedeliveries=1]]])]
>> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
>> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
>> Rows:6
>> Position:0
>> Conversion
>> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
>> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
>> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
>> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
>> , Name:SYSTEMID Length:3 Start:0 End:0
>> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
>> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
>> , Name:FUND_CODE Length:5 Start:0 End:0
>> , Name:PRODUCT_CODE Length:5 Start:0 End:0
>> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
>> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
>> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
>> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
>> , Name:FUND_TYPE Length:1 Start:0 End:0
>> , Name:UNIT_SIGN Length:1 Start:0 End:0
>> , Name:DATA Length:412 Start:0 End:0
>> ]
>> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10,
>> systemid=1,
>> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0,
>> data=12,
>> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
>> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
>> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
>> 
>> Regards,
>> 
>> Charles
>> 
>> 
>> 
>> James.Strachan wrote:
>>> 
>>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>>> to use. The downside is the data format kinda has to turn the DataSet
>>> into a List of Maps to make it easier to work with; but at least the
>>> routes are a bit simpler. e.g. this is from the
>>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>>> currently)
>>> 
>>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>>     <route>
>>>       <from uri="file://src/test/data/delim?noop=true"/>
>>>       <unmarshal ref="delimitedFormat"/>
>>>       <splitter>
>>>         <simple>in.body</simple>
>>>         <convertBodyTo type="java.util.Map"/>
>>>         <to uri="mock:results"/>
>>>       </splitter>
>>>     </route>
>>>   </camelContext>
>>> 
>>>   <bean id="delimitedFormat"
>>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>>   </bean>
>>> 
>>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>>> provides an iterator over rows of the file; I wasn't sure if this
>>>> properly matched the dataformat model of marshalling or unmarshalling
>>>> a payload (which currently assumes that unmarshalling generates a
>>>> single result).
>>>>
>>>> Maybe we need to enhance the dataformat concept to work with this use
>>>> case too?
>>>>
>>>> So maybe if we want to process each row of a file we do something
>>>> like...
>>>>
>>>> from("file://whatnot").
>>>> 
>>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>>  split().body().
>>>>  to("whatever");
>>>>
>>>> where the split().body() would do the iteration over each DataSet?
>>>>
>>>>
>>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>>> Hi
>>>>>
>>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>>> created it it works as if it is a dataformat. You just have to do it
>>>>> in
>>>>> to..from from..to steps.
>>>>>
>>>>>
>>>>> from("file://c:/test").to("flatpack:foo:
>>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>>
>>>>> from("flatpack:foo").to("whatever you want");
>>>>>
>>>>> As a component it gets more attention, as people can find it on the
>>>>> component list. The dataformats are kinda hidden in the wiki
>>>>> documentation.
>>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>>
>>>>> Med venlig hilsen
>>>>>
>>>>> Claus Ibsen
>>>>> ......................................
>>>>> Silverbullet
>>>>> Skovsgårdsvænget 21
>>>>> 8362 Hørning
>>>>> Tlf. +45 2962 7576
>>>>> Web: www.silverbullet.dk
>>>>>
>>>>> -----Original Message-----
>>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>>> Sent: 15. juli 2008 09:32
>>>>> To: camel-user@activemq.apache.org
>>>>> Subject: Re: Component to read FixedLength file ?
>>>>>
>>>>>
>>>>> Many thanks.
>>>>>
>>>>> I will test the integration of this component within Camel.
>>>>>
>>>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>>>> The
>>>>> project has been integrated like a component but from my point of
>>>>> view,
>>>>> it
>>>>> should be integrated (for consistency reason) like a Dataformat.
>>>>> Because
>>>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>>> ...)
>>>>> into another format (Collection of Java objects)
>>>>>
>>>>> E.g.
>>>>>
>>>>> DataFormat flatpack = new
>>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>>
>>>>> from("file:///c:/test")
>>>>> .unmarshall(flatpack)
>>>>> .to("whatever you want")
>>>>>
>>>>> Regards,
>>>>>
>>>>> Charles
>>>>>
>>>>>
>>>>>
>>>>> James.Strachan wrote:
>>>>>>
>>>>>> BTW trunk has an early spike of a flatpack component. Needs more
>>>>>> work,
>>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>>
>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>> I've raised a JIRA to track this...
>>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>>
>>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>>> like
>>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>>
>>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>>
>>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>>> with
>>>>>>>> a nice design UI that does this with Camel...
>>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>>
>>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>>
>>>>>>>> --
>>>>>>>> James
>>>>>>>> -------
>>>>>>>> http://macstrac.blogspot.com/
>>>>>>>>
>>>>>>>> Open Source Integration
>>>>>>>> http://open.iona.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> -----
>>>>> Enterprise Architect
>>>>>
>>>>> Xpectis
>>>>> 12, route d'Esch
>>>>> L-1470 Luxembourg
>>>>>
>>>>> Phone +352 25 10 70 470
>>>>> Mobile +352 621 45 36 22
>>>>>
>>>>> e-mail : cmoulliard@xpectis.com
>>>>> web site : www.xpectis.com
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>> 
>>> 
>>> 
>>> -- 
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>> 
>>> Open Source Integration
>>> http://open.iona.com
>>> 
>>> 
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18483512.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18489977.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
Hi,

I don't understand what you propose.

Regards,

Charles Moulliard



Claus Ibsen wrote:
> 
> Hi Charles
> 
> A workaround for your current problem is to make a mistake in the
> header/trailer definition and flatpack will skip it, at least in my
> current unit test.
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 16. juli 2008 11:11
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> 
> The error comes from the fact that the convertBody(Map.class) does not
> take
> into account that the first record and the last one are the header/trailer
> of the message. 
> 
> In this case, three different map have to be created :
> - Header,
> - Body 
> - Trailer
> 
> The property of the message and attribute of the DSL/URI must be adapted
> to
> use a new keywork -> flatpack:fixedHeader instead of flatpack:fixed to
> inform the Body converter How to transform the
> net.sf.flatpack.DefaultDataSet into their three Maps
> 
> Regards,
> 
> Charles
> 
>  
> 
> cmoulliard wrote:
>> 
>> I have a new error when I use the following routes. 
>> 
>> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>>                
>> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>>                
>>                
>>        
>> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>>                 .convertBodyTo(Map.class)
>>                 .to("activemq:queue:cutas.in.queue");
>> 
>> and xml file
>> 
>> <?xml version='1.0'?>
>> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
>> 
>> <PZMAP>
>> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>> 		<COLUMN name="INDICATOR" length="3" />
>> 		<COLUMN name="HEADERDATA" length="26" />
>> 	</RECORD>
>> 	<COLUMN name="RECORDTYPE" length="1" />
>> 	<COLUMN name="SYSTEMID" length="3" />
>> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
>> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
>> 	<COLUMN name="FUND_CODE" length="5" />
>> 	<COLUMN name="PRODUCT_CODE" length="5" />
>> 	<COLUMN name="PRODUCT_TYPE" length="4" />
>> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
>> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
>> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
>> 	<COLUMN name="FUND_TYPE" length="1" />
>> 	<COLUMN name="UNIT_SIGN" length="1" />
>> 	<COLUMN name="DATA" length="412" />
>> 	
>> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
>> 		<COLUMN name="INDICATOR" length="3" />
>> 		<COLUMN name="TRAILERDATA" length="26" />
>> 	</RECORD>
>> 
>> </PZMAP> 
>> 
>> Remark : In my case, we have a header / Trailer in the message.
>> 
>> Error reported :
>> 
>> 
>> 43org.apache.camel.RuntimeCamelException:
>> java.util.NoSuchElementException: Column [recordtype] does not exist,
>> check case/spelling. key:trailer
>> 	at
>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
>> 	at
>> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
>> 	at
>> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
>> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
>> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
>> 	at
>> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>> 	at
>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>> 	at
>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>> 	at
>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>> 	at
>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
>> 	at
>> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
>> 	at
>> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
>> 	at
>> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
>> 	at
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>> 	at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>> 	at
>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
>> 	at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
>> 	at
>> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
>> 	at
>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
>> 	at
>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>> 	at
>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>> 	at
>> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
>> 	at
>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
>> 	at
>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
>> 	at
>> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
>> 	at
>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
>> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
>> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
>> Source)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
>> Source)
>> 	at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
>> Source)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
>> Source)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> 	at java.lang.Thread.run(Unknown Source)
>> Caused by: java.util.NoSuchElementException: Column [recordtype] does not
>> exist, check case/spelling. key:trailer
>> 	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
>> 	at
>> net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
>> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
>> 	at
>> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
>> 	at
>> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>> 	at java.lang.reflect.Method.invoke(Unknown Source)
>> 	at
>> org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
>> 	... 55 more
>> 
>> Content of the message :
>> 
>> 4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>> ID-dell-charles/1446-1216195452000/2-0 -> to7
>> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
>> InOnly Properties:{CamelCauseException=null,
>> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
>> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
>> exclusive valid],
>> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
>> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
>> Body:HBT00032700000327190620071714
>> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
>> +0000000010000000000009076000000138137A000090760EUR  
>> GBPSS808800714636FIEU
>> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
>> 00001522000 000000000000000O                  0000000000000000                                        
>> 20070907155200                                                                             
>> X
>> C00111a30571b170007B000005        APAA          200805271554  
>> +0000000001835910000003941700000039417A000332147USD  
>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>> 000000000000000000000000000 00000000000 000000000000000B                 
>> 0000000000000000                                         20070619101000                                                                             
>> X
>> C00111a30571a600006B000005        APAA          200805271554  
>> +0000000001855550000003941700000039417A000052147GBP  
>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>> 000000000000000000000000000 00000000000 000000000000000X                 
>> 0000000000000000                                         20070619101000                                                                             
>> X
>> C00111a30571c2a0009B000005        APAA          200805271554  
>> +0000000001855550000003941700000039417A000052147GBP  
>> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
>> 000000000000000000000000000 00000000000 000000000000000                  
>> 0000000000000000                                         20070619101000                                                                             
>> X
>> FBT00032700000001190620071714
>> 
>> Info displayed on the console :
>> 
>> 4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
>> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
>> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
>> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
>> null]]), sendTo(Endpoint[seda:errors]),
>> RedeliveryPolicy[maximumRedeliveries=1]],
>> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
>> sendTo(Endpoint[seda:errors]),
>> RedeliveryPolicy[maximumRedeliveries=1]]])]
>> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
>> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
>> Rows:6
>> Position:0
>> Conversion
>> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
>> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
>> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
>> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
>> , Name:SYSTEMID Length:3 Start:0 End:0
>> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
>> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
>> , Name:FUND_CODE Length:5 Start:0 End:0
>> , Name:PRODUCT_CODE Length:5 Start:0 End:0
>> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
>> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
>> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
>> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
>> , Name:FUND_TYPE Length:1 Start:0 End:0
>> , Name:UNIT_SIGN Length:1 Start:0 End:0
>> , Name:DATA Length:412 Start:0 End:0
>> ]
>> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10,
>> systemid=1,
>> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0,
>> data=12,
>> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
>> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
>> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
>> 
>> Regards,
>> 
>> Charles
>> 
>> 
>> 
>> James.Strachan wrote:
>>> 
>>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>>> to use. The downside is the data format kinda has to turn the DataSet
>>> into a List of Maps to make it easier to work with; but at least the
>>> routes are a bit simpler. e.g. this is from the
>>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>>> currently)
>>> 
>>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>>     <route>
>>>       <from uri="file://src/test/data/delim?noop=true"/>
>>>       <unmarshal ref="delimitedFormat"/>
>>>       <splitter>
>>>         <simple>in.body</simple>
>>>         <convertBodyTo type="java.util.Map"/>
>>>         <to uri="mock:results"/>
>>>       </splitter>
>>>     </route>
>>>   </camelContext>
>>> 
>>>   <bean id="delimitedFormat"
>>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>>   </bean>
>>> 
>>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>>> provides an iterator over rows of the file; I wasn't sure if this
>>>> properly matched the dataformat model of marshalling or unmarshalling
>>>> a payload (which currently assumes that unmarshalling generates a
>>>> single result).
>>>>
>>>> Maybe we need to enhance the dataformat concept to work with this use
>>>> case too?
>>>>
>>>> So maybe if we want to process each row of a file we do something
>>>> like...
>>>>
>>>> from("file://whatnot").
>>>> 
>>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>>  split().body().
>>>>  to("whatever");
>>>>
>>>> where the split().body() would do the iteration over each DataSet?
>>>>
>>>>
>>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>>> Hi
>>>>>
>>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>>> created it it works as if it is a dataformat. You just have to do it
>>>>> in
>>>>> to..from from..to steps.
>>>>>
>>>>>
>>>>> from("file://c:/test").to("flatpack:foo:
>>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>>
>>>>> from("flatpack:foo").to("whatever you want");
>>>>>
>>>>> As a component it gets more attention, as people can find it on the
>>>>> component list. The dataformats are kinda hidden in the wiki
>>>>> documentation.
>>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>>
>>>>> Med venlig hilsen
>>>>>
>>>>> Claus Ibsen
>>>>> ......................................
>>>>> Silverbullet
>>>>> Skovsgårdsvænget 21
>>>>> 8362 Hørning
>>>>> Tlf. +45 2962 7576
>>>>> Web: www.silverbullet.dk
>>>>>
>>>>> -----Original Message-----
>>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>>> Sent: 15. juli 2008 09:32
>>>>> To: camel-user@activemq.apache.org
>>>>> Subject: Re: Component to read FixedLength file ?
>>>>>
>>>>>
>>>>> Many thanks.
>>>>>
>>>>> I will test the integration of this component within Camel.
>>>>>
>>>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>>>> The
>>>>> project has been integrated like a component but from my point of
>>>>> view,
>>>>> it
>>>>> should be integrated (for consistency reason) like a Dataformat.
>>>>> Because
>>>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>>> ...)
>>>>> into another format (Collection of Java objects)
>>>>>
>>>>> E.g.
>>>>>
>>>>> DataFormat flatpack = new
>>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>>
>>>>> from("file:///c:/test")
>>>>> .unmarshall(flatpack)
>>>>> .to("whatever you want")
>>>>>
>>>>> Regards,
>>>>>
>>>>> Charles
>>>>>
>>>>>
>>>>>
>>>>> James.Strachan wrote:
>>>>>>
>>>>>> BTW trunk has an early spike of a flatpack component. Needs more
>>>>>> work,
>>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>>
>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>> I've raised a JIRA to track this...
>>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>>
>>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>>> like
>>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>>
>>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>>
>>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>>> with
>>>>>>>> a nice design UI that does this with Camel...
>>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>>
>>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>>
>>>>>>>> --
>>>>>>>> James
>>>>>>>> -------
>>>>>>>> http://macstrac.blogspot.com/
>>>>>>>>
>>>>>>>> Open Source Integration
>>>>>>>> http://open.iona.com
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> -----
>>>>> Enterprise Architect
>>>>>
>>>>> Xpectis
>>>>> 12, route d'Esch
>>>>> L-1470 Luxembourg
>>>>>
>>>>> Phone +352 25 10 70 470
>>>>> Mobile +352 621 45 36 22
>>>>>
>>>>> e-mail : cmoulliard@xpectis.com
>>>>> web site : www.xpectis.com
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>> 
>>> 
>>> 
>>> -- 
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>> 
>>> Open Source Integration
>>> http://open.iona.com
>>> 
>>> 
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18483512.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18489977.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Hi Charles

A workaround for your current problem is to make a mistake in the header/trailer definition and flatpack will skip it, at least in my current unit test.

Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: cmoulliard [mailto:cmoulliard@gmail.com] 
Sent: 16. juli 2008 11:11
To: camel-user@activemq.apache.org
Subject: Re: Component to read FixedLength file ?


The error comes from the fact that the convertBody(Map.class) does not take
into account that the first record and the last one are the header/trailer
of the message. 

In this case, three different map have to be created :
- Header,
- Body 
- Trailer

The property of the message and attribute of the DSL/URI must be adapted to
use a new keywork -> flatpack:fixedHeader instead of flatpack:fixed to
inform the Body converter How to transform the
net.sf.flatpack.DefaultDataSet into their three Maps

Regards,

Charles

 

cmoulliard wrote:
> 
> I have a new error when I use the following routes. 
> 
> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>                
> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>                
>                
>        
> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>                 .convertBodyTo(Map.class)
>                 .to("activemq:queue:cutas.in.queue");
> 
> and xml file
> 
> <?xml version='1.0'?>
> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
> 
> <PZMAP>
> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="HEADERDATA" length="26" />
> 	</RECORD>
> 	<COLUMN name="RECORDTYPE" length="1" />
> 	<COLUMN name="SYSTEMID" length="3" />
> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
> 	<COLUMN name="FUND_CODE" length="5" />
> 	<COLUMN name="PRODUCT_CODE" length="5" />
> 	<COLUMN name="PRODUCT_TYPE" length="4" />
> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
> 	<COLUMN name="FUND_TYPE" length="1" />
> 	<COLUMN name="UNIT_SIGN" length="1" />
> 	<COLUMN name="DATA" length="412" />
> 	
> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="TRAILERDATA" length="26" />
> 	</RECORD>
> 
> </PZMAP> 
> 
> Remark : In my case, we have a header / Trailer in the message.
> 
> Error reported :
> 
> 
> 43org.apache.camel.RuntimeCamelException:
> java.util.NoSuchElementException: Column [recordtype] does not exist,
> check case/spelling. key:trailer
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
> 	at
> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
> 	at
> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
> 	at
> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
> 	at
> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
> 	at
> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
> 	at
> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
> 	at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
> 	at
> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
> 	at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: java.util.NoSuchElementException: Column [recordtype] does not
> exist, check case/spelling. key:trailer
> 	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
> 	at net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 	at java.lang.reflect.Method.invoke(Unknown Source)
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
> 	... 55 more
> 
> Content of the message :
> 
> 4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-0 -> to7
> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
> InOnly Properties:{CamelCauseException=null,
> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
> exclusive valid],
> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
> Body:HBT00032700000327190620071714
> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
> +0000000010000000000009076000000138137A000090760EUR  
> GBPSS808800714636FIEU
> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
> 00001522000 000000000000000O                  0000000000000000                                        
> 20070907155200                                                                             
> X
> C00111a30571b170007B000005        APAA          200805271554  
> +0000000001835910000003941700000039417A000332147USD  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000B                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571a600006B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000X                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571c2a0009B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000                  
> 0000000000000000                                         20070619101000                                                                             
> X
> FBT00032700000001190620071714
> 
> Info displayed on the console :
> 
> 4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
> null]]), sendTo(Endpoint[seda:errors]),
> RedeliveryPolicy[maximumRedeliveries=1]],
> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
> sendTo(Endpoint[seda:errors]), RedeliveryPolicy[maximumRedeliveries=1]]])]
> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
> Rows:6
> Position:0
> Conversion
> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
> , Name:SYSTEMID Length:3 Start:0 End:0
> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
> , Name:FUND_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
> , Name:FUND_TYPE Length:1 Start:0 End:0
> , Name:UNIT_SIGN Length:1 Start:0 End:0
> , Name:DATA Length:412 Start:0 End:0
> ]
> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10, systemid=1,
> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0, data=12,
> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
> 
> Regards,
> 
> Charles
> 
> 
> 
> James.Strachan wrote:
>> 
>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>> to use. The downside is the data format kinda has to turn the DataSet
>> into a List of Maps to make it easier to work with; but at least the
>> routes are a bit simpler. e.g. this is from the
>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>> currently)
>> 
>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>     <route>
>>       <from uri="file://src/test/data/delim?noop=true"/>
>>       <unmarshal ref="delimitedFormat"/>
>>       <splitter>
>>         <simple>in.body</simple>
>>         <convertBodyTo type="java.util.Map"/>
>>         <to uri="mock:results"/>
>>       </splitter>
>>     </route>
>>   </camelContext>
>> 
>>   <bean id="delimitedFormat"
>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>   </bean>
>> 
>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>> provides an iterator over rows of the file; I wasn't sure if this
>>> properly matched the dataformat model of marshalling or unmarshalling
>>> a payload (which currently assumes that unmarshalling generates a
>>> single result).
>>>
>>> Maybe we need to enhance the dataformat concept to work with this use
>>> case too?
>>>
>>> So maybe if we want to process each row of a file we do something
>>> like...
>>>
>>> from("file://whatnot").
>>> 
>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>  split().body().
>>>  to("whatever");
>>>
>>> where the split().body() would do the iteration over each DataSet?
>>>
>>>
>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>> Hi
>>>>
>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>> created it it works as if it is a dataformat. You just have to do it in
>>>> to..from from..to steps.
>>>>
>>>>
>>>> from("file://c:/test").to("flatpack:foo:
>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>
>>>> from("flatpack:foo").to("whatever you want");
>>>>
>>>> As a component it gets more attention, as people can find it on the
>>>> component list. The dataformats are kinda hidden in the wiki
>>>> documentation.
>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>
>>>> Med venlig hilsen
>>>>
>>>> Claus Ibsen
>>>> ......................................
>>>> Silverbullet
>>>> Skovsgårdsvænget 21
>>>> 8362 Hørning
>>>> Tlf. +45 2962 7576
>>>> Web: www.silverbullet.dk
>>>>
>>>> -----Original Message-----
>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>> Sent: 15. juli 2008 09:32
>>>> To: camel-user@activemq.apache.org
>>>> Subject: Re: Component to read FixedLength file ?
>>>>
>>>>
>>>> Many thanks.
>>>>
>>>> I will test the integration of this component within Camel.
>>>>
>>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>>> The
>>>> project has been integrated like a component but from my point of view,
>>>> it
>>>> should be integrated (for consistency reason) like a Dataformat.
>>>> Because
>>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>> ...)
>>>> into another format (Collection of Java objects)
>>>>
>>>> E.g.
>>>>
>>>> DataFormat flatpack = new
>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>
>>>> from("file:///c:/test")
>>>> .unmarshall(flatpack)
>>>> .to("whatever you want")
>>>>
>>>> Regards,
>>>>
>>>> Charles
>>>>
>>>>
>>>>
>>>> James.Strachan wrote:
>>>>>
>>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> I've raised a JIRA to track this...
>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>
>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>> like
>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>
>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>
>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>> with
>>>>>>> a nice design UI that does this with Camel...
>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>
>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>>
>>>>
>>>>
>>>> -----
>>>> Enterprise Architect
>>>>
>>>> Xpectis
>>>> 12, route d'Esch
>>>> L-1470 Luxembourg
>>>>
>>>> Phone +352 25 10 70 470
>>>> Mobile +352 621 45 36 22
>>>>
>>>> e-mail : cmoulliard@xpectis.com
>>>> web site : www.xpectis.com
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>> 
>> 
>> 
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>> 
>> Open Source Integration
>> http://open.iona.com
>> 
>> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18483512.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
The error comes from the fact that the convertBody(Map.class) does not take
into account that the first record and the last one are the header/trailer
of the message. 

In this case, three different map have to be created :
- Header,
- Body 
- Trailer

The property of the message and attribute of the DSL/URI must be adapted to
use a new keywork -> flatpack:fixedHeader instead of flatpack:fixed to
inform the Body converter How to transform the
net.sf.flatpack.DefaultDataSet into their three Maps

Regards,

Charles

 

cmoulliard wrote:
> 
> I have a new error when I use the following routes. 
> 
> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>                
> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>                
>                
>        
> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>                 .convertBodyTo(Map.class)
>                 .to("activemq:queue:cutas.in.queue");
> 
> and xml file
> 
> <?xml version='1.0'?>
> <!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >
> 
> <PZMAP>
> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="HEADERDATA" length="26" />
> 	</RECORD>
> 	<COLUMN name="RECORDTYPE" length="1" />
> 	<COLUMN name="SYSTEMID" length="3" />
> 	<COLUMN name="THIRD_PARTY_REF" length="15" />
> 	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
> 	<COLUMN name="FUND_CODE" length="5" />
> 	<COLUMN name="PRODUCT_CODE" length="5" />
> 	<COLUMN name="PRODUCT_TYPE" length="4" />
> 	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
> 	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
> 	<COLUMN name="TRANSACTION_TYPE" length="2" />
> 	<COLUMN name="FUND_TYPE" length="1" />
> 	<COLUMN name="UNIT_SIGN" length="1" />
> 	<COLUMN name="DATA" length="412" />
> 	
> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="TRAILERDATA" length="26" />
> 	</RECORD>
> 
> </PZMAP> 
> 
> Remark : In my case, we have a header / Trailer in the message.
> 
> Error reported :
> 
> 
> 43org.apache.camel.RuntimeCamelException:
> java.util.NoSuchElementException: Column [recordtype] does not exist,
> check case/spelling. key:trailer
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
> 	at
> org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
> 	at
> org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
> 	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
> 	at
> org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
> 	at
> org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
> 	at
> org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
> 	at
> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
> 	at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
> 	at
> org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
> 	at
> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
> 	at
> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
> 	at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> 	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
> Source)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: java.util.NoSuchElementException: Column [recordtype] does not
> exist, check case/spelling. key:trailer
> 	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
> 	at net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
> 	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
> 	at
> org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 	at java.lang.reflect.Method.invoke(Unknown Source)
> 	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
> 	... 55 more
> 
> Content of the message :
> 
> 4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-0 -> to7
> To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
> InOnly Properties:{CamelCauseException=null,
> org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
> exclusive valid],
> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
> Body:HBT00032700000327190620071714
> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
> +0000000010000000000009076000000138137A000090760EUR  
> GBPSS808800714636FIEU
> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
> 00001522000 000000000000000O                  0000000000000000                                        
> 20070907155200                                                                             
> X
> C00111a30571b170007B000005        APAA          200805271554  
> +0000000001835910000003941700000039417A000332147USD  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000B                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571a600006B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000X                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571c2a0009B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000                  
> 0000000000000000                                         20070619101000                                                                             
> X
> FBT00032700000001190620071714
> 
> Info displayed on the console :
> 
> 4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
> Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
> null]]), sendTo(Endpoint[seda:errors]),
> RedeliveryPolicy[maximumRedeliveries=1]],
> DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
> sendTo(Endpoint[seda:errors]), RedeliveryPolicy[maximumRedeliveries=1]]])]
> InOnly Properties:{} Headers:{camelFlatpackCounter=0}
> BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
> Rows:6
> Position:0
> Conversion
> Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
> java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
> java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
> MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
> , Name:SYSTEMID Length:3 Start:0 End:0
> , Name:THIRD_PARTY_REF Length:15 Start:0 End:0
> , Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
> , Name:FUND_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_CODE Length:5 Start:0 End:0
> , Name:PRODUCT_TYPE Length:4 Start:0 End:0
> , Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
> , Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
> , Name:TRANSACTION_TYPE Length:2 Start:0 End:0
> , Name:FUND_TYPE Length:1 Start:0 End:0
> , Name:UNIT_SIGN Length:1 Start:0 End:0
> , Name:DATA Length:412 Start:0 End:0
> ]
> Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10, systemid=1,
> cutas_account_nr=3, product_code=5, product_type=6, recordtype=0, data=12,
> cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
> XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
> trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}
> 
> Regards,
> 
> Charles
> 
> 
> 
> James.Strachan wrote:
>> 
>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>> to use. The downside is the data format kinda has to turn the DataSet
>> into a List of Maps to make it easier to work with; but at least the
>> routes are a bit simpler. e.g. this is from the
>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>> currently)
>> 
>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>     <route>
>>       <from uri="file://src/test/data/delim?noop=true"/>
>>       <unmarshal ref="delimitedFormat"/>
>>       <splitter>
>>         <simple>in.body</simple>
>>         <convertBodyTo type="java.util.Map"/>
>>         <to uri="mock:results"/>
>>       </splitter>
>>     </route>
>>   </camelContext>
>> 
>>   <bean id="delimitedFormat"
>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>   </bean>
>> 
>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>> provides an iterator over rows of the file; I wasn't sure if this
>>> properly matched the dataformat model of marshalling or unmarshalling
>>> a payload (which currently assumes that unmarshalling generates a
>>> single result).
>>>
>>> Maybe we need to enhance the dataformat concept to work with this use
>>> case too?
>>>
>>> So maybe if we want to process each row of a file we do something
>>> like...
>>>
>>> from("file://whatnot").
>>> 
>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>  split().body().
>>>  to("whatever");
>>>
>>> where the split().body() would do the iteration over each DataSet?
>>>
>>>
>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>> Hi
>>>>
>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>> created it it works as if it is a dataformat. You just have to do it in
>>>> to..from from..to steps.
>>>>
>>>>
>>>> from("file://c:/test").to("flatpack:foo:
>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>
>>>> from("flatpack:foo").to("whatever you want");
>>>>
>>>> As a component it gets more attention, as people can find it on the
>>>> component list. The dataformats are kinda hidden in the wiki
>>>> documentation.
>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>
>>>> Med venlig hilsen
>>>>
>>>> Claus Ibsen
>>>> ......................................
>>>> Silverbullet
>>>> Skovsgårdsvænget 21
>>>> 8362 Hørning
>>>> Tlf. +45 2962 7576
>>>> Web: www.silverbullet.dk
>>>>
>>>> -----Original Message-----
>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>> Sent: 15. juli 2008 09:32
>>>> To: camel-user@activemq.apache.org
>>>> Subject: Re: Component to read FixedLength file ?
>>>>
>>>>
>>>> Many thanks.
>>>>
>>>> I will test the integration of this component within Camel.
>>>>
>>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>>> The
>>>> project has been integrated like a component but from my point of view,
>>>> it
>>>> should be integrated (for consistency reason) like a Dataformat.
>>>> Because
>>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>> ...)
>>>> into another format (Collection of Java objects)
>>>>
>>>> E.g.
>>>>
>>>> DataFormat flatpack = new
>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>
>>>> from("file:///c:/test")
>>>> .unmarshall(flatpack)
>>>> .to("whatever you want")
>>>>
>>>> Regards,
>>>>
>>>> Charles
>>>>
>>>>
>>>>
>>>> James.Strachan wrote:
>>>>>
>>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> I've raised a JIRA to track this...
>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>
>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>> like
>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>
>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>
>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>> with
>>>>>>> a nice design UI that does this with Camel...
>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>
>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>>
>>>>
>>>>
>>>> -----
>>>> Enterprise Architect
>>>>
>>>> Xpectis
>>>> 12, route d'Esch
>>>> L-1470 Luxembourg
>>>>
>>>> Phone +352 25 10 70 470
>>>> Mobile +352 621 45 36 22
>>>>
>>>> e-mail : cmoulliard@xpectis.com
>>>> web site : www.xpectis.com
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>> 
>> 
>> 
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>> 
>> Open Source Integration
>> http://open.iona.com
>> 
>> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18483512.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
I have a new error when I use the following routes. 

from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
               
.to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
               
               
       
from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
                .convertBodyTo(Map.class)
                .to("activemq:queue:cutas.in.queue");

and xml file

<?xml version='1.0'?>
<!DOCTYPE PZMAP SYSTEM "flatpack.dtd" >

<PZMAP>
	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="HEADERDATA" length="26" />
	</RECORD>
	<COLUMN name="RECORDTYPE" length="1" />
	<COLUMN name="SYSTEMID" length="3" />
	<COLUMN name="THIRD_PARTY_REF" length="15" />
	<COLUMN name="CUTAS_ACCOUNT_NR" length="15" />
	<COLUMN name="FUND_CODE" length="5" />
	<COLUMN name="PRODUCT_CODE" length="5" />
	<COLUMN name="PRODUCT_TYPE" length="4" />
	<COLUMN name="CUTAS_REPLY_DATE" length="8" />
	<COLUMN name="CUTAS_REPLY_HOUR" length="4" />
	<COLUMN name="TRANSACTION_TYPE" length="2" />
	<COLUMN name="FUND_TYPE" length="1" />
	<COLUMN name="UNIT_SIGN" length="1" />
	<COLUMN name="DATA" length="412" />
	
	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="TRAILERDATA" length="26" />
	</RECORD>

</PZMAP> 

Remark : In my case, we have a header / Trailer in the message.

Error reported :


43org.apache.camel.RuntimeCamelException: java.util.NoSuchElementException:
Column [recordtype] does not exist, check case/spelling. key:trailer
	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:411)
	at
org.apache.camel.impl.converter.StaticMethodTypeConverter.convertTo(StaticMethodTypeConverter.java:42)
	at
org.apache.camel.impl.converter.DefaultTypeConverter.convertTo(DefaultTypeConverter.java:67)
	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:59)
	at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:50)
	at
org.apache.camel.processor.ConvertBodyProcessor.process(ConvertBodyProcessor.java:41)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
	at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
	at org.apache.camel.processor.Pipeline.process(Pipeline.java:85)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
	at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
	at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
	at
org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:66)
	at
org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:41)
	at
org.apache.camel.component.flatpack.FixedLengthEndpoint.processDataSet(FixedLengthEndpoint.java:67)
	at
org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:41)
	at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:61)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:69)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:41)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.processNext(DelegateProcessor.java:50)
	at
org.apache.camel.processor.DelegateProcessor.proceed(DelegateProcessor.java:79)
	at
org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:57)
	at
org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
	at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
	at
org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
	at org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
	at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
	at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown
Source)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown
Source)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.util.NoSuchElementException: Column [recordtype] does not
exist, check case/spelling. key:trailer
	at net.sf.flatpack.util.ParserUtils.getColumnIndex(ParserUtils.java:785)
	at net.sf.flatpack.DefaultDataSet.getStringValue(DefaultDataSet.java:184)
	at net.sf.flatpack.DefaultDataSet.getString(DefaultDataSet.java:262)
	at
org.apache.camel.component.flatpack.FlatpackConverter.putValues(FlatpackConverter.java:48)
	at
org.apache.camel.component.flatpack.FlatpackConverter.toMap(FlatpackConverter.java:35)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:407)
	... 55 more

Content of the message :

4349422 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/1446-1216195452000/2-0 -> to7
To[flatpack:fixed:com/bravura/babel/cutas/transformation/in/CUTAS.xml]
InOnly Properties:{CamelCauseException=null,
org.apache.camel.file.lock.name=c:\temp\cutas\ST000355.txt.cameLock,
org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
exclusive valid],
org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@190725e}
Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
Body:HBT00032700000327190620071714
C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
+0000000010000000000009076000000138137A000090760EUR   GBPSS808800714636FIEU
/0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
00001522000 000000000000000O                  0000000000000000                                        
20070907155200                                                                             
X
C00111a30571b170007B000005        APAA          200805271554  
+0000000001835910000003941700000039417A000332147USD  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000B                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571a600006B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000X                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571c2a0009B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000                  
0000000000000000                                         20070619101000                                                                             
X
FBT00032700000001190620071714

Info displayed on the console :

4349609 [Thread: 1 org.apache.camel.component.file.FileComponent@1b78953]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/1446-1216195452000/2-1 -> interceptor4
Interceptor[Delegate(Pipeline[DeadLetterChannel[Delegate(TraceInterceptor[convertBodyTo[
null]]), sendTo(Endpoint[seda:errors]),
RedeliveryPolicy[maximumRedeliveries=1]],
DeadLetterChannel[Delegate(TraceInterceptor[To[activemq:queue:cutas.in.queue]]),
sendTo(Endpoint[seda:errors]), RedeliveryPolicy[maximumRedeliveries=1]]])]
InOnly Properties:{} Headers:{camelFlatpackCounter=0}
BodyType:net.sf.flatpack.DefaultDataSet Body:Errors:0
Rows:6
Position:0
Conversion Props:{java.lang.Double=net.sf.flatpack.converter.ConvertDouble,
java.math.BigDecimal=net.sf.flatpack.converter.ConvertBigDecimal,
java.lang.Integer=net.sf.flatpack.converter.ConvertInteger}
MetaData:Col Names:[Name:RECORDTYPE Length:1 Start:0 End:0
, Name:SYSTEMID Length:3 Start:0 End:0
, Name:THIRD_PARTY_REF Length:15 Start:0 End:0
, Name:CUTAS_ACCOUNT_NR Length:15 Start:0 End:0
, Name:FUND_CODE Length:5 Start:0 End:0
, Name:PRODUCT_CODE Length:5 Start:0 End:0
, Name:PRODUCT_TYPE Length:4 Start:0 End:0
, Name:CUTAS_REPLY_DATE Length:8 Start:0 End:0
, Name:CUTAS_REPLY_HOUR Length:4 Start:0 End:0
, Name:TRANSACTION_TYPE Length:2 Start:0 End:0
, Name:FUND_TYPE Length:1 Start:0 End:0
, Name:UNIT_SIGN Length:1 Start:0 End:0
, Name:DATA Length:412 Start:0 End:0
]
Col Index Map:{unit_sign=11, transaction_type=9, fund_type=10, systemid=1,
cutas_account_nr=3, product_code=5, product_type=6, recordtype=0, data=12,
cutas_reply_date=7, third_party_ref=2, fund_code=4, cutas_reply_hour=8}
XML Record Elements:{header=net.sf.flatpack.xml.XMLRecordElement@11d95,
trailer=net.sf.flatpack.xml.XMLRecordElement@13d4c49}

Regards,

Charles



James.Strachan wrote:
> 
> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
> to use. The downside is the data format kinda has to turn the DataSet
> into a List of Maps to make it easier to work with; but at least the
> routes are a bit simpler. e.g. this is from the
> DelimitedWithUnmarshalTest (the convertBodyTo is not required
> currently)
> 
>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>     <route>
>       <from uri="file://src/test/data/delim?noop=true"/>
>       <unmarshal ref="delimitedFormat"/>
>       <splitter>
>         <simple>in.body</simple>
>         <convertBodyTo type="java.util.Map"/>
>         <to uri="mock:results"/>
>       </splitter>
>     </route>
>   </camelContext>
> 
>   <bean id="delimitedFormat"
> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>   </bean>
> 
> 2008/7/15 James Strachan <ja...@gmail.com>:
>> Yeah - I did ponder if it should be a dataformat or a component; or
>> maybe even both. The bit I wasn't sure about was flatpack really just
>> provides an iterator over rows of the file; I wasn't sure if this
>> properly matched the dataformat model of marshalling or unmarshalling
>> a payload (which currently assumes that unmarshalling generates a
>> single result).
>>
>> Maybe we need to enhance the dataformat concept to work with this use
>> case too?
>>
>> So maybe if we want to process each row of a file we do something like...
>>
>> from("file://whatnot").
>> 
>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>  split().body().
>>  to("whatever");
>>
>> where the split().body() would do the iteration over each DataSet?
>>
>>
>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>> Hi
>>>
>>> Yes I thought also at first it was a dataformat ;) But how James created
>>> it it works as if it is a dataformat. You just have to do it in to..from
>>> from..to steps.
>>>
>>>
>>> from("file://c:/test").to("flatpack:foo:
>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>
>>> from("flatpack:foo").to("whatever you want");
>>>
>>> As a component it gets more attention, as people can find it on the
>>> component list. The dataformats are kinda hidden in the wiki
>>> documentation.
>>> Maybe they need a "dataformat" top menu as well ;)
>>>
>>> Med venlig hilsen
>>>
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>>
>>> -----Original Message-----
>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>> Sent: 15. juli 2008 09:32
>>> To: camel-user@activemq.apache.org
>>> Subject: Re: Component to read FixedLength file ?
>>>
>>>
>>> Many thanks.
>>>
>>> I will test the integration of this component within Camel.
>>>
>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>> The
>>> project has been integrated like a component but from my point of view,
>>> it
>>> should be integrated (for consistency reason) like a Dataformat. Because
>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>>> into another format (Collection of Java objects)
>>>
>>> E.g.
>>>
>>> DataFormat flatpack = new
>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>
>>> from("file:///c:/test")
>>> .unmarshall(flatpack)
>>> .to("whatever you want")
>>>
>>> Regards,
>>>
>>> Charles
>>>
>>>
>>>
>>> James.Strachan wrote:
>>>>
>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>> but there's a basic fixed-width reader example there for the curious
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> I've raised a JIRA to track this...
>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>> like
>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>
>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>
>>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>>> a nice design UI that does this with Camel...
>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>
>>>>>> but it'd be nice to have an OSS option :)
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>>
>>>
>>>
>>> -----
>>> Enterprise Architect
>>>
>>> Xpectis
>>> 12, route d'Esch
>>> L-1470 Luxembourg
>>>
>>> Phone +352 25 10 70 470
>>> Mobile +352 621 45 36 22
>>>
>>> e-mail : cmoulliard@xpectis.com
>>> web site : www.xpectis.com
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18482019.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
The body is empty before to go to the endpoint so adding
.convertBodyTo(Map.class) has no effect

Regards,

Charles Moulliard


Claus Ibsen wrote:
> 
> Hi
> 
> I think you need to convert it to a Map afterwards to get your data. Our
> use the iterator on the body (it's a List).
> 
> List list = (List) exchange.getIn().getBody();
> list.iterator() will give you a Map of each row
> 
> 
> Or add the convertBodyTo (I'm not 100 sure)
> 		from(endpointName.getFileCutasIn())
> 		.unmarshal(flatpack)
>             .convertBodyTo(Map.class);
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 17. juli 2008 13:36
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> 
> Hi James,
> 
> When I try to unmarshall my fixedlenghtwithHeaderTrailer file, the body
> which is send to my bean is empty. Is there a bug ?
> 
> Here is the RouteBuilder class
> 
> public class A_ParseAndUnmarshallCutasMessages extends RouteBuilder {
> 	
> 	private static final Log LOG =
> LogFactory.getLog(A_ParseAndUnmarshallCutasMessages.class);
> 	private FlatpackDataFormat flatpack;
> 
> 	@Override
> 	public void configure() {
> 		
> 		getContext().addInterceptStrategy(new Tracer());
> 		
> 		from(endpointName.getFileCutasIn())
> 		.unmarshal(flatpack)
> 		.to("bean:convert?methodName=CutasToBabel")
> 		.to(endpointName.getAcceptedCutasQueueIn());
> 		
> 	}
> 	
> 	@Autowired
> 	public void setFlatpackDataFormat(FlatpackDataFormat fixedFormat) {
> 		this.flatpack = fixedFormat;
> 	}
> 
> }
> 
> And the camel context
> 
> 	<bean id="flatpackDataFormat"
> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
> 		<property name="definition"
> value="com/myApplication/transformation/cutas/in/CUTAS.xml" />
> 	</bean>
> 
> and the trace :
> 
> 4316656 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/3853-1216294265750/2-1 -> unmarshal1
> Marshal[org.apache.camel.model.dataformat.DataFormatType@5f1ba8] InOnly
> Properties:{CamelCauseException=null,
> org.apache.camel.file.lock.name=c:\temp\cutas\input\clientA\ST000355.txt.cameLock,
> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
> exclusive valid],
> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@1f4b0dc}
> Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
> Body:HBT00032700000327190620071714
> C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
> +0000000010000000000009076000000138137A000090760EUR  
> GBPSS808800714636FIEU
> /0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
> 00001522000 000000000000000O                  0000000000000000                                        
> 20070907155200                                                                             
> X
> C00111a30571b170007B000005        APAA          200805271554  
> +0000000001835910000003941700000039417A000332147USD  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000B                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571a600006B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000X                 
> 0000000000000000                                         20070619101000                                                                             
> X
> C00111a30571c2a0009B000005        APAA          200805271554  
> +0000000001855550000003941700000039417A000052147GBP  
> GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
> 000000000000000000000000000 00000000000 000000000000000                  
> 0000000000000000                                         20070619101000                                                                             
> X
> FBT00032700000001190620071714
> 4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
> DEBUG org.apache.camel.management.InstrumentationProcessor  - Recording
> duration: 187.148981 millis for exchange: Exchange[FileMessage:
> c:\temp\cutas\input\clientA\ST000355.txt]
> 4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
> INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
> ID-dell-charles/3853-1216294265750/2-1 -> to3
> To[bean:convert?methodName=CutasToBabel] InOnly
> Properties:{CamelCauseException=null,
> org.apache.camel.file.lock.name=c:\temp\cutas\input\clientA\ST000355.txt.cameLock,
> org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
> exclusive valid],
> org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@1f4b0dc}
> Headers:{org.apache.camel.file.name=ST000355.txt}
> BodyType:org.apache.camel.component.flatpack.DataSetList Body:[]
> 4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
> DEBUG org.apache.camel.component.bean.BeanProcessor  - >>>> invoking
> method
> for: Exchange[Message: []]
> 
> Regards,
> 
> Charles
> 
> 
> 
> James.Strachan wrote:
>> 
>> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
>> to use. The downside is the data format kinda has to turn the DataSet
>> into a List of Maps to make it easier to work with; but at least the
>> routes are a bit simpler. e.g. this is from the
>> DelimitedWithUnmarshalTest (the convertBodyTo is not required
>> currently)
>> 
>>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>>     <route>
>>       <from uri="file://src/test/data/delim?noop=true"/>
>>       <unmarshal ref="delimitedFormat"/>
>>       <splitter>
>>         <simple>in.body</simple>
>>         <convertBodyTo type="java.util.Map"/>
>>         <to uri="mock:results"/>
>>       </splitter>
>>     </route>
>>   </camelContext>
>> 
>>   <bean id="delimitedFormat"
>> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>>   </bean>
>> 
>> 2008/7/15 James Strachan <ja...@gmail.com>:
>>> Yeah - I did ponder if it should be a dataformat or a component; or
>>> maybe even both. The bit I wasn't sure about was flatpack really just
>>> provides an iterator over rows of the file; I wasn't sure if this
>>> properly matched the dataformat model of marshalling or unmarshalling
>>> a payload (which currently assumes that unmarshalling generates a
>>> single result).
>>>
>>> Maybe we need to enhance the dataformat concept to work with this use
>>> case too?
>>>
>>> So maybe if we want to process each row of a file we do something
>>> like...
>>>
>>> from("file://whatnot").
>>> 
>>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>>  split().body().
>>>  to("whatever");
>>>
>>> where the split().body() would do the iteration over each DataSet?
>>>
>>>
>>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>>> Hi
>>>>
>>>> Yes I thought also at first it was a dataformat ;) But how James
>>>> created
>>>> it it works as if it is a dataformat. You just have to do it in
>>>> to..from
>>>> from..to steps.
>>>>
>>>>
>>>> from("file://c:/test").to("flatpack:foo:
>>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>>
>>>> from("flatpack:foo").to("whatever you want");
>>>>
>>>> As a component it gets more attention, as people can find it on the
>>>> component list. The dataformats are kinda hidden in the wiki
>>>> documentation.
>>>> Maybe they need a "dataformat" top menu as well ;)
>>>>
>>>> Med venlig hilsen
>>>>
>>>> Claus Ibsen
>>>> ......................................
>>>> Silverbullet
>>>> Skovsgårdsvænget 21
>>>> 8362 Hørning
>>>> Tlf. +45 2962 7576
>>>> Web: www.silverbullet.dk
>>>>
>>>> -----Original Message-----
>>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>>> Sent: 15. juli 2008 09:32
>>>> To: camel-user@activemq.apache.org
>>>> Subject: Re: Component to read FixedLength file ?
>>>>
>>>>
>>>> Many thanks.
>>>>
>>>> I will test the integration of this component within Camel.
>>>>
>>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>>> The
>>>> project has been integrated like a component but from my point of view,
>>>> it
>>>> should be integrated (for consistency reason) like a Dataformat.
>>>> Because
>>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>>> XStream, ....), they transform the content from a format (XML, CSV,
>>>> ...)
>>>> into another format (Collection of Java objects)
>>>>
>>>> E.g.
>>>>
>>>> DataFormat flatpack = new
>>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>>
>>>> from("file:///c:/test")
>>>> .unmarshall(flatpack)
>>>> .to("whatever you want")
>>>>
>>>> Regards,
>>>>
>>>> Charles
>>>>
>>>>
>>>>
>>>> James.Strachan wrote:
>>>>>
>>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>>> but there's a basic fixed-width reader example there for the curious
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> I've raised a JIRA to track this...
>>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>>
>>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>>> like
>>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>>
>>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>>
>>>>>>> there's a commercial tool called Artix Data Services which comes
>>>>>>> with
>>>>>>> a nice design UI that does this with Camel...
>>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>>
>>>>>>> but it'd be nice to have an OSS option :)
>>>>>>>
>>>>>>> --
>>>>>>> James
>>>>>>> -------
>>>>>>> http://macstrac.blogspot.com/
>>>>>>>
>>>>>>> Open Source Integration
>>>>>>> http://open.iona.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>>
>>>>
>>>>
>>>> -----
>>>> Enterprise Architect
>>>>
>>>> Xpectis
>>>> 12, route d'Esch
>>>> L-1470 Luxembourg
>>>>
>>>> Phone +352 25 10 70 470
>>>> Mobile +352 621 45 36 22
>>>>
>>>> e-mail : cmoulliard@xpectis.com
>>>> web site : www.xpectis.com
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>> 
>> 
>> 
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>> 
>> Open Source Integration
>> http://open.iona.com
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18506498.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18506999.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Hi

I think you need to convert it to a Map afterwards to get your data. Our use the iterator on the body (it's a List).

List list = (List) exchange.getIn().getBody();
list.iterator() will give you a Map of each row


Or add the convertBodyTo (I'm not 100 sure)
		from(endpointName.getFileCutasIn())
		.unmarshal(flatpack)
            .convertBodyTo(Map.class);


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: cmoulliard [mailto:cmoulliard@gmail.com] 
Sent: 17. juli 2008 13:36
To: camel-user@activemq.apache.org
Subject: Re: Component to read FixedLength file ?


Hi James,

When I try to unmarshall my fixedlenghtwithHeaderTrailer file, the body
which is send to my bean is empty. Is there a bug ?

Here is the RouteBuilder class

public class A_ParseAndUnmarshallCutasMessages extends RouteBuilder {
	
	private static final Log LOG =
LogFactory.getLog(A_ParseAndUnmarshallCutasMessages.class);
	private FlatpackDataFormat flatpack;

	@Override
	public void configure() {
		
		getContext().addInterceptStrategy(new Tracer());
		
		from(endpointName.getFileCutasIn())
		.unmarshal(flatpack)
		.to("bean:convert?methodName=CutasToBabel")
		.to(endpointName.getAcceptedCutasQueueIn());
		
	}
	
	@Autowired
	public void setFlatpackDataFormat(FlatpackDataFormat fixedFormat) {
		this.flatpack = fixedFormat;
	}

}

And the camel context

	<bean id="flatpackDataFormat"
class="org.apache.camel.component.flatpack.FlatpackDataFormat">
		<property name="definition"
value="com/myApplication/transformation/cutas/in/CUTAS.xml" />
	</bean>

and the trace :

4316656 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/3853-1216294265750/2-1 -> unmarshal1
Marshal[org.apache.camel.model.dataformat.DataFormatType@5f1ba8] InOnly
Properties:{CamelCauseException=null,
org.apache.camel.file.lock.name=c:\temp\cutas\input\clientA\ST000355.txt.cameLock,
org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
exclusive valid],
org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@1f4b0dc}
Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
Body:HBT00032700000327190620071714
C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
+0000000010000000000009076000000138137A000090760EUR   GBPSS808800714636FIEU
/0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
00001522000 000000000000000O                  0000000000000000                                        
20070907155200                                                                             
X
C00111a30571b170007B000005        APAA          200805271554  
+0000000001835910000003941700000039417A000332147USD  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000B                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571a600006B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000X                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571c2a0009B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000                  
0000000000000000                                         20070619101000                                                                             
X
FBT00032700000001190620071714
4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
DEBUG org.apache.camel.management.InstrumentationProcessor  - Recording
duration: 187.148981 millis for exchange: Exchange[FileMessage:
c:\temp\cutas\input\clientA\ST000355.txt]
4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/3853-1216294265750/2-1 -> to3
To[bean:convert?methodName=CutasToBabel] InOnly
Properties:{CamelCauseException=null,
org.apache.camel.file.lock.name=c:\temp\cutas\input\clientA\ST000355.txt.cameLock,
org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
exclusive valid],
org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@1f4b0dc}
Headers:{org.apache.camel.file.name=ST000355.txt}
BodyType:org.apache.camel.component.flatpack.DataSetList Body:[]
4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
DEBUG org.apache.camel.component.bean.BeanProcessor  - >>>> invoking method
for: Exchange[Message: []]

Regards,

Charles



James.Strachan wrote:
> 
> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
> to use. The downside is the data format kinda has to turn the DataSet
> into a List of Maps to make it easier to work with; but at least the
> routes are a bit simpler. e.g. this is from the
> DelimitedWithUnmarshalTest (the convertBodyTo is not required
> currently)
> 
>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>     <route>
>       <from uri="file://src/test/data/delim?noop=true"/>
>       <unmarshal ref="delimitedFormat"/>
>       <splitter>
>         <simple>in.body</simple>
>         <convertBodyTo type="java.util.Map"/>
>         <to uri="mock:results"/>
>       </splitter>
>     </route>
>   </camelContext>
> 
>   <bean id="delimitedFormat"
> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>   </bean>
> 
> 2008/7/15 James Strachan <ja...@gmail.com>:
>> Yeah - I did ponder if it should be a dataformat or a component; or
>> maybe even both. The bit I wasn't sure about was flatpack really just
>> provides an iterator over rows of the file; I wasn't sure if this
>> properly matched the dataformat model of marshalling or unmarshalling
>> a payload (which currently assumes that unmarshalling generates a
>> single result).
>>
>> Maybe we need to enhance the dataformat concept to work with this use
>> case too?
>>
>> So maybe if we want to process each row of a file we do something like...
>>
>> from("file://whatnot").
>> 
>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>  split().body().
>>  to("whatever");
>>
>> where the split().body() would do the iteration over each DataSet?
>>
>>
>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>> Hi
>>>
>>> Yes I thought also at first it was a dataformat ;) But how James created
>>> it it works as if it is a dataformat. You just have to do it in to..from
>>> from..to steps.
>>>
>>>
>>> from("file://c:/test").to("flatpack:foo:
>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>
>>> from("flatpack:foo").to("whatever you want");
>>>
>>> As a component it gets more attention, as people can find it on the
>>> component list. The dataformats are kinda hidden in the wiki
>>> documentation.
>>> Maybe they need a "dataformat" top menu as well ;)
>>>
>>> Med venlig hilsen
>>>
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>>
>>> -----Original Message-----
>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>> Sent: 15. juli 2008 09:32
>>> To: camel-user@activemq.apache.org
>>> Subject: Re: Component to read FixedLength file ?
>>>
>>>
>>> Many thanks.
>>>
>>> I will test the integration of this component within Camel.
>>>
>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>> The
>>> project has been integrated like a component but from my point of view,
>>> it
>>> should be integrated (for consistency reason) like a Dataformat. Because
>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>>> into another format (Collection of Java objects)
>>>
>>> E.g.
>>>
>>> DataFormat flatpack = new
>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>
>>> from("file:///c:/test")
>>> .unmarshall(flatpack)
>>> .to("whatever you want")
>>>
>>> Regards,
>>>
>>> Charles
>>>
>>>
>>>
>>> James.Strachan wrote:
>>>>
>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>> but there's a basic fixed-width reader example there for the curious
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> I've raised a JIRA to track this...
>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>> like
>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>
>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>
>>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>>> a nice design UI that does this with Camel...
>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>
>>>>>> but it'd be nice to have an OSS option :)
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>>
>>>
>>>
>>> -----
>>> Enterprise Architect
>>>
>>> Xpectis
>>> 12, route d'Esch
>>> L-1470 Luxembourg
>>>
>>> Phone +352 25 10 70 470
>>> Mobile +352 621 45 36 22
>>>
>>> e-mail : cmoulliard@xpectis.com
>>> web site : www.xpectis.com
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18506498.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
Hi James,

When I try to unmarshall my fixedlenghtwithHeaderTrailer file, the body
which is send to my bean is empty. Is there a bug ?

Here is the RouteBuilder class

public class A_ParseAndUnmarshallCutasMessages extends RouteBuilder {
	
	private static final Log LOG =
LogFactory.getLog(A_ParseAndUnmarshallCutasMessages.class);
	private FlatpackDataFormat flatpack;

	@Override
	public void configure() {
		
		getContext().addInterceptStrategy(new Tracer());
		
		from(endpointName.getFileCutasIn())
		.unmarshal(flatpack)
		.to("bean:convert?methodName=CutasToBabel")
		.to(endpointName.getAcceptedCutasQueueIn());
		
	}
	
	@Autowired
	public void setFlatpackDataFormat(FlatpackDataFormat fixedFormat) {
		this.flatpack = fixedFormat;
	}

}

And the camel context

	<bean id="flatpackDataFormat"
class="org.apache.camel.component.flatpack.FlatpackDataFormat">
		<property name="definition"
value="com/myApplication/transformation/cutas/in/CUTAS.xml" />
	</bean>

and the trace :

4316656 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/3853-1216294265750/2-1 -> unmarshal1
Marshal[org.apache.camel.model.dataformat.DataFormatType@5f1ba8] InOnly
Properties:{CamelCauseException=null,
org.apache.camel.file.lock.name=c:\temp\cutas\input\clientA\ST000355.txt.cameLock,
org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
exclusive valid],
org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@1f4b0dc}
Headers:{org.apache.camel.file.name=ST000355.txt} BodyType:java.io.File
Body:HBT00032700000327190620071714
C00111a30571b6b0008Z001412        FIEU BFM      200805271554  
+0000000010000000000009076000000138137A000090760EUR   GBPSS808800714636FIEU
/0008031800000000000000000000000000000000000000000000000000090760000000000000000000000000000000000138137413092007000000000000000000001522000
00001522000 000000000000000O                  0000000000000000                                        
20070907155200                                                                             
X
C00111a30571b170007B000005        APAA          200805271554  
+0000000001835910000003941700000039417A000332147USD  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000B                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571a600006B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000X                 
0000000000000000                                         20070619101000                                                                             
X
C00111a30571c2a0009B000005        APAA          200805271554  
+0000000001855550000003941700000039417A000052147GBP  
GBPAGENT188888888BDBTA/0000292200000000000000000000000000000000000000000000000039417GBP000000000000000000000000000000000039417        
000000000000000000000000000 00000000000 000000000000000                  
0000000000000000                                         20070619101000                                                                             
X
FBT00032700000001190620071714
4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
DEBUG org.apache.camel.management.InstrumentationProcessor  - Recording
duration: 187.148981 millis for exchange: Exchange[FileMessage:
c:\temp\cutas\input\clientA\ST000355.txt]
4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
INFO  org.apache.camel.processor.interceptor.TraceInterceptor  -
ID-dell-charles/3853-1216294265750/2-1 -> to3
To[bean:convert?methodName=CutasToBabel] InOnly
Properties:{CamelCauseException=null,
org.apache.camel.file.lock.name=c:\temp\cutas\input\clientA\ST000355.txt.cameLock,
org.apache.camel.file.lock=sun.nio.ch.FileLockImpl[0:9223372036854775807
exclusive valid],
org.apache.camel.fileChannel=sun.nio.ch.FileChannelImpl@1f4b0dc}
Headers:{org.apache.camel.file.name=ST000355.txt}
BodyType:org.apache.camel.component.flatpack.DataSetList Body:[]
4316890 [Thread: 2 org.apache.camel.component.file.FileComponent@14f5021]
DEBUG org.apache.camel.component.bean.BeanProcessor  - >>>> invoking method
for: Exchange[Message: []]

Regards,

Charles



James.Strachan wrote:
> 
> FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
> to use. The downside is the data format kinda has to turn the DataSet
> into a List of Maps to make it easier to work with; but at least the
> routes are a bit simpler. e.g. this is from the
> DelimitedWithUnmarshalTest (the convertBodyTo is not required
> currently)
> 
>   <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
>     <route>
>       <from uri="file://src/test/data/delim?noop=true"/>
>       <unmarshal ref="delimitedFormat"/>
>       <splitter>
>         <simple>in.body</simple>
>         <convertBodyTo type="java.util.Map"/>
>         <to uri="mock:results"/>
>       </splitter>
>     </route>
>   </camelContext>
> 
>   <bean id="delimitedFormat"
> class="org.apache.camel.component.flatpack.FlatpackDataFormat">
>     <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
>   </bean>
> 
> 2008/7/15 James Strachan <ja...@gmail.com>:
>> Yeah - I did ponder if it should be a dataformat or a component; or
>> maybe even both. The bit I wasn't sure about was flatpack really just
>> provides an iterator over rows of the file; I wasn't sure if this
>> properly matched the dataformat model of marshalling or unmarshalling
>> a payload (which currently assumes that unmarshalling generates a
>> single result).
>>
>> Maybe we need to enhance the dataformat concept to work with this use
>> case too?
>>
>> So maybe if we want to process each row of a file we do something like...
>>
>> from("file://whatnot").
>> 
>> unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>>  split().body().
>>  to("whatever");
>>
>> where the split().body() would do the iteration over each DataSet?
>>
>>
>> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>>> Hi
>>>
>>> Yes I thought also at first it was a dataformat ;) But how James created
>>> it it works as if it is a dataformat. You just have to do it in to..from
>>> from..to steps.
>>>
>>>
>>> from("file://c:/test").to("flatpack:foo:
>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>
>>> from("flatpack:foo").to("whatever you want");
>>>
>>> As a component it gets more attention, as people can find it on the
>>> component list. The dataformats are kinda hidden in the wiki
>>> documentation.
>>> Maybe they need a "dataformat" top menu as well ;)
>>>
>>> Med venlig hilsen
>>>
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>>
>>> -----Original Message-----
>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>> Sent: 15. juli 2008 09:32
>>> To: camel-user@activemq.apache.org
>>> Subject: Re: Component to read FixedLength file ?
>>>
>>>
>>> Many thanks.
>>>
>>> I will test the integration of this component within Camel.
>>>
>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>> The
>>> project has been integrated like a component but from my point of view,
>>> it
>>> should be integrated (for consistency reason) like a Dataformat. Because
>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>>> into another format (Collection of Java objects)
>>>
>>> E.g.
>>>
>>> DataFormat flatpack = new
>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>
>>> from("file:///c:/test")
>>> .unmarshall(flatpack)
>>> .to("whatever you want")
>>>
>>> Regards,
>>>
>>> Charles
>>>
>>>
>>>
>>> James.Strachan wrote:
>>>>
>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>> but there's a basic fixed-width reader example there for the curious
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> I've raised a JIRA to track this...
>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>> like
>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>
>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>
>>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>>> a nice design UI that does this with Camel...
>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>
>>>>>> but it'd be nice to have an OSS option :)
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>>
>>>
>>>
>>> -----
>>> Enterprise Architect
>>>
>>> Xpectis
>>> 12, route d'Esch
>>> L-1470 Luxembourg
>>>
>>> Phone +352 25 10 70 470
>>> Mobile +352 621 45 36 22
>>>
>>> e-mail : cmoulliard@xpectis.com
>>> web site : www.xpectis.com
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18506498.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
FWIW I've just checked in a FlatpackDataFormat; which is a tad easier
to use. The downside is the data format kinda has to turn the DataSet
into a List of Maps to make it easier to work with; but at least the
routes are a bit simpler. e.g. this is from the
DelimitedWithUnmarshalTest (the convertBodyTo is not required
currently)

  <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
    <route>
      <from uri="file://src/test/data/delim?noop=true"/>
      <unmarshal ref="delimitedFormat"/>
      <splitter>
        <simple>in.body</simple>
        <convertBodyTo type="java.util.Map"/>
        <to uri="mock:results"/>
      </splitter>
    </route>
  </camelContext>

  <bean id="delimitedFormat"
class="org.apache.camel.component.flatpack.FlatpackDataFormat">
    <property name="definition" value="INVENTORY-Delimited.pzmap.xml"/>
  </bean>

2008/7/15 James Strachan <ja...@gmail.com>:
> Yeah - I did ponder if it should be a dataformat or a component; or
> maybe even both. The bit I wasn't sure about was flatpack really just
> provides an iterator over rows of the file; I wasn't sure if this
> properly matched the dataformat model of marshalling or unmarshalling
> a payload (which currently assumes that unmarshalling generates a
> single result).
>
> Maybe we need to enhance the dataformat concept to work with this use case too?
>
> So maybe if we want to process each row of a file we do something like...
>
> from("file://whatnot").
>  unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
>  split().body().
>  to("whatever");
>
> where the split().body() would do the iteration over each DataSet?
>
>
> 2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
>> Hi
>>
>> Yes I thought also at first it was a dataformat ;) But how James created it it works as if it is a dataformat. You just have to do it in to..from from..to steps.
>>
>>
>> from("file://c:/test").to("flatpack:foo: locationOftheXMLConfigurationfile.pzmap.xml");
>>
>> from("flatpack:foo").to("whatever you want");
>>
>> As a component it gets more attention, as people can find it on the component list. The dataformats are kinda hidden in the wiki documentation.
>> Maybe they need a "dataformat" top menu as well ;)
>>
>> Med venlig hilsen
>>
>> Claus Ibsen
>> ......................................
>> Silverbullet
>> Skovsgårdsvænget 21
>> 8362 Hørning
>> Tlf. +45 2962 7576
>> Web: www.silverbullet.dk
>>
>> -----Original Message-----
>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>> Sent: 15. juli 2008 09:32
>> To: camel-user@activemq.apache.org
>> Subject: Re: Component to read FixedLength file ?
>>
>>
>> Many thanks.
>>
>> I will test the integration of this component within Camel.
>>
>> Remark : I'm a little confused about how Flatpack has been integrated. The
>> project has been integrated like a component but from my point of view, it
>> should be integrated (for consistency reason) like a Dataformat. Because
>> like the other framework/technology implemented in Camel (JAXB, Artix,
>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>> into another format (Collection of Java objects)
>>
>> E.g.
>>
>> DataFormat flatpack = new
>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>
>> from("file:///c:/test")
>> .unmarshall(flatpack)
>> .to("whatever you want")
>>
>> Regards,
>>
>> Charles
>>
>>
>>
>> James.Strachan wrote:
>>>
>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>> but there's a basic fixed-width reader example there for the curious
>>>
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> I've raised a JIRA to track this...
>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>
>>>>> It should be pretty easy to write a flatpack component.
>>>>>
>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>> a nice design UI that does this with Camel...
>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>
>>>>> but it'd be nice to have an OSS option :)
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>>
>>
>>
>> -----
>> Enterprise Architect
>>
>> Xpectis
>> 12, route d'Esch
>> L-1470 Luxembourg
>>
>> Phone +352 25 10 70 470
>> Mobile +352 621 45 36 22
>>
>> e-mail : cmoulliard@xpectis.com
>> web site : www.xpectis.com
>> --
>> View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
Yeah - I did ponder if it should be a dataformat or a component; or
maybe even both. The bit I wasn't sure about was flatpack really just
provides an iterator over rows of the file; I wasn't sure if this
properly matched the dataformat model of marshalling or unmarshalling
a payload (which currently assumes that unmarshalling generates a
single result).

Maybe we need to enhance the dataformat concept to work with this use case too?

So maybe if we want to process each row of a file we do something like...

from("file://whatnot").
  unmarshall().flatpack().fixed("locationOftheXMLConfigurationfile.pzmap.xml").
  split().body().
  to("whatever");

where the split().body() would do the iteration over each DataSet?


2008/7/15 Claus Ibsen <ci...@silverbullet.dk>:
> Hi
>
> Yes I thought also at first it was a dataformat ;) But how James created it it works as if it is a dataformat. You just have to do it in to..from from..to steps.
>
>
> from("file://c:/test").to("flatpack:foo: locationOftheXMLConfigurationfile.pzmap.xml");
>
> from("flatpack:foo").to("whatever you want");
>
> As a component it gets more attention, as people can find it on the component list. The dataformats are kinda hidden in the wiki documentation.
> Maybe they need a "dataformat" top menu as well ;)
>
> Med venlig hilsen
>
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
>
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com]
> Sent: 15. juli 2008 09:32
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
>
>
> Many thanks.
>
> I will test the integration of this component within Camel.
>
> Remark : I'm a little confused about how Flatpack has been integrated. The
> project has been integrated like a component but from my point of view, it
> should be integrated (for consistency reason) like a Dataformat. Because
> like the other framework/technology implemented in Camel (JAXB, Artix,
> XStream, ....), they transform the content from a format (XML, CSV, ...)
> into another format (Collection of Java objects)
>
> E.g.
>
> DataFormat flatpack = new
> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>
> from("file:///c:/test")
> .unmarshall(flatpack)
> .to("whatever you want")
>
> Regards,
>
> Charles
>
>
>
> James.Strachan wrote:
>>
>> BTW trunk has an early spike of a flatpack component. Needs more work,
>> but there's a basic fixed-width reader example there for the curious
>>
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> I've raised a JIRA to track this...
>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>
>>>> It should be pretty easy to write a flatpack component.
>>>>
>>>> there's a commercial tool called Artix Data Services which comes with
>>>> a nice design UI that does this with Camel...
>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>
>>>> but it'd be nice to have an OSS option :)
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
>>
>
>
> -----
> Enterprise Architect
>
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
>
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
>
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com
> --
> View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
I found the error. In my case I had ýp (Hex FF FE which is a unicode marker)
that my Editor (Ultraedit) was inserting.


James.Strachan wrote:
> 
> Are you sure the XML file is valid? e.g. there's no newlines before
> the <?xml prolog?
> 
> 2008/7/15 cmoulliard <cm...@gmail.com>:
>>
>> I had an error when I use Flatpack :
>>
>> 55891 [Thread: 1 org.apache.camel.component.file.FileComponent@1b14530]
>> ERROR org.apache.camel.processor.DeadLetterChannel  - Failed delivery for
>> exchangeId: ID-dell-charles/4143-1216120045671/0-0. On delivery attempt:
>> 5
>> caught: net.sf.flatpack.InitialisationException:
>> org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
>> Content is not allowed in prolog.
>> 43net.sf.flatpack.InitialisationException:
>> org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
>> Content is not allowed in prolog.
>>        at
>> net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:118)
>>        at net.sf.flatpack.AbstractParser.parse(AbstractParser.java:111)
>>        at
>> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:38)
>>        at
>> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>>        at
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:75)
>>        at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>>        at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>>        at
>> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>>        at
>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>>        at
>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>>        at
>> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
>>        at
>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
>>        at
>> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
>>        at
>> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
>>        at
>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
>>        at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
>>        at
>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
>>        at
>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
>>        at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
>>        at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
>>        at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>>        at java.lang.Thread.run(Thread.java:595)
>> Caused by: org.jdom.input.JDOMParseException: Error on line 1 of document
>> file:///: Content is not allowed in prolog.
>>        at org.jdom.input.SAXBuilder.build(SAXBuilder.java:465)
>>        at org.jdom.input.SAXBuilder.build(SAXBuilder.java:875)
>>        at net.sf.flatpack.xml.MapParser.parse(MapParser.java:139)
>>        at net.sf.flatpack.xml.MapParser.parseMap(MapParser.java:278)
>>        at
>> net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:107)
>>        ... 23 more
>> Caused by: org.xml.sax.SAXParseException: Content is not allowed in
>> prolog.
>>        at
>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:236)
>>        at
>> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:215)
>>        at
>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:386)
>>        at
>> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:316)
>>        at
>> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1438)
>>        at
>> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(XMLDocumentScannerImpl.java:899)
>>        at
>> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
>>        at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
>>        at
>> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
>>        at
>> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
>>        at
>> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
>>        at org.jdom.input.SAXBuilder.build(SAXBuilder.java:453)
>>        ... 27 more
>>
>> Here is the content of my route
>>
>>
>> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>>               
>> .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>>
>>
>>
>> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>>                .convertBodyTo(Map.class)
>>                .to("activemq:queue:cutas.in.queue");
>>
>> and xml file
>>
>> <?xml version='1.0'?>
>> <!DOCTYPE PZMAP SYSTEM  "flatpack.dtd" >
>>
>> <PZMAP>
>>        <RECORD id="header" startPosition="1" endPosition="3"
>> indicator="HBT">
>>                <COLUMN name="INDICATOR" length="3" />
>>                <COLUMN name="HEADERDATA" length="26" />
>>        </RECORD>
>>        <COLUMN name="RECORDTYPE" length="1" />
>>        <COLUMN name="SYSTEMID" length="3" />
>>        <COLUMN name="THIRDPARTYREF" length="15" />
>>        <COLUMN name="DATA" length="472" />
>>        <RECORD id="trailer" startPosition="1" endPosition="3"
>> indicator="FBT">
>>                <COLUMN name="INDICATOR" length="3" />
>>                <COLUMN name="TRAILERDATA" length="26" />
>>        </RECORD>
>>
>> </PZMAP>
>>
>> The problem is not at all related to Camel but any help would be
>> appreciate.
>>
>> Regards,
>>
>> Charles
>>
>>
>>
>> Claus Ibsen wrote:
>>>
>>> Hi
>>>
>>> Yes I thought also at first it was a dataformat ;) But how James created
>>> it it works as if it is a dataformat. You just have to do it in to..from
>>> from..to steps.
>>>
>>>
>>> from("file://c:/test").to("flatpack:foo:
>>> locationOftheXMLConfigurationfile.pzmap.xml");
>>>
>>> from("flatpack:foo").to("whatever you want");
>>>
>>> As a component it gets more attention, as people can find it on the
>>> component list. The dataformats are kinda hidden in the wiki
>>> documentation.
>>> Maybe they need a "dataformat" top menu as well ;)
>>>
>>> Med venlig hilsen
>>>
>>> Claus Ibsen
>>> ......................................
>>> Silverbullet
>>> Skovsgårdsvænget 21
>>> 8362 Hørning
>>> Tlf. +45 2962 7576
>>> Web: www.silverbullet.dk
>>>
>>> -----Original Message-----
>>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>>> Sent: 15. juli 2008 09:32
>>> To: camel-user@activemq.apache.org
>>> Subject: Re: Component to read FixedLength file ?
>>>
>>>
>>> Many thanks.
>>>
>>> I will test the integration of this component within Camel.
>>>
>>> Remark : I'm a little confused about how Flatpack has been integrated.
>>> The
>>> project has been integrated like a component but from my point of view,
>>> it
>>> should be integrated (for consistency reason) like a Dataformat. Because
>>> like the other framework/technology implemented in Camel (JAXB, Artix,
>>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>>> into another format (Collection of Java objects)
>>>
>>> E.g.
>>>
>>> DataFormat flatpack = new
>>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>>
>>> from("file:///c:/test")
>>> .unmarshall(flatpack)
>>> .to("whatever you want")
>>>
>>> Regards,
>>>
>>> Charles
>>>
>>>
>>>
>>> James.Strachan wrote:
>>>>
>>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>>> but there's a basic fixed-width reader example there for the curious
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> I've raised a JIRA to track this...
>>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>>
>>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a component expected to read fixedlength files in Camel
>>>>>>> like
>>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>>
>>>>>> It should be pretty easy to write a flatpack component.
>>>>>>
>>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>>> a nice design UI that does this with Camel...
>>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>>
>>>>>> but it'd be nice to have an OSS option :)
>>>>>>
>>>>>> --
>>>>>> James
>>>>>> -------
>>>>>> http://macstrac.blogspot.com/
>>>>>>
>>>>>> Open Source Integration
>>>>>> http://open.iona.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>>
>>>
>>>
>>> -----
>>> Enterprise Architect
>>>
>>> Xpectis
>>> 12, route d'Esch
>>> L-1470 Luxembourg
>>>
>>> Phone +352 25 10 70 470
>>> Mobile +352 621 45 36 22
>>>
>>> e-mail : cmoulliard@xpectis.com
>>> web site : www.xpectis.com
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>>
>>
>>
>> -----
>> Enterprise Architect
>>
>> Xpectis
>> 12, route d'Esch
>> L-1470 Luxembourg
>>
>> Phone +352 25 10 70 470
>> Mobile +352 621 45 36 22
>>
>> e-mail : cmoulliard@xpectis.com
>> web site : www.xpectis.com
>> --
>> View this message in context:
>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18462576.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18466472.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
Are you sure the XML file is valid? e.g. there's no newlines before
the <?xml prolog?

2008/7/15 cmoulliard <cm...@gmail.com>:
>
> I had an error when I use Flatpack :
>
> 55891 [Thread: 1 org.apache.camel.component.file.FileComponent@1b14530]
> ERROR org.apache.camel.processor.DeadLetterChannel  - Failed delivery for
> exchangeId: ID-dell-charles/4143-1216120045671/0-0. On delivery attempt: 5
> caught: net.sf.flatpack.InitialisationException:
> org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
> Content is not allowed in prolog.
> 43net.sf.flatpack.InitialisationException:
> org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
> Content is not allowed in prolog.
>        at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:118)
>        at net.sf.flatpack.AbstractParser.parse(AbstractParser.java:111)
>        at
> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:38)
>        at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
>        at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:75)
>        at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>        at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
>        at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
>        at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
>        at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
>        at
> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
>        at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
>        at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
>        at org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
>        at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
>        at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
>        at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
>        at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
>        at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>        at java.lang.Thread.run(Thread.java:595)
> Caused by: org.jdom.input.JDOMParseException: Error on line 1 of document
> file:///: Content is not allowed in prolog.
>        at org.jdom.input.SAXBuilder.build(SAXBuilder.java:465)
>        at org.jdom.input.SAXBuilder.build(SAXBuilder.java:875)
>        at net.sf.flatpack.xml.MapParser.parse(MapParser.java:139)
>        at net.sf.flatpack.xml.MapParser.parseMap(MapParser.java:278)
>        at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:107)
>        ... 23 more
> Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
>        at
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:236)
>        at
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:215)
>        at
> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:386)
>        at
> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:316)
>        at
> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1438)
>        at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(XMLDocumentScannerImpl.java:899)
>        at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
>        at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
>        at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
>        at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
>        at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
>        at org.jdom.input.SAXBuilder.build(SAXBuilder.java:453)
>        ... 27 more
>
> Here is the content of my route
>
>
> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
>                .to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
>
>
>
> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
>                .convertBodyTo(Map.class)
>                .to("activemq:queue:cutas.in.queue");
>
> and xml file
>
> <?xml version='1.0'?>
> <!DOCTYPE PZMAP SYSTEM  "flatpack.dtd" >
>
> <PZMAP>
>        <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
>                <COLUMN name="INDICATOR" length="3" />
>                <COLUMN name="HEADERDATA" length="26" />
>        </RECORD>
>        <COLUMN name="RECORDTYPE" length="1" />
>        <COLUMN name="SYSTEMID" length="3" />
>        <COLUMN name="THIRDPARTYREF" length="15" />
>        <COLUMN name="DATA" length="472" />
>        <RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
>                <COLUMN name="INDICATOR" length="3" />
>                <COLUMN name="TRAILERDATA" length="26" />
>        </RECORD>
>
> </PZMAP>
>
> The problem is not at all related to Camel but any help would be appreciate.
>
> Regards,
>
> Charles
>
>
>
> Claus Ibsen wrote:
>>
>> Hi
>>
>> Yes I thought also at first it was a dataformat ;) But how James created
>> it it works as if it is a dataformat. You just have to do it in to..from
>> from..to steps.
>>
>>
>> from("file://c:/test").to("flatpack:foo:
>> locationOftheXMLConfigurationfile.pzmap.xml");
>>
>> from("flatpack:foo").to("whatever you want");
>>
>> As a component it gets more attention, as people can find it on the
>> component list. The dataformats are kinda hidden in the wiki
>> documentation.
>> Maybe they need a "dataformat" top menu as well ;)
>>
>> Med venlig hilsen
>>
>> Claus Ibsen
>> ......................................
>> Silverbullet
>> Skovsgårdsvænget 21
>> 8362 Hørning
>> Tlf. +45 2962 7576
>> Web: www.silverbullet.dk
>>
>> -----Original Message-----
>> From: cmoulliard [mailto:cmoulliard@gmail.com]
>> Sent: 15. juli 2008 09:32
>> To: camel-user@activemq.apache.org
>> Subject: Re: Component to read FixedLength file ?
>>
>>
>> Many thanks.
>>
>> I will test the integration of this component within Camel.
>>
>> Remark : I'm a little confused about how Flatpack has been integrated. The
>> project has been integrated like a component but from my point of view, it
>> should be integrated (for consistency reason) like a Dataformat. Because
>> like the other framework/technology implemented in Camel (JAXB, Artix,
>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>> into another format (Collection of Java objects)
>>
>> E.g.
>>
>> DataFormat flatpack = new
>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>>
>> from("file:///c:/test")
>> .unmarshall(flatpack)
>> .to("whatever you want")
>>
>> Regards,
>>
>> Charles
>>
>>
>>
>> James.Strachan wrote:
>>>
>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>> but there's a basic fixed-width reader example there for the curious
>>>
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> I've raised a JIRA to track this...
>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>
>>>>> It should be pretty easy to write a flatpack component.
>>>>>
>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>> a nice design UI that does this with Camel...
>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>
>>>>> but it'd be nice to have an OSS option :)
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>>
>>
>>
>> -----
>> Enterprise Architect
>>
>> Xpectis
>> 12, route d'Esch
>> L-1470 Luxembourg
>>
>> Phone +352 25 10 70 470
>> Mobile +352 621 45 36 22
>>
>> e-mail : cmoulliard@xpectis.com
>> web site : www.xpectis.com
>> --
>> View this message in context:
>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>>
>
>
> -----
> Enterprise Architect
>
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
>
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
>
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com
> --
> View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18462576.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

RE: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
Do you think that its makes sense to use Xerces instead of what sun proposes
in its JDK 1.5 ?

If this is the case, how can I switch from Sun to Xerces ? What will be the
impact on Camel's components like JAXB, .... ?


Claus Ibsen wrote:
> 
> Its probably those XML parsers. Make sure you are using the correct/never
> version of Xerces. 
> 
> What version of Java are you using? SUN bundles their XML framework in
> Java also.
> 
> To find help search for "Content is not allowed in prolog.". It has been
> reported many times.
> 
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 15. juli 2008 13:14
> To: camel-user@activemq.apache.org
> Subject: RE: Component to read FixedLength file ?
> 
> 
> I had an error when I use Flatpack :
> 
> 55891 [Thread: 1 org.apache.camel.component.file.FileComponent@1b14530]
> ERROR org.apache.camel.processor.DeadLetterChannel  - Failed delivery for
> exchangeId: ID-dell-charles/4143-1216120045671/0-0. On delivery attempt: 5
> caught: net.sf.flatpack.InitialisationException:
> org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
> Content is not allowed in prolog.
> 43net.sf.flatpack.InitialisationException:
> org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
> Content is not allowed in prolog.
> 	at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:118)
> 	at net.sf.flatpack.AbstractParser.parse(AbstractParser.java:111)
> 	at
> org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:38)
> 	at
> org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
> 	at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:75)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
> 	at
> org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
> 	at
> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
> 	at
> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
> 	at
> org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
> 	at
> org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
> 	at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
> 	at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
> 	at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
> 	at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
> 	at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
> 	at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
> 	at java.lang.Thread.run(Thread.java:595)
> Caused by: org.jdom.input.JDOMParseException: Error on line 1 of document
> file:///: Content is not allowed in prolog.
> 	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:465)
> 	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:875)
> 	at net.sf.flatpack.xml.MapParser.parse(MapParser.java:139)
> 	at net.sf.flatpack.xml.MapParser.parseMap(MapParser.java:278)
> 	at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:107)
> 	... 23 more
> Caused by: org.xml.sax.SAXParseException: Content is not allowed in
> prolog.
> 	at
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:236)
> 	at
> com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:215)
> 	at
> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:386)
> 	at
> com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:316)
> 	at
> com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1438)
> 	at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(XMLDocumentScannerImpl.java:899)
> 	at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
> 	at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
> 	at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
> 	at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
> 	at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
> 	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:453)
> 	... 27 more
> 
> Here is the content of my route 
> 
> 	
> from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
> 		.to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
> 		
> 		
> 	
> from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
> 		.convertBodyTo(Map.class)
> 		.to("activemq:queue:cutas.in.queue");
> 
> and xml file
> 
> <?xml version='1.0'?>
> <!DOCTYPE PZMAP SYSTEM	"flatpack.dtd" >
> 
> <PZMAP>
> 	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="HEADERDATA" length="26" />
> 	</RECORD>
> 	<COLUMN name="RECORDTYPE" length="1" />
> 	<COLUMN name="SYSTEMID" length="3" />
> 	<COLUMN name="THIRDPARTYREF" length="15" />
> 	<COLUMN name="DATA" length="472" />
> 	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
> 		<COLUMN name="INDICATOR" length="3" />
> 		<COLUMN name="TRAILERDATA" length="26" />
> 	</RECORD>
> 
> </PZMAP>
> 
> The problem is not at all related to Camel but any help would be
> appreciate.
> 
> Regards,
> 
> Charles
> 
> 
> 
> Claus Ibsen wrote:
>> 
>> Hi
>> 
>> Yes I thought also at first it was a dataformat ;) But how James created
>> it it works as if it is a dataformat. You just have to do it in to..from
>> from..to steps.
>> 
>> 
>> from("file://c:/test").to("flatpack:foo:
>> locationOftheXMLConfigurationfile.pzmap.xml");
>> 
>> from("flatpack:foo").to("whatever you want");
>> 
>> As a component it gets more attention, as people can find it on the
>> component list. The dataformats are kinda hidden in the wiki
>> documentation.
>> Maybe they need a "dataformat" top menu as well ;)
>> 
>> Med venlig hilsen
>>  
>> Claus Ibsen
>> ......................................
>> Silverbullet
>> Skovsgårdsvænget 21
>> 8362 Hørning
>> Tlf. +45 2962 7576
>> Web: www.silverbullet.dk
>> 
>> -----Original Message-----
>> From: cmoulliard [mailto:cmoulliard@gmail.com] 
>> Sent: 15. juli 2008 09:32
>> To: camel-user@activemq.apache.org
>> Subject: Re: Component to read FixedLength file ?
>> 
>> 
>> Many thanks.
>> 
>> I will test the integration of this component within Camel.
>> 
>> Remark : I'm a little confused about how Flatpack has been integrated.
>> The
>> project has been integrated like a component but from my point of view,
>> it
>> should be integrated (for consistency reason) like a Dataformat. Because
>> like the other framework/technology implemented in Camel (JAXB, Artix,
>> XStream, ....), they transform the content from a format (XML, CSV, ...)
>> into another format (Collection of Java objects)
>> 
>> E.g.
>> 
>> DataFormat flatpack = new
>> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
>> 
>> from("file:///c:/test")
>> .unmarshall(flatpack)
>> .to("whatever you want")
>> 
>> Regards,
>> 
>> Charles
>> 
>> 
>> 
>> James.Strachan wrote:
>>> 
>>> BTW trunk has an early spike of a flatpack component. Needs more work,
>>> but there's a basic fixed-width reader example there for the curious
>>> 
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> I've raised a JIRA to track this...
>>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>>
>>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>>
>>>>> It should be pretty easy to write a flatpack component.
>>>>>
>>>>> there's a commercial tool called Artix Data Services which comes with
>>>>> a nice design UI that does this with Camel...
>>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>>
>>>>> but it'd be nice to have an OSS option :)
>>>>>
>>>>> --
>>>>> James
>>>>> -------
>>>>> http://macstrac.blogspot.com/
>>>>>
>>>>> Open Source Integration
>>>>> http://open.iona.com
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>> 
>>> 
>>> 
>>> -- 
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>> 
>>> Open Source Integration
>>> http://open.iona.com
>>> 
>>> 
>> 
>> 
>> -----
>> Enterprise Architect
>> 
>> Xpectis
>> 12, route d'Esch
>> L-1470 Luxembourg
>> 
>> Phone +352 25 10 70 470
>> Mobile +352 621 45 36 22
>> 
>> e-mail : cmoulliard@xpectis.com
>> web site : www.xpectis.com 
>> -- 
>> View this message in context:
>> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>> 
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18462576.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18463994.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Its probably those XML parsers. Make sure you are using the correct/never version of Xerces. 

What version of Java are you using? SUN bundles their XML framework in Java also.

To find help search for "Content is not allowed in prolog.". It has been reported many times.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
-----Original Message-----
From: cmoulliard [mailto:cmoulliard@gmail.com] 
Sent: 15. juli 2008 13:14
To: camel-user@activemq.apache.org
Subject: RE: Component to read FixedLength file ?


I had an error when I use Flatpack :

55891 [Thread: 1 org.apache.camel.component.file.FileComponent@1b14530]
ERROR org.apache.camel.processor.DeadLetterChannel  - Failed delivery for
exchangeId: ID-dell-charles/4143-1216120045671/0-0. On delivery attempt: 5
caught: net.sf.flatpack.InitialisationException:
org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
Content is not allowed in prolog.
43net.sf.flatpack.InitialisationException:
org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
Content is not allowed in prolog.
	at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:118)
	at net.sf.flatpack.AbstractParser.parse(AbstractParser.java:111)
	at
org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:38)
	at
org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
	at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:75)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
	at
org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
	at org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
	at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
	at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)
Caused by: org.jdom.input.JDOMParseException: Error on line 1 of document
file:///: Content is not allowed in prolog.
	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:465)
	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:875)
	at net.sf.flatpack.xml.MapParser.parse(MapParser.java:139)
	at net.sf.flatpack.xml.MapParser.parseMap(MapParser.java:278)
	at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:107)
	... 23 more
Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
	at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:236)
	at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:215)
	at
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:386)
	at
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:316)
	at
com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1438)
	at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(XMLDocumentScannerImpl.java:899)
	at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
	at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
	at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
	at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
	at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:453)
	... 27 more

Here is the content of my route 

	
from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
		.to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
		
		
	
from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
		.convertBodyTo(Map.class)
		.to("activemq:queue:cutas.in.queue");

and xml file

<?xml version='1.0'?>
<!DOCTYPE PZMAP SYSTEM	"flatpack.dtd" >

<PZMAP>
	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="HEADERDATA" length="26" />
	</RECORD>
	<COLUMN name="RECORDTYPE" length="1" />
	<COLUMN name="SYSTEMID" length="3" />
	<COLUMN name="THIRDPARTYREF" length="15" />
	<COLUMN name="DATA" length="472" />
	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="TRAILERDATA" length="26" />
	</RECORD>

</PZMAP>

The problem is not at all related to Camel but any help would be appreciate.

Regards,

Charles



Claus Ibsen wrote:
> 
> Hi
> 
> Yes I thought also at first it was a dataformat ;) But how James created
> it it works as if it is a dataformat. You just have to do it in to..from
> from..to steps.
> 
> 
> from("file://c:/test").to("flatpack:foo:
> locationOftheXMLConfigurationfile.pzmap.xml");
> 
> from("flatpack:foo").to("whatever you want");
> 
> As a component it gets more attention, as people can find it on the
> component list. The dataformats are kinda hidden in the wiki
> documentation.
> Maybe they need a "dataformat" top menu as well ;)
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 15. juli 2008 09:32
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> 
> Many thanks.
> 
> I will test the integration of this component within Camel.
> 
> Remark : I'm a little confused about how Flatpack has been integrated. The
> project has been integrated like a component but from my point of view, it
> should be integrated (for consistency reason) like a Dataformat. Because
> like the other framework/technology implemented in Camel (JAXB, Artix,
> XStream, ....), they transform the content from a format (XML, CSV, ...)
> into another format (Collection of Java objects)
> 
> E.g.
> 
> DataFormat flatpack = new
> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
> 
> from("file:///c:/test")
> .unmarshall(flatpack)
> .to("whatever you want")
> 
> Regards,
> 
> Charles
> 
> 
> 
> James.Strachan wrote:
>> 
>> BTW trunk has an early spike of a flatpack component. Needs more work,
>> but there's a basic fixed-width reader example there for the curious
>> 
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> I've raised a JIRA to track this...
>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>
>>>> It should be pretty easy to write a flatpack component.
>>>>
>>>> there's a commercial tool called Artix Data Services which comes with
>>>> a nice design UI that does this with Camel...
>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>
>>>> but it'd be nice to have an OSS option :)
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>> 
>> 
>> 
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>> 
>> Open Source Integration
>> http://open.iona.com
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18462576.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
I had an error when I use Flatpack :

55891 [Thread: 1 org.apache.camel.component.file.FileComponent@1b14530]
ERROR org.apache.camel.processor.DeadLetterChannel  - Failed delivery for
exchangeId: ID-dell-charles/4143-1216120045671/0-0. On delivery attempt: 5
caught: net.sf.flatpack.InitialisationException:
org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
Content is not allowed in prolog.
43net.sf.flatpack.InitialisationException:
org.jdom.input.JDOMParseException: Error on line 1 of document file:///:
Content is not allowed in prolog.
	at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:118)
	at net.sf.flatpack.AbstractParser.parse(AbstractParser.java:111)
	at
org.apache.camel.component.flatpack.FlatpackProducer.process(FlatpackProducer.java:38)
	at
org.apache.camel.impl.converter.AsyncProcessorTypeConverter$ProcessorToAsyncProcessorBridge.process(AsyncProcessorTypeConverter.java:43)
	at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:75)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:155)
	at
org.apache.camel.processor.DeadLetterChannel.process(DeadLetterChannel.java:91)
	at
org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:57)
	at
org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:39)
	at
org.apache.camel.component.file.FileConsumer.pollFile(FileConsumer.java:128)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:77)
	at
org.apache.camel.component.file.FileConsumer.pollFileOrDirectory(FileConsumer.java:84)
	at org.apache.camel.component.file.FileConsumer.poll(FileConsumer.java:57)
	at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:65)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
	at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
	at java.lang.Thread.run(Thread.java:595)
Caused by: org.jdom.input.JDOMParseException: Error on line 1 of document
file:///: Content is not allowed in prolog.
	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:465)
	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:875)
	at net.sf.flatpack.xml.MapParser.parse(MapParser.java:139)
	at net.sf.flatpack.xml.MapParser.parseMap(MapParser.java:278)
	at net.sf.flatpack.FixedLengthParser.init(FixedLengthParser.java:107)
	... 23 more
Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
	at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:236)
	at
com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:215)
	at
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:386)
	at
com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:316)
	at
com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1438)
	at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDispatcher.dispatch(XMLDocumentScannerImpl.java:899)
	at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:368)
	at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:834)
	at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
	at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:148)
	at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1242)
	at org.jdom.input.SAXBuilder.build(SAXBuilder.java:453)
	... 27 more

Here is the content of my route 

	
from("file:///c:/temp/cutas/?delete=false&lock=true&consumer.delay=500&consumer.initialDelay=10")
		.to("flatpack:fixed:com/company/transformation/in/CUTAS-ST.pzmap.xml");
		
		
	
from("flatpack:fixed:com/company/cutas/transformation/in/CUTAS-ST.pzmap.xml")
		.convertBodyTo(Map.class)
		.to("activemq:queue:cutas.in.queue");

and xml file

<?xml version='1.0'?>
<!DOCTYPE PZMAP SYSTEM	"flatpack.dtd" >

<PZMAP>
	<RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="HEADERDATA" length="26" />
	</RECORD>
	<COLUMN name="RECORDTYPE" length="1" />
	<COLUMN name="SYSTEMID" length="3" />
	<COLUMN name="THIRDPARTYREF" length="15" />
	<COLUMN name="DATA" length="472" />
	<RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
		<COLUMN name="INDICATOR" length="3" />
		<COLUMN name="TRAILERDATA" length="26" />
	</RECORD>

</PZMAP>

The problem is not at all related to Camel but any help would be appreciate.

Regards,

Charles



Claus Ibsen wrote:
> 
> Hi
> 
> Yes I thought also at first it was a dataformat ;) But how James created
> it it works as if it is a dataformat. You just have to do it in to..from
> from..to steps.
> 
> 
> from("file://c:/test").to("flatpack:foo:
> locationOftheXMLConfigurationfile.pzmap.xml");
> 
> from("flatpack:foo").to("whatever you want");
> 
> As a component it gets more attention, as people can find it on the
> component list. The dataformats are kinda hidden in the wiki
> documentation.
> Maybe they need a "dataformat" top menu as well ;)
> 
> Med venlig hilsen
>  
> Claus Ibsen
> ......................................
> Silverbullet
> Skovsgårdsvænget 21
> 8362 Hørning
> Tlf. +45 2962 7576
> Web: www.silverbullet.dk
> 
> -----Original Message-----
> From: cmoulliard [mailto:cmoulliard@gmail.com] 
> Sent: 15. juli 2008 09:32
> To: camel-user@activemq.apache.org
> Subject: Re: Component to read FixedLength file ?
> 
> 
> Many thanks.
> 
> I will test the integration of this component within Camel.
> 
> Remark : I'm a little confused about how Flatpack has been integrated. The
> project has been integrated like a component but from my point of view, it
> should be integrated (for consistency reason) like a Dataformat. Because
> like the other framework/technology implemented in Camel (JAXB, Artix,
> XStream, ....), they transform the content from a format (XML, CSV, ...)
> into another format (Collection of Java objects)
> 
> E.g.
> 
> DataFormat flatpack = new
> DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")
> 
> from("file:///c:/test")
> .unmarshall(flatpack)
> .to("whatever you want")
> 
> Regards,
> 
> Charles
> 
> 
> 
> James.Strachan wrote:
>> 
>> BTW trunk has an early spike of a flatpack component. Needs more work,
>> but there's a basic fixed-width reader example there for the curious
>> 
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> I've raised a JIRA to track this...
>>> https://issues.apache.org/activemq/browse/CAMEL-717
>>>
>>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Is there a component expected to read fixedlength files in Camel like
>>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>>
>>>> It should be pretty easy to write a flatpack component.
>>>>
>>>> there's a commercial tool called Artix Data Services which comes with
>>>> a nice design UI that does this with Camel...
>>>> http://activemq.apache.org/camel/artix-data-services.html
>>>>
>>>> but it'd be nice to have an OSS option :)
>>>>
>>>> --
>>>> James
>>>> -------
>>>> http://macstrac.blogspot.com/
>>>>
>>>> Open Source Integration
>>>> http://open.iona.com
>>>>
>>>
>>>
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>> 
>> 
>> 
>> -- 
>> James
>> -------
>> http://macstrac.blogspot.com/
>> 
>> Open Source Integration
>> http://open.iona.com
>> 
>> 
> 
> 
> -----
> Enterprise Architect
> 
> Xpectis
> 12, route d'Esch
> L-1470 Luxembourg
> 
> Phone +352 25 10 70 470
> Mobile +352 621 45 36 22
> 
> e-mail : cmoulliard@xpectis.com
> web site : www.xpectis.com 
> -- 
> View this message in context:
> http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18462576.html
Sent from the Camel - Users mailing list archive at Nabble.com.


RE: Component to read FixedLength file ?

Posted by Claus Ibsen <ci...@silverbullet.dk>.
Hi

Yes I thought also at first it was a dataformat ;) But how James created it it works as if it is a dataformat. You just have to do it in to..from from..to steps.


from("file://c:/test").to("flatpack:foo: locationOftheXMLConfigurationfile.pzmap.xml");

from("flatpack:foo").to("whatever you want");

As a component it gets more attention, as people can find it on the component list. The dataformats are kinda hidden in the wiki documentation.
Maybe they need a "dataformat" top menu as well ;)

Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: cmoulliard [mailto:cmoulliard@gmail.com] 
Sent: 15. juli 2008 09:32
To: camel-user@activemq.apache.org
Subject: Re: Component to read FixedLength file ?


Many thanks.

I will test the integration of this component within Camel.

Remark : I'm a little confused about how Flatpack has been integrated. The
project has been integrated like a component but from my point of view, it
should be integrated (for consistency reason) like a Dataformat. Because
like the other framework/technology implemented in Camel (JAXB, Artix,
XStream, ....), they transform the content from a format (XML, CSV, ...)
into another format (Collection of Java objects)

E.g.

DataFormat flatpack = new
DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")

from("file:///c:/test")
.unmarshall(flatpack)
.to("whatever you want")

Regards,

Charles



James.Strachan wrote:
> 
> BTW trunk has an early spike of a flatpack component. Needs more work,
> but there's a basic fixed-width reader example there for the curious
> 
> 2008/7/14 James Strachan <ja...@gmail.com>:
>> I've raised a JIRA to track this...
>> https://issues.apache.org/activemq/browse/CAMEL-717
>>
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>
>>>> Hi,
>>>>
>>>> Is there a component expected to read fixedlength files in Camel like
>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>
>>> It should be pretty easy to write a flatpack component.
>>>
>>> there's a commercial tool called Artix Data Services which comes with
>>> a nice design UI that does this with Camel...
>>> http://activemq.apache.org/camel/artix-data-services.html
>>>
>>> but it'd be nice to have an OSS option :)
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by cmoulliard <cm...@gmail.com>.
Many thanks.

I will test the integration of this component within Camel.

Remark : I'm a little confused about how Flatpack has been integrated. The
project has been integrated like a component but from my point of view, it
should be integrated (for consistency reason) like a Dataformat. Because
like the other framework/technology implemented in Camel (JAXB, Artix,
XStream, ....), they transform the content from a format (XML, CSV, ...)
into another format (Collection of Java objects)

E.g.

DataFormat flatpack = new
DataFormat("locationOftheXMLConfigurationfile.pzmap.xml.xml")

from("file:///c:/test")
.unmarshall(flatpack)
.to("whatever you want")

Regards,

Charles



James.Strachan wrote:
> 
> BTW trunk has an early spike of a flatpack component. Needs more work,
> but there's a basic fixed-width reader example there for the curious
> 
> 2008/7/14 James Strachan <ja...@gmail.com>:
>> I've raised a JIRA to track this...
>> https://issues.apache.org/activemq/browse/CAMEL-717
>>
>> 2008/7/14 James Strachan <ja...@gmail.com>:
>>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>>
>>>> Hi,
>>>>
>>>> Is there a component expected to read fixedlength files in Camel like
>>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>>
>>> It should be pretty easy to write a flatpack component.
>>>
>>> there's a commercial tool called Artix Data Services which comes with
>>> a nice design UI that does this with Camel...
>>> http://activemq.apache.org/camel/artix-data-services.html
>>>
>>> but it'd be nice to have an OSS option :)
>>>
>>> --
>>> James
>>> -------
>>> http://macstrac.blogspot.com/
>>>
>>> Open Source Integration
>>> http://open.iona.com
>>>
>>
>>
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
> 
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 


-----
Enterprise Architect

Xpectis
12, route d'Esch
L-1470 Luxembourg

Phone +352 25 10 70 470
Mobile +352 621 45 36 22

e-mail : cmoulliard@xpectis.com
web site : www.xpectis.com 
-- 
View this message in context: http://www.nabble.com/Component-to-read-FixedLength-file---tp18444614s22882p18459545.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
BTW trunk has an early spike of a flatpack component. Needs more work,
but there's a basic fixed-width reader example there for the curious

2008/7/14 James Strachan <ja...@gmail.com>:
> I've raised a JIRA to track this...
> https://issues.apache.org/activemq/browse/CAMEL-717
>
> 2008/7/14 James Strachan <ja...@gmail.com>:
>> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>>
>>> Hi,
>>>
>>> Is there a component expected to read fixedlength files in Camel like
>>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>>
>> It should be pretty easy to write a flatpack component.
>>
>> there's a commercial tool called Artix Data Services which comes with
>> a nice design UI that does this with Camel...
>> http://activemq.apache.org/camel/artix-data-services.html
>>
>> but it'd be nice to have an OSS option :)
>>
>> --
>> James
>> -------
>> http://macstrac.blogspot.com/
>>
>> Open Source Integration
>> http://open.iona.com
>>
>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
I've raised a JIRA to track this...
https://issues.apache.org/activemq/browse/CAMEL-717

2008/7/14 James Strachan <ja...@gmail.com>:
> 2008/7/14 cmoulliard <cm...@gmail.com>:
>>
>> Hi,
>>
>> Is there a component expected to read fixedlength files in Camel like
>> flatpack (http://flatpack.sourceforge.net/) allow to do it ?
>
> It should be pretty easy to write a flatpack component.
>
> there's a commercial tool called Artix Data Services which comes with
> a nice design UI that does this with Camel...
> http://activemq.apache.org/camel/artix-data-services.html
>
> but it'd be nice to have an OSS option :)
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Component to read FixedLength file ?

Posted by James Strachan <ja...@gmail.com>.
2008/7/14 cmoulliard <cm...@gmail.com>:
>
> Hi,
>
> Is there a component expected to read fixedlength files in Camel like
> flatpack (http://flatpack.sourceforge.net/) allow to do it ?

It should be pretty easy to write a flatpack component.

there's a commercial tool called Artix Data Services which comes with
a nice design UI that does this with Camel...
http://activemq.apache.org/camel/artix-data-services.html

but it'd be nice to have an OSS option :)

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com