You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Louisa <Le...@hotmail.fr> on 2016/09/05 15:18:58 UTC
Cut a file into multiple files
Hello evereybody,
I use Java Camel to my project (Talend).
I need your help for a method to solve this problem. I have this file:
D0000000000000000000
EEASTDE
ELASPDF
ELASQML
EEASTDE
ELASPDF
ELASQHG
F00000006
How I can cut in java this way (in 2 file) ?
EEASTDE
ELASPDF
ELASQML
And:
EEASTDE
ELASPDF
ELASQHG
I have to cut the file according to the head EEASTDE. It's an example, I
could have several times the header EEASTDE (3, 4, 5...).
I thought of the HashMap, the arraylist but I don't know how to make the
code because I'm not indicate the line EEASTDE as a unique key.
Here is my test code but it doesn't work:
public class bean_test implements Processor{
private static final String MDS_ENDPOINT_NAME = "MDSEndpoint";
private static final String EEASTD_HEADER = "EEASTDE";
private static final int EEASTD_HEADER_START_POSITION = 0;
private static final int EEASTD_HEADER_END_POSITION = 7;
private static final String START_OF_FILE_TAG = "D";
private static final String END_OF_FILE_TAG = "F";
@Override
public void process(Exchange exchange) throws Exception {
ProducerTemplate producerTemplate =
exchange.getContext().createProducerTemplate();
String MDSEndpoint = exchange.getIn().getHeader(MDS_ENDPOINT_NAME,
String.class);
InputStream is = new
ByteArrayInputStream(exchange.getIn().getBody(String.class).getBytes());
extractFromMDS(producerTemplate, is, MDSEndpoint, new HashMap<String,
Object>(exchange.getIn().getHeaders()));
}
private void extractFromMDS(ProducerTemplate producerTemplate, InputStream
content, String MDSEndpoint, Map<String, Object> headers){
BufferedReader br = new BufferedReader(new InputStreamReader(content));
String line;
try {
while((line = br.readLine()) != null){
if(!line.startsWith(START_OF_FILE_TAG) &&
!line.startsWith(END_OF_FILE_TAG)){
StringBuilder stringBuilder = null;
if(line.substring(EEASTD_HEADER_START_POSITION,
EEASTD_HEADER_END_POSITION).equals(EEASTD_HEADER)){
stringBuilder = new StringBuilder();
stringBuilder.append("\n"+(line));
}
producerTemplate.sendBodyAndHeaders(MDSEndpoint,
stringBuilder.toString(), headers);
}
}
} catch (IOException e) {
e.printStackTrace();
}
finally{
try {
if(br != null)br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
Have you an idea ?
Thank you in advance.
Natacha.
--
View this message in context: http://camel.465427.n5.nabble.com/Cut-a-file-into-multiple-files-tp5787210.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Cut a file into multiple files
Posted by Brad Johnson <br...@mediadriver.com>.
As Chris said tthe splitter can work well if this is a simple file. Are
all new records starting with EEASTDE? If so then a splitter with
tokenize("\n") would cut every single line and send it down the Camel
route. You could then have a simple handler bean like:
public class BeanBuilder {
@EndpointInject(uri="seda:output")
ProducerTemplate producer;
private RecordBean recordBean;
private int fileCounter;
public void handleLine(String line){
if("EEASTDE".equals(line){
//do logic to finish previous bean and send it if it exists and create a
new recordBean.
//have a setFileName on your record bean and differentiate by setting the
fileName("record"+i+".txt");
//Inside your record bean have a String buffer that permits you to keep
adding lines which you append with line+"\n".
//at some point you write out your bean.
producer.sendBody(recordBean); //Obviously you have to have a second route
defined as from("seda:output") in order for that to work.
}
}
The only real thing you have to take care of in your handler is to check if
you already have a bean or if it is null when you get the "EEASTDE". If it
is null, you just create a new one and set the fileName on it. If you
already have one you send it to the next route via the ProducerTemplate,
create new bean and set it's file name.
In the receiving route set the header to the fileName, then transform the
bean via its toString. That should just dump the internal StringBuffer you
have inside it.
There are lot of ways to do this in Camel. That's its beauty and
sometimes its frustration. But if you find yourself doing all the heavy
lifting and I/O in a Processor you are almost always doing it in a way that
isn't using the framework for what it does well.
On Tue, Sep 6, 2016 at 5:26 PM, Brad Johnson <br...@mediadriver.com>
wrote:
> I'd definitely look at Beanio and the Camel bundle/library for it. It's
> relatively easy to create identifiers in that way and segregate those out
> as beans. So every bean class might have a header (or not since it seems
> static and throwaway) and then 0...M lines that could be mapped into the
> bean. Then you end up with a bean for every set of records that you can
> route and write out. You could then use a file endpoint in the route
> instead of trying to write them out yourself. That also means that handlers
> and validators down the line do not have to be Processors requiring the
> handling of a raw exchange but can be beans themselves that just have a
> method that takes your bean like as follows. Then you can also unit test
> the individual handlers/validators or whatever you want to call them
> independent of any Camel libraries or routes.
>
> public class BeanHandler {
>
> public void invoke(MyBeanModel model)
> {
> //some logic. You have even return the model or a different one.
> }
>
> }
>
> On Mon, Sep 5, 2016 at 4:54 PM, Chris Odom <ch...@mediadriver.com>
> wrote:
>
>> I would try using the Camel EIP Splitter
>>
>> http://camel.apache.org/splitter.html
>>
>>
>> *Chris Odom | Sr Consultant | Media Driver*
>> 512.799.0270 | chris.odom@mediadriver.com
>> 810 Hester's Crossing, Suite 165, Round Rock, TX 78681
>>
>> On Mon, Sep 5, 2016 at 10:18 AM, Louisa <Le...@hotmail.fr> wrote:
>>
>> > Hello evereybody,
>> >
>> > I use Java Camel to my project (Talend).
>> > I need your help for a method to solve this problem. I have this file:
>> >
>> > D0000000000000000000
>> > EEASTDE
>> > ELASPDF
>> > ELASQML
>> > EEASTDE
>> > ELASPDF
>> > ELASQHG
>> > F00000006
>> >
>> > How I can cut in java this way (in 2 file) ?
>> >
>> > EEASTDE
>> > ELASPDF
>> > ELASQML
>> >
>> > And:
>> >
>> > EEASTDE
>> > ELASPDF
>> > ELASQHG
>> >
>> >
>> > I have to cut the file according to the head EEASTDE. It's an example, I
>> > could have several times the header EEASTDE (3, 4, 5...).
>> > I thought of the HashMap, the arraylist but I don't know how to make the
>> > code because I'm not indicate the line EEASTDE as a unique key.
>> >
>> > Here is my test code but it doesn't work:
>> >
>> >
>> > public class bean_test implements Processor{
>> >
>> >
>> > private static final String MDS_ENDPOINT_NAME = "MDSEndpoint";
>> > private static final String EEASTD_HEADER = "EEASTDE";
>> > private static final int EEASTD_HEADER_START_POSITION = 0;
>> > private static final int EEASTD_HEADER_END_POSITION = 7;
>> > private static final String START_OF_FILE_TAG = "D";
>> > private static final String END_OF_FILE_TAG = "F";
>> >
>> > @Override
>> > public void process(Exchange exchange) throws Exception {
>> >
>> > ProducerTemplate producerTemplate =
>> > exchange.getContext().createProducerTemplate();
>> > String MDSEndpoint = exchange.getIn().getHeader(
>> > MDS_ENDPOINT_NAME,
>> > String.class);
>> > InputStream is = new
>> > ByteArrayInputStream(exchange.getIn().getBody(String.class).
>> getBytes());
>> > extractFromMDS(producerTemplate, is, MDSEndpoint, new
>> > HashMap<String,
>> > Object>(exchange.getIn().getHeaders()));
>> >
>> > }
>> >
>> > private void extractFromMDS(ProducerTemplate producerTemplate,
>> > InputStream
>> > content, String MDSEndpoint, Map<String, Object> headers){
>> > BufferedReader br = new BufferedReader(new
>> > InputStreamReader(content));
>> > String line;
>> > try {
>> > while((line = br.readLine()) != null){
>> > if(!line.startsWith(START_OF_FILE_TAG)
>> &&
>> > !line.startsWith(END_OF_FILE_TAG)){
>> > StringBuilder stringBuilder =
>> null;
>> > if(line.substring(EEASTD_
>> > HEADER_START_POSITION,
>> > EEASTD_HEADER_END_POSITION).equals(EEASTD_HEADER)){
>> > stringBuilder = new
>> > StringBuilder();
>> >
>> stringBuilder.append("\n"+(
>> > line));
>> >
>> > }
>> > producerTemplate.sendBodyAndHeaders(
>> > MDSEndpoint,
>> > stringBuilder.toString(), headers);
>> > }
>> > }
>> > } catch (IOException e) {
>> > e.printStackTrace();
>> > }
>> > finally{
>> > try {
>> > if(br != null)br.close();
>> > } catch (IOException e) {
>> > e.printStackTrace();
>> > }
>> > }
>> > }
>> > }
>> >
>> > Have you an idea ?
>> >
>> > Thank you in advance.
>> >
>> > Natacha.
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context: http://camel.465427.n5.nabble.
>> > com/Cut-a-file-into-multiple-files-tp5787210.html
>> > Sent from the Camel - Users mailing list archive at Nabble.com.
>> >
>>
>
>
Re: Cut a file into multiple files
Posted by Brad Johnson <br...@mediadriver.com>.
I'd definitely look at Beanio and the Camel bundle/library for it. It's
relatively easy to create identifiers in that way and segregate those out
as beans. So every bean class might have a header (or not since it seems
static and throwaway) and then 0...M lines that could be mapped into the
bean. Then you end up with a bean for every set of records that you can
route and write out. You could then use a file endpoint in the route
instead of trying to write them out yourself. That also means that handlers
and validators down the line do not have to be Processors requiring the
handling of a raw exchange but can be beans themselves that just have a
method that takes your bean like as follows. Then you can also unit test
the individual handlers/validators or whatever you want to call them
independent of any Camel libraries or routes.
public class BeanHandler {
public void invoke(MyBeanModel model)
{
//some logic. You have even return the model or a different one.
}
}
On Mon, Sep 5, 2016 at 4:54 PM, Chris Odom <ch...@mediadriver.com>
wrote:
> I would try using the Camel EIP Splitter
>
> http://camel.apache.org/splitter.html
>
>
> *Chris Odom | Sr Consultant | Media Driver*
> 512.799.0270 | chris.odom@mediadriver.com
> 810 Hester's Crossing, Suite 165, Round Rock, TX 78681
>
> On Mon, Sep 5, 2016 at 10:18 AM, Louisa <Le...@hotmail.fr> wrote:
>
> > Hello evereybody,
> >
> > I use Java Camel to my project (Talend).
> > I need your help for a method to solve this problem. I have this file:
> >
> > D0000000000000000000
> > EEASTDE
> > ELASPDF
> > ELASQML
> > EEASTDE
> > ELASPDF
> > ELASQHG
> > F00000006
> >
> > How I can cut in java this way (in 2 file) ?
> >
> > EEASTDE
> > ELASPDF
> > ELASQML
> >
> > And:
> >
> > EEASTDE
> > ELASPDF
> > ELASQHG
> >
> >
> > I have to cut the file according to the head EEASTDE. It's an example, I
> > could have several times the header EEASTDE (3, 4, 5...).
> > I thought of the HashMap, the arraylist but I don't know how to make the
> > code because I'm not indicate the line EEASTDE as a unique key.
> >
> > Here is my test code but it doesn't work:
> >
> >
> > public class bean_test implements Processor{
> >
> >
> > private static final String MDS_ENDPOINT_NAME = "MDSEndpoint";
> > private static final String EEASTD_HEADER = "EEASTDE";
> > private static final int EEASTD_HEADER_START_POSITION = 0;
> > private static final int EEASTD_HEADER_END_POSITION = 7;
> > private static final String START_OF_FILE_TAG = "D";
> > private static final String END_OF_FILE_TAG = "F";
> >
> > @Override
> > public void process(Exchange exchange) throws Exception {
> >
> > ProducerTemplate producerTemplate =
> > exchange.getContext().createProducerTemplate();
> > String MDSEndpoint = exchange.getIn().getHeader(
> > MDS_ENDPOINT_NAME,
> > String.class);
> > InputStream is = new
> > ByteArrayInputStream(exchange.getIn().getBody(String.class).getBytes());
> > extractFromMDS(producerTemplate, is, MDSEndpoint, new
> > HashMap<String,
> > Object>(exchange.getIn().getHeaders()));
> >
> > }
> >
> > private void extractFromMDS(ProducerTemplate producerTemplate,
> > InputStream
> > content, String MDSEndpoint, Map<String, Object> headers){
> > BufferedReader br = new BufferedReader(new
> > InputStreamReader(content));
> > String line;
> > try {
> > while((line = br.readLine()) != null){
> > if(!line.startsWith(START_OF_FILE_TAG)
> &&
> > !line.startsWith(END_OF_FILE_TAG)){
> > StringBuilder stringBuilder =
> null;
> > if(line.substring(EEASTD_
> > HEADER_START_POSITION,
> > EEASTD_HEADER_END_POSITION).equals(EEASTD_HEADER)){
> > stringBuilder = new
> > StringBuilder();
> >
> stringBuilder.append("\n"+(
> > line));
> >
> > }
> > producerTemplate.sendBodyAndHeaders(
> > MDSEndpoint,
> > stringBuilder.toString(), headers);
> > }
> > }
> > } catch (IOException e) {
> > e.printStackTrace();
> > }
> > finally{
> > try {
> > if(br != null)br.close();
> > } catch (IOException e) {
> > e.printStackTrace();
> > }
> > }
> > }
> > }
> >
> > Have you an idea ?
> >
> > Thank you in advance.
> >
> > Natacha.
> >
> >
> >
> >
> > --
> > View this message in context: http://camel.465427.n5.nabble.
> > com/Cut-a-file-into-multiple-files-tp5787210.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.
> >
>
Re: Cut a file into multiple files
Posted by Chris Odom <ch...@mediadriver.com>.
I would try using the Camel EIP Splitter
http://camel.apache.org/splitter.html
*Chris Odom | Sr Consultant | Media Driver*
512.799.0270 | chris.odom@mediadriver.com
810 Hester's Crossing, Suite 165, Round Rock, TX 78681
On Mon, Sep 5, 2016 at 10:18 AM, Louisa <Le...@hotmail.fr> wrote:
> Hello evereybody,
>
> I use Java Camel to my project (Talend).
> I need your help for a method to solve this problem. I have this file:
>
> D0000000000000000000
> EEASTDE
> ELASPDF
> ELASQML
> EEASTDE
> ELASPDF
> ELASQHG
> F00000006
>
> How I can cut in java this way (in 2 file) ?
>
> EEASTDE
> ELASPDF
> ELASQML
>
> And:
>
> EEASTDE
> ELASPDF
> ELASQHG
>
>
> I have to cut the file according to the head EEASTDE. It's an example, I
> could have several times the header EEASTDE (3, 4, 5...).
> I thought of the HashMap, the arraylist but I don't know how to make the
> code because I'm not indicate the line EEASTDE as a unique key.
>
> Here is my test code but it doesn't work:
>
>
> public class bean_test implements Processor{
>
>
> private static final String MDS_ENDPOINT_NAME = "MDSEndpoint";
> private static final String EEASTD_HEADER = "EEASTDE";
> private static final int EEASTD_HEADER_START_POSITION = 0;
> private static final int EEASTD_HEADER_END_POSITION = 7;
> private static final String START_OF_FILE_TAG = "D";
> private static final String END_OF_FILE_TAG = "F";
>
> @Override
> public void process(Exchange exchange) throws Exception {
>
> ProducerTemplate producerTemplate =
> exchange.getContext().createProducerTemplate();
> String MDSEndpoint = exchange.getIn().getHeader(
> MDS_ENDPOINT_NAME,
> String.class);
> InputStream is = new
> ByteArrayInputStream(exchange.getIn().getBody(String.class).getBytes());
> extractFromMDS(producerTemplate, is, MDSEndpoint, new
> HashMap<String,
> Object>(exchange.getIn().getHeaders()));
>
> }
>
> private void extractFromMDS(ProducerTemplate producerTemplate,
> InputStream
> content, String MDSEndpoint, Map<String, Object> headers){
> BufferedReader br = new BufferedReader(new
> InputStreamReader(content));
> String line;
> try {
> while((line = br.readLine()) != null){
> if(!line.startsWith(START_OF_FILE_TAG) &&
> !line.startsWith(END_OF_FILE_TAG)){
> StringBuilder stringBuilder = null;
> if(line.substring(EEASTD_
> HEADER_START_POSITION,
> EEASTD_HEADER_END_POSITION).equals(EEASTD_HEADER)){
> stringBuilder = new
> StringBuilder();
> stringBuilder.append("\n"+(
> line));
>
> }
> producerTemplate.sendBodyAndHeaders(
> MDSEndpoint,
> stringBuilder.toString(), headers);
> }
> }
> } catch (IOException e) {
> e.printStackTrace();
> }
> finally{
> try {
> if(br != null)br.close();
> } catch (IOException e) {
> e.printStackTrace();
> }
> }
> }
> }
>
> Have you an idea ?
>
> Thank you in advance.
>
> Natacha.
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.
> com/Cut-a-file-into-multiple-files-tp5787210.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>