You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by GaryLeeMills <ga...@ge.com> on 2017/01/19 19:18:31 UTC
pollEnrich and variable file names - time stamp
Hello, thanks Camel lovers !! greatly appreciated , I need help, can’t
figure it out.
I am in need of help, I am trying to use the pollEnrich EIP, of which It
works, in the sense that no dynamic building takes place.
The situation is that I have files coming in from email, and sftp, that have
a time stamp at the end of the file name, to maintain uniqueness.
Example:
CMAS_LPD22_MPDE1A_PG1_2016-12-13_090023.csv
I need to be able to use wild cards or something to be able to retrieve the
file into the pollEnrich route.
This is what I’ve tried.
<endpoint id="navSeaJoinRoute"
uri="{{fileJoinEndpoint}}?fileName=CMAS_LPD22_MPDE1A_PG2*.csv" />
<route
id="navsea.core.fileJoinRoute"
autoStartup="true">
<from uri="{{fileJoinEndpoint}}" />
<choice>
<when>
<simple>${header.CamelFileName} contains
"CMAS_LPD22_MPDE1A_PG1" </simple>
<log message="${file:onlyname.noext} ready for
pollEnrichment" />
<jboss.fuse.bom.version>6.2.0.redhat-133</jboss.fuse.bom.version>
<camel-version>2.15.1.redhat-620133</camel-version>
--
View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: pollEnrich and variable file names - time stamp
Posted by GaryLeeMills <ga...@ge.com>.
thanks Clause, I will try this and get back with you. I appreciate your
response.
I do actually have an aggregator : and this works fine. it is the ability to
accept files on the other path to the pollEnrich EIP. where it says
fileName=CMAS... I need this to be dynamic due to a timestamp at the end of
the file.
<pollEnrich strategyRef="navSeaAggregationStrategy"
uri="{{fileJoinEndpoint}}?fileName=CMAS_LPD22_MPDE1A_PG2.csv&noop=true"/>
public class NavSeaAggregationStrategy implements AggregationStrategy {
private String record1 = "";
private String record2 = "";
private String record = "";
private Logger log =
LoggerFactory.getLogger(NavSeaAggregationStrategy.class.getName());
@Override
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
//Theory
//-------------------------------------------------------------------------------------
// Arrived | oldExchange | newExchange | Description
//-------------------------------------------------------------------------------------
// A | NULL | A | first message arrives for the first
group
// B | A | B | second message arrives for the first group
// F | NULL | F | first message arrives for the second group
// C | AB | C | third message arrives for the first group
//---------------------------------------------------------------------------------------
log.info("NAV SEA Aggregation Strategy :: Start");
if ( newExchange == null ) {
return oldExchange;
}
String newBody = newExchange.getIn().getBody(String.class); //newExchange
is the enriched source pollEnrich
String oldBody = oldExchange.getIn().getBody(String.class); //oldExchange
is the original message
StringBuilder sb1 = new StringBuilder(oldBody); //original msg
log.info("original or first msg: " + sb1.toString());
StringBuilder sb2 = new StringBuilder(newBody); //emrich msg path msg
log.info("pollEnrich route msg: " + sb2.toString());
List<String> serviceRecords = new ArrayList<String>();
//both bodies have the same number of rows
Scanner sc1 = new Scanner(sb1.toString()).useDelimiter("\\n"); //orig msg
1st incoming
Scanner sc2 = new Scanner(sb2.toString()).useDelimiter("\\n"); //msg to
enrich original with
while ( sc1.hasNext() ) {
record1 = sc1.next();
log.info("record 1: " + record1);
if ( sc2.hasNext() ) {
record2 = sc2.next();
log.info("record 2:" + record2);
String rc2 = record2.substring(record2.indexOf(',', 0) ); // get past
Time Stamp
record2 = rc2;
log.info("record 2 after removed time stamp - rc2: " + record2);
}
record = record1.trim() + record2.trim() + System.lineSeparator();
log.info("combined record: " + record);
serviceRecords.add(record);
}
StringBuilder sbout = new StringBuilder();
for ( Object o : serviceRecords) {
sbout.append(o.toString());
}
oldExchange.getIn().setBody( sbout.toString() );
log.info("NAV SEA Aggregation Strategy :: Finish");
return oldExchange;
} //Exchange process
} //class AggregationStrategy
--
View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827p5792880.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: pollEnrich and variable file names - time stamp
Posted by GaryLeeMills <ga...@ge.com>.
thanks for a reply, but no, I shouldn't be asking this question on a JBoss
Fuse site. Because this is Camel Spring framework. I am using Camel Spring.
The EIP is the pollEnrich. this is the problem. it doesn't ( at least I
can't figure it out ) allow for dynamic fileName as you can see from my
post. this is about the pollEnrich EIP camel Spring functionality - not
Jboss fuse.
--
View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827p5792879.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: pollEnrich and variable file names - time stamp
Posted by souciance <so...@gmail.com>.
You seem to be using JBoss Fuse. Shouldn't you be asking this on the JBoss
Fuse form?
Still, it is a bit hard to know what your requirement is. Do you want to
poll files that begin with some string? You can use the file language and
set the filename dynamically like:
CMAS_LPD22_MPDE1A_PG2*-${date:now:yyyyMMdd}.csv*
On Fri, Jan 20, 2017 at 9:31 PM, GaryLeeMills [via Camel] <
ml-node+s465427n5792872h95@n5.nabble.com> wrote:
> is this the right place for these questions? if not please send me to
> the correct place. thank you
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://camel.465427.n5.nabble.com/pollEnrich-and-variable-
> file-names-time-stamp-tp5792827p5792872.html
> To start a new topic under Camel - Users, email
> ml-node+s465427n465428h31@n5.nabble.com
> To unsubscribe from Camel - Users, click here
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=c291Y2lhbmNlLmVxZGFtLnJhc2h0aUBnbWFpbC5jb218NDY1NDI4fDE1MzI5MTE2NTY=>
> .
> NAML
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827p5792873.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: pollEnrich and variable file names - time stamp
Posted by GaryLeeMills <ga...@ge.com>.
Clause, thank you so much, I'm working somewhat. Can I ask you if there are
any known issues with the pollEnrich API in that it does not release the
file locks ? it is always on the pollenrichment path.
<pollEnrich strategyRef="navSeaAggregationStrategy"
uri="{{CMAS_LPD22_MPDE1A}}?fileName=CMAS_LPD22_MPDE1A_PG2.csv&delay=2000"
timeout="60000" />
It leaves a messy file system, and then any additional events do not work
until the locks are removed. ???? the PG2 is always on the pollEnrich path
as the appended file or data.
└── tobeprocessed
│ ├── LPD221A
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE1A_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE1A_PG2.csv
│ │ ├── CMAS_LPD22_MPDE1A_PG2.csv
│ │ └── CMAS_LPD22_MPDE1A_PG2.csv.camelLock
│ ├── LPD221B
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE1B_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE1B_PG2.csv
│ │ ├── CMAS_LPD22_MPDE1B_PG2.csv
│ │ └── CMAS_LPD22_MPDE1B_PG2.csv.camelLock
│ ├── LPD222A
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE2A_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE2A_PG2.csv
│ │ ├── CMAS_LPD22_MPDE2A_PG2.csv
│ │ └── CMAS_LPD22_MPDE2A_PG2.csv.camelLock
│ ├── LPD222B
│ │ ├── .camel
│ │ │ ├── CMAS_LPD22_MPDE2B_PG1.csv
│ │ │ └── CMAS_LPD22_MPDE2B_PG2.csv
│ │ ├── CMAS_LPD22_MPDE2B_PG2.csv
│ │ └── CMAS_LPD22_MPDE2B_PG2.csv.camelLock
--
View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827p5793037.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: pollEnrich and variable file names - time stamp
Posted by Claus Ibsen <cl...@gmail.com>.
See this page
http://camel.apache.org/content-enricher.html
Notice what those boxes says about dynamic values in poll enrich and
which Camel version.
If you just want to pickup a file, then you can just do that use
<transform> and <simple> and build the file name dynamically, and set
the resultType of simple to be java.io.File.
Or write a few lines of java code to read the file and aggregate /
merge whatever you want.
On Fri, Jan 20, 2017 at 9:31 PM, GaryLeeMills <ga...@ge.com> wrote:
> is this the right place for these questions? if not please send me to the
> correct place. thank you
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827p5792872.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: pollEnrich and variable file names - time stamp
Posted by GaryLeeMills <ga...@ge.com>.
is this the right place for these questions? if not please send me to the
correct place. thank you
--
View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827p5792872.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: pollEnrich and variable file names - time stamp
Posted by GaryLeeMills <ga...@ge.com>.
<route
id="navsea.core.fileJoinRoute"
autoStartup="true">
<from uri="{{fileJoinEndpoint}}" />
<choice>
<when>
<simple>${header.CamelFileName} contains "CMAS_LPD22_MPDE1A_PG1"
</simple>
<log message="${file:onlyname.noext} ready for pollEnrichment" />
works <pollEnrich strategyRef="navSeaAggregationStrategy"
uri="file:ge-ip/core/tobeprocessed?fileName=CMAS_LPD22_MPDE1A_PG2_2016-12-13_090045.csv"/>
works <pollEnrich strategyRef="navSeaAggregationStrategy"
uri="{{fileJoinEndpoint}}?fileName=CMAS_LPD22_MPDE1A_PG2_2016-12-13_090045.csv"/>
no work <pollEnrich strategyRef="navSeaAggregationStrategy"
uri="{{fileJoinEndpoint}}?fileName=CMAS_LPD22_MPDE1A_PG2_.*.csv"/> -->
no work <pollEnrich strategyRef="navSeaAggregationStrategy"
uri="{{fileJoinEndpoint}}?fileName=CMAS_LPD22_MPDE1A_PG2_${date:file:yyyyMMdd}.csv"/>
no work <pollEnrich strategyRef="navSeaAggregationStrategy"
uri="ref:navSeaJoinRoute"/> -->
<pollEnrich strategyRef="navSeaAggregationStrategy"
uri="{{fileJoinEndpoint}}?fileName=CMAS_LPD22_MPDE1A_PG2_2016-12-13_090045.csv"/>
<setHeader headerName="messageDateTime">
<simple>${date:now:MM-dd-yyyy-HH:mm:ss}</simple>
</setHeader>
<setHeader headerName="CamelFileName" >
<simple>CMAS_LPD22_MPDE1A_${header.messageDateTime}.csv</simple>
</setHeader>
<to uri="{{OutputDirectory}}"/>
</when>
</choice>
--
View this message in context: http://camel.465427.n5.nabble.com/pollEnrich-and-variable-file-names-time-stamp-tp5792827p5792831.html
Sent from the Camel - Users mailing list archive at Nabble.com.