CAMEL SFTP WITH MULTIPLE CONSUMERS

[NikheelRanjan <ni...@gmail.com>]

Problem  statement:
1.Multiple text files(assume  10k)  of  varying  size from  100 to  800  kb
come  through an external file upload  program to one specific directory .
And we have to  read and and  save  the data  in a  DB from those  text 
files.Now  the requirement is:

 a.)Process the  files  ASAP.That means  if due  to certain maintenance
reasons the consumer was  down  for 24  hours  and  the  time  it came up
and saw some 10k  files  as backlogged  items,it  should  consume  them 
ASAP  with  multiple threads.But  FTP/SFTP  is  not  supporting  multiple 
threads  for one  endpoint/directory.
We  are using camel-2.15.*  with red hat  fuse 6.2.*

Proposed  workaround for  faster processing:
Use  a  separate persistent  Queue  and  read  the directory  through a
single thread  and pass  the text message to the Queue  and  keep  one route
with concurrent consumers  from the jms endpoint.

Looking for  different  ideas.
Thanks.



--
View this message in context: http://camel.465427.n5.nabble.com/CAMEL-SFTP-WITH-MULTIPLE-CONSUMERS-tp5803235.html
Sent from the Camel - Users mailing list archive at Nabble.com.