[Trinidad] Large file upload and processing ideas

[Paul Mander <pa...@gmail.com>]

Related to Trinidad in that this is the main application/UI technology to be
used. We have a requirement to process a large amount of data and insert it
into a database. The data will be in a csv format and have millions of
entries.

We have 2 main issues here...

1. Getting the data into the app server to process
2. Getting the app server to process the data

Now, my feeling is that this large amount of data is too much for the app
server to handle efficiently and unless anyone has any thoughts I am
starting to think of a separate java process on the app server that wraps
sqlldr and parses a local stored csv (large size).

The trinidad ui's responsibility would simply to configure the run of sqlldr
and monitor its results/progress (I could create some sort of sockets
interface to communicate with the java wrapper on the server).

This sounds achievable but I'm still wondering how I get the large file onto
the server to begin with. Does anyone know of any utility (user friendly as
this app is targeted at business users) that can achieve this? Failing any
ideas I will be forced to define a samba share or ftp size to do the file
transfer...
-- 
View this message in context: http://old.nabble.com/-Trinidad--Large-file-upload-and-processing-ideas-tp29212729p29212729.html
Sent from the MyFaces - Users mailing list archive at Nabble.com.