File component with more than one thread

["maria.iracheta@gmail.com" <ma...@gmail.com>]

Hello,

I'm trying to use the File component that comes with Camel 2.6.0 and I'm
having 2 different problems based on whether I'm using "markerFile" or
"fileLock" as a readLock strategy.

If I use the default (markerFile), there are files being left behind after
they have been processed. The files left undeleted have the extension
.camelLock.

The file endpoint I'm using is:

file://directoryName?move=moveDirectory&moveFailed=failedDirectory

To avoid that problem I tried changing the readLock strategy and added
"&readLock=fileLock" at the end of my file endpoint:

file://directoryName?move=moveDirectory&moveFailed=failedDirectory&readLock=fileLock

In this case the files are deleted as expected but there are exception being
thrown:

13:39:17.507 [Camel (camel-1) thread #2 - Threads] ERROR
o.a.c.c.file.GenericFileOnCompletion - Caused by:
[java.nio.channels.ClosedChannelException - null]
java.nio.channels.ClosedChannelException: null
    at sun.nio.ch.FileLockImpl.release(FileLockImpl.java:54) ~[na:1.6.0_20]
    at
org.apache.camel.component.file.strategy.FileLockExclusiveReadLockStrategy.releaseExclusiveReadLock(FileLockExclusiveReadLockStrategy.java:122)
~[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.commit(GenericFileProcessStrategySupport.java:58)
~[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:72)
~[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:142)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:84)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:53)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:60)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:166)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:140)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:109)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.processor.DefaultChannel$1.done(DefaultChannel.java:275)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.processor.RedeliveryErrorHandler$1.done(RedeliveryErrorHandler.java:311)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.management.InstrumentationProcessor$1.done(InstrumentationProcessor.java:78)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at org.apache.camel.processor.Pipeline$1.done(Pipeline.java:156)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
org.apache.camel.processor.ThreadsProcessor$ProcessCall.run(ThreadsProcessor.java:57)
[org.apache.camel.camel-core_2.6.0.jar:2.6.0]
    at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[na:1.6.0_20]
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[na:1.6.0_20]
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[na:1.6.0_20]
    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
[na:1.6.0_20]
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[na:1.6.0_20]
    at java.lang.Thread.run(Thread.java:636) [na:1.6.0_20]


My route looks like:

from(fileEndpoint).threads(numberOfConcurrentThx)
					.process(myProcessor);

If the NumberOfConcurrentThx is decreased to be 1 the problem with the
MarkerFileExclusiveReadLockStrategy seems to disappear.

Is there any extra configuration I should be aware of? Could it be a race
condition?

Thanks,

Maria Iracheta


--
View this message in context: http://camel.465427.n5.nabble.com/File-component-with-more-than-one-thread-tp3880492p3880492.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: File component with more than one thread

["maria.iracheta@gmail.com" <ma...@gmail.com>]

I think that
org.apache.camel.component.file.strategy.MarkerFileExclusiveReadLockStrategy
is not thread-safe. It is using the global variable lock and lockFileName to
decide which files to delete in the releaseExclusiveReadLock method. That
matches the behaviour I'm seeing when only the last file to be read is the
one that is deleted after.


--
View this message in context: http://camel.465427.n5.nabble.com/File-component-with-more-than-one-thread-tp3880492p3885874.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: File component with more than one thread

["maria.iracheta@gmail.com" <ma...@gmail.com>]

I forgot to mention I'm using Linux.

Thanks,

Maria Iracheta

--
View this message in context: http://camel.465427.n5.nabble.com/File-component-with-more-than-one-thread-tp3880492p3882888.html
Sent from the Camel - Users mailing list archive at Nabble.com.