You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Ephemeris Lappis <ep...@gmail.com> on 2022/11/02 14:21:49 UTC
Re: File component fails with charsets
Hello again.
Sorry to answer my own question...
I've read again the file component documentation looking for any
solution to fix ou issue, and I've seen that this is exactly the
example that it's given for using charset :
The charset option allows for configuring an encoding of the files on
both the consumer and producer endpoints. For example if you read
utf-8 files, and want to convert the files to iso-8859-1, you can do:
from("file:inbox?charset=utf-8")
.to("file:outbox?charset=iso-8859-1")
And this seems to be always failing with the
UnmappableCharacterException when a character from the input has no
mapped character in the output charset...
Nobody has reported it yet ?
Thanks for your feedback.
Regards.
Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
<ep...@gmail.com> a écrit :
>
> Hello.
>
> I've been using the File component for years, and discover today a
> strange exception when using a charset on a producer endpoint.
>
> Example :
>
> <route id="pma-file">
> <from uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> <log message="PMA : processing ${file:name} (length ${file:length})..."/>
> <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> </route>
>
> If I put a file that contains only characters that exist in
> ISO-8859-1, no problem.
> If the file contains an € (euro) for example, an exception is thrown
> (see at the end of the mail) :
> java.nio.charset.UnmappableCharacterException: Input length = 1
>
> A trivial java program, with the same jvm on the same machine, that
> reads a file in UTF-8 and writes its content into another file with
> ISO-8859-1 only replaces the missing unmapped characters with ?
>
> Is it a bug (I can't imagine nobody saw that before) or just a missing
> option on the endpoint ?
> I've not found any option for that...
>
> Thanks for your help.
>
> Regards.
>
> The exception :
> org.apache.camel.component.file.GenericFileOperationFailedException:
> Cannot store file: C:\TMP\PMA\output\F1.txt
> at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> ~[!/:2.25.4]
> at org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> ~[!/:2.25.4]
> at org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> ~[!/:2.25.4]
> at org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> ~[!/:2.25.4]
> at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> ~[!/:2.25.4]
> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> ~[!/:2.25.4]
> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> [!/:2.25.4]
> at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> [!/:2.25.4]
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> [!/:2.25.4]
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> [!/:2.25.4]
> at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> [!/:2.25.4]
> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> [!/:2.25.4]
> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> [!/:2.25.4]
> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> [!/:2.25.4]
> at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> [!/:2.25.4]
> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> [!/:2.25.4]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_242]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [?:1.8.0_242]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [?:1.8.0_242]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [?:1.8.0_242]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_242]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_242]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> Caused by: java.nio.charset.UnmappableCharacterException: Input length = 1
> at java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> ~[?:1.8.0_242]
> at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> ~[?:1.8.0_242]
> at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_242]
> at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> ~[?:1.8.0_242]
> at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> ~[?:1.8.0_242]
> at java.io.BufferedWriter.flush(BufferedWriter.java:253) ~[?:1.8.0_242]
> at org.apache.camel.util.IOHelper.copy(IOHelper.java:250) ~[!/:2.25.4]
> at org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> ~[!/:2.25.4]
> at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> ~[!/:2.25.4]
> ... 22 more
> Suppressed: java.nio.charset.UnmappableCharacterException:
> Input length = 1
> at
> java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> ~[?:1.8.0_242]
> at
> sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> ~[?:1.8.0_242]
> at
> sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_242]
> at
> java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> ~[?:1.8.0_242]
> at
> java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> ~[?:1.8.0_242]
> at
> java.io.BufferedWriter.close(BufferedWriter.java:265) ~[?:1.8.0_242]
> at
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> ~[!/:2.25.4]
> at
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> ~[!/:2.25.4]
> at
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> ~[!/:2.25.4]
> at
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> ~[!/:2.25.4]
> at
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> ~[!/:2.25.4]
> at
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> ~[!/:2.25.4]
> at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> ~[!/:2.25.4]
> at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> [!/:2.25.4]
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> [!/:2.25.4]
> at
> org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> [!/:2.25.4]
> at
> org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> [!/:2.25.4]
> at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> [!/:2.25.4]
> at
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> [!/:2.25.4]
> at
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> [!/:2.25.4]
> at
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> [!/:2.25.4]
> at
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> [!/:2.25.4]
> at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> [!/:2.25.4]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_242]
> at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [?:1.8.0_242]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [?:1.8.0_242]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [?:1.8.0_242]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_242]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_242]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> 1
Re: File component fails with charsets
Posted by Ephemeris Lappis <ep...@gmail.com>.
Hello again.
Perhaps you're right about RedHat Fuse. I can ask our customer to open a case...
For other environments where we use pure Karaf+Camel I'm afraid we
have to find the solution elsewhere...
I don't understand the origin of this issue. In my understanding
charsets and streams operations are not directly concerned by Camel,
but instead are provided by common Java classes. I'd like to know what
may be involved in the stack (Camel/Karaf/Felix/JDK) that could be the
cause of a different behavior for a simple charset conversion.
Can we be sure that Camel is really the direct cause ?
Regards.
Le jeu. 3 nov. 2022 à 13:12, Zheng Feng <zf...@redhat.com> a écrit :
>
> Well, since you are using Fuse 6.3 and I think it is supported by Red Hat.
> So it would be better to open a case to get more support from them.
>
> On Thu, Nov 3, 2022 at 7:49 PM Ephemeris Lappis <ep...@gmail.com>
> wrote:
>
> > Hello.
> >
> > I'm afraid we can't switch to a newer version. Our customer production
> > system is Fuse 6.3 with its validated Camel version.
> > And other systems are using the versions I've previously listed.
> >
> > For the future we can plan to use the latest Camel versions, but today
> > it's almost impossible...
> >
> > No idea of what changes lead to this very strange behavior on some
> > quite trivial operations ?
> >
> > Thanks again.
> >
> > Regards.
> >
> > Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
> > <an...@gmail.com> a écrit :
> > >
> > > Hi,
> > >
> > > I wouldn't know about the reason for this behavior, but my suggestion is
> > to
> > > try with a newer version of Camel. Camel version 2.x is not maintained
> > > anymore.
> > >
> > > Please, can you try with a newer version of Camel? Something like 3.18.3
> > > ... and let us know if this is still a problem?
> > >
> > > Kind regards
> > >
> > > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
> > ephemeris.lappis@gmail.com>
> > > wrote:
> > >
> > > > Hello.
> > > >
> > > > The issue happens on different versions of Camel with different
> > > > configurations :
> > > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > > > - 2.25.4 / karaf 4.4.1 / jdk 18
> > > > - 2.17.0 / fuse 6.3 / jdk 1.8
> > > >
> > > > The last one is the production environment where the issue is
> > critical...
> > > >
> > > > What's strange, is that the same little blueprint works as expected
> > > > without any error with :
> > > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> > > >
> > > > Any idea of the origin of the issue, and if a workaround exists : an
> > > > endpoint option, some configuration, anything ?
> > > >
> > > > Thanks.
> > > >
> > > > Regards.
> > > >
> > > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > > > <an...@gmail.com> a écrit :
> > > > >
> > > > > Hello,
> > > > >
> > > > > What version of Camel are you using?
> > > > >
> > > > > Kind regards
> > > > >
> > > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > > > ephemeris.lappis@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello again.
> > > > > >
> > > > > > Sorry to answer my own question...
> > > > > >
> > > > > > I've read again the file component documentation looking for any
> > > > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > > > example that it's given for using charset :
> > > > > >
> > > > > > The charset option allows for configuring an encoding of the files
> > on
> > > > > > both the consumer and producer endpoints. For example if you read
> > > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
> > do:
> > > > > >
> > > > > > from("file:inbox?charset=utf-8")
> > > > > > .to("file:outbox?charset=iso-8859-1")
> > > > > >
> > > > > > And this seems to be always failing with the
> > > > > > UnmappableCharacterException when a character from the input has no
> > > > > > mapped character in the output charset...
> > > > > >
> > > > > > Nobody has reported it yet ?
> > > > > >
> > > > > > Thanks for your feedback.
> > > > > >
> > > > > > Regards.
> > > > > >
> > > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > > > <ep...@gmail.com> a écrit :
> > > > > > >
> > > > > > > Hello.
> > > > > > >
> > > > > > > I've been using the File component for years, and discover today
> > a
> > > > > > > strange exception when using a charset on a producer endpoint.
> > > > > > >
> > > > > > > Example :
> > > > > > >
> > > > > > > <route id="pma-file">
> > > > > > > <from
> > > > > >
> > > >
> > uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > > > <log message="PMA : processing ${file:name} (length
> > > > ${file:length})..."/>
> > > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > > > </route>
> > > > > > >
> > > > > > > If I put a file that contains only characters that exist in
> > > > > > > ISO-8859-1, no problem.
> > > > > > > If the file contains an € (euro) for example, an exception is
> > thrown
> > > > > > > (see at the end of the mail) :
> > > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > > > >
> > > > > > > A trivial java program, with the same jvm on the same machine,
> > that
> > > > > > > reads a file in UTF-8 and writes its content into another file
> > with
> > > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > > > >
> > > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > > > missing
> > > > > > > option on the endpoint ?
> > > > > > > I've not found any option for that...
> > > > > > >
> > > > > > > Thanks for your help.
> > > > > > >
> > > > > > > Regards.
> > > > > > >
> > > > > > > The exception :
> > > > > > >
> > org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > > [?:1.8.0_242]
> > > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > > > length =
> > > > > > 1
> > > > > > > at
> > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > > > ~[?:1.8.0_242]
> > > > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > > ~[!/:2.25.4]
> > > > > > > ... 22 more
> > > > > > > Suppressed:
> > java.nio.charset.UnmappableCharacterException:
> > > > > > > Input length = 1
> > > > > > > at
> > > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
> > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > > [?:1.8.0_242]
> > > > > > > at java.lang.Thread.run(Thread.java:748)
> > > > [?:1.8.0_242]
> > > > > > > 1
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Otavio R. Piske
> > > > > http://orpiske.net
> > > >
> > >
> > >
> > > --
> > > Otavio R. Piske
> > > http://orpiske.net
> >
> >
Re: File component fails with charsets
Posted by Zheng Feng <zf...@redhat.com>.
Well, since you are using Fuse 6.3 and I think it is supported by Red Hat.
So it would be better to open a case to get more support from them.
On Thu, Nov 3, 2022 at 7:49 PM Ephemeris Lappis <ep...@gmail.com>
wrote:
> Hello.
>
> I'm afraid we can't switch to a newer version. Our customer production
> system is Fuse 6.3 with its validated Camel version.
> And other systems are using the versions I've previously listed.
>
> For the future we can plan to use the latest Camel versions, but today
> it's almost impossible...
>
> No idea of what changes lead to this very strange behavior on some
> quite trivial operations ?
>
> Thanks again.
>
> Regards.
>
> Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
> <an...@gmail.com> a écrit :
> >
> > Hi,
> >
> > I wouldn't know about the reason for this behavior, but my suggestion is
> to
> > try with a newer version of Camel. Camel version 2.x is not maintained
> > anymore.
> >
> > Please, can you try with a newer version of Camel? Something like 3.18.3
> > ... and let us know if this is still a problem?
> >
> > Kind regards
> >
> > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
> ephemeris.lappis@gmail.com>
> > wrote:
> >
> > > Hello.
> > >
> > > The issue happens on different versions of Camel with different
> > > configurations :
> > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > > - 2.25.4 / karaf 4.4.1 / jdk 18
> > > - 2.17.0 / fuse 6.3 / jdk 1.8
> > >
> > > The last one is the production environment where the issue is
> critical...
> > >
> > > What's strange, is that the same little blueprint works as expected
> > > without any error with :
> > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> > >
> > > Any idea of the origin of the issue, and if a workaround exists : an
> > > endpoint option, some configuration, anything ?
> > >
> > > Thanks.
> > >
> > > Regards.
> > >
> > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > > <an...@gmail.com> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > What version of Camel are you using?
> > > >
> > > > Kind regards
> > > >
> > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > > ephemeris.lappis@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello again.
> > > > >
> > > > > Sorry to answer my own question...
> > > > >
> > > > > I've read again the file component documentation looking for any
> > > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > > example that it's given for using charset :
> > > > >
> > > > > The charset option allows for configuring an encoding of the files
> on
> > > > > both the consumer and producer endpoints. For example if you read
> > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
> do:
> > > > >
> > > > > from("file:inbox?charset=utf-8")
> > > > > .to("file:outbox?charset=iso-8859-1")
> > > > >
> > > > > And this seems to be always failing with the
> > > > > UnmappableCharacterException when a character from the input has no
> > > > > mapped character in the output charset...
> > > > >
> > > > > Nobody has reported it yet ?
> > > > >
> > > > > Thanks for your feedback.
> > > > >
> > > > > Regards.
> > > > >
> > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > > <ep...@gmail.com> a écrit :
> > > > > >
> > > > > > Hello.
> > > > > >
> > > > > > I've been using the File component for years, and discover today
> a
> > > > > > strange exception when using a charset on a producer endpoint.
> > > > > >
> > > > > > Example :
> > > > > >
> > > > > > <route id="pma-file">
> > > > > > <from
> > > > >
> > >
> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > > <log message="PMA : processing ${file:name} (length
> > > ${file:length})..."/>
> > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > > </route>
> > > > > >
> > > > > > If I put a file that contains only characters that exist in
> > > > > > ISO-8859-1, no problem.
> > > > > > If the file contains an € (euro) for example, an exception is
> thrown
> > > > > > (see at the end of the mail) :
> > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > > >
> > > > > > A trivial java program, with the same jvm on the same machine,
> that
> > > > > > reads a file in UTF-8 and writes its content into another file
> with
> > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > > >
> > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > > missing
> > > > > > option on the endpoint ?
> > > > > > I've not found any option for that...
> > > > > >
> > > > > > Thanks for your help.
> > > > > >
> > > > > > Regards.
> > > > > >
> > > > > > The exception :
> > > > > >
> org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > > length =
> > > > > 1
> > > > > > at
> > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > > ~[?:1.8.0_242]
> > > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > ... 22 more
> > > > > > Suppressed:
> java.nio.charset.UnmappableCharacterException:
> > > > > > Input length = 1
> > > > > > at
> > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748)
> > > [?:1.8.0_242]
> > > > > > 1
> > > > >
> > > >
> > > >
> > > > --
> > > > Otavio R. Piske
> > > > http://orpiske.net
> > >
> >
> >
> > --
> > Otavio R. Piske
> > http://orpiske.net
>
>
Re: File component fails with charsets
Posted by Ephemeris Lappis <ep...@gmail.com>.
Hello.
I'm afraid we can't switch to a newer version. Our customer production
system is Fuse 6.3 with its validated Camel version.
And other systems are using the versions I've previously listed.
For the future we can plan to use the latest Camel versions, but today
it's almost impossible...
No idea of what changes lead to this very strange behavior on some
quite trivial operations ?
Thanks again.
Regards.
Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
<an...@gmail.com> a écrit :
>
> Hi,
>
> I wouldn't know about the reason for this behavior, but my suggestion is to
> try with a newer version of Camel. Camel version 2.x is not maintained
> anymore.
>
> Please, can you try with a newer version of Camel? Something like 3.18.3
> ... and let us know if this is still a problem?
>
> Kind regards
>
> On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <ep...@gmail.com>
> wrote:
>
> > Hello.
> >
> > The issue happens on different versions of Camel with different
> > configurations :
> > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > - 2.25.4 / karaf 4.4.1 / jdk 18
> > - 2.17.0 / fuse 6.3 / jdk 1.8
> >
> > The last one is the production environment where the issue is critical...
> >
> > What's strange, is that the same little blueprint works as expected
> > without any error with :
> > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> >
> > Any idea of the origin of the issue, and if a workaround exists : an
> > endpoint option, some configuration, anything ?
> >
> > Thanks.
> >
> > Regards.
> >
> > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > <an...@gmail.com> a écrit :
> > >
> > > Hello,
> > >
> > > What version of Camel are you using?
> > >
> > > Kind regards
> > >
> > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > ephemeris.lappis@gmail.com>
> > > wrote:
> > >
> > > > Hello again.
> > > >
> > > > Sorry to answer my own question...
> > > >
> > > > I've read again the file component documentation looking for any
> > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > example that it's given for using charset :
> > > >
> > > > The charset option allows for configuring an encoding of the files on
> > > > both the consumer and producer endpoints. For example if you read
> > > > utf-8 files, and want to convert the files to iso-8859-1, you can do:
> > > >
> > > > from("file:inbox?charset=utf-8")
> > > > .to("file:outbox?charset=iso-8859-1")
> > > >
> > > > And this seems to be always failing with the
> > > > UnmappableCharacterException when a character from the input has no
> > > > mapped character in the output charset...
> > > >
> > > > Nobody has reported it yet ?
> > > >
> > > > Thanks for your feedback.
> > > >
> > > > Regards.
> > > >
> > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > <ep...@gmail.com> a écrit :
> > > > >
> > > > > Hello.
> > > > >
> > > > > I've been using the File component for years, and discover today a
> > > > > strange exception when using a charset on a producer endpoint.
> > > > >
> > > > > Example :
> > > > >
> > > > > <route id="pma-file">
> > > > > <from
> > > >
> > uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > <log message="PMA : processing ${file:name} (length
> > ${file:length})..."/>
> > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > </route>
> > > > >
> > > > > If I put a file that contains only characters that exist in
> > > > > ISO-8859-1, no problem.
> > > > > If the file contains an € (euro) for example, an exception is thrown
> > > > > (see at the end of the mail) :
> > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > >
> > > > > A trivial java program, with the same jvm on the same machine, that
> > > > > reads a file in UTF-8 and writes its content into another file with
> > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > >
> > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > missing
> > > > > option on the endpoint ?
> > > > > I've not found any option for that...
> > > > >
> > > > > Thanks for your help.
> > > > >
> > > > > Regards.
> > > > >
> > > > > The exception :
> > > > > org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > at
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > [!/:2.25.4]
> > > > > at
> > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > [?:1.8.0_242]
> > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > length =
> > > > 1
> > > > > at
> > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > ~[?:1.8.0_242]
> > > > > at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > ~[?:1.8.0_242]
> > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > ~[?:1.8.0_242]
> > > > > at
> > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > ~[?:1.8.0_242]
> > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > ~[?:1.8.0_242]
> > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > ~[!/:2.25.4]
> > > > > ... 22 more
> > > > > Suppressed: java.nio.charset.UnmappableCharacterException:
> > > > > Input length = 1
> > > > > at
> > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_242]
> > > > > at
> > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > java.io.BufferedWriter.close(BufferedWriter.java:265) ~[?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > [!/:2.25.4]
> > > > > at
> > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > [?:1.8.0_242]
> > > > > at java.lang.Thread.run(Thread.java:748)
> > [?:1.8.0_242]
> > > > > 1
> > > >
> > >
> > >
> > > --
> > > Otavio R. Piske
> > > http://orpiske.net
> >
>
>
> --
> Otavio R. Piske
> http://orpiske.net
Re: File component fails with charsets
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
Thank you for checking this with Camel 3.x!
I created a ticket for it: https://issues.apache.org/jira/browse/CAMEL-18685
This behavior of the Charset is configurable and can be adjusted to report
(the default), ignore or replace the unmappable character.
Kind regards
On Thu, Nov 3, 2022 at 4:57 PM ski n <ra...@gmail.com> wrote:
> @Otavia I run the test and this also happens on Camel 3.19.0 (JDK11). The
> result:
>
> Caused by: java.nio.charset.UnmappableCharacterException: Input length = 1
>
> I am not expert on charsets, but when a character is not available /
> unmappable I would expect that it is replaced by for example a '?'.
> Obviously this
> does not happen. Not directly a Camel issue, but more a Java
> (java.nio.charset) issue.
>
> Some suggestions:
>
> 1. Replace the character to mappable character. Something like
>
> .transform(simple( "${body.replace('&', '&')}"))
>
> 2. Use convertBody. Something like this:
>
> <route id="charset_test">
> <from
> uri="file:./input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> <log message="PMA : processing ${file:name} (length
> ${file:length})..."/>
> <convertBodyTo charset="ISO-8859-1" type="java.lang.String"/>
> <to uri="file:./output"/>
> </route>
>
> 3. Write a processor
>
> Maybe avoiding "java.nio.charset" by using a processor:
>
> .process(exchange -> {
> String encodedWithUTF8 = exchange.getIn().getBody(String.class);
> String decodedToISO88591 = new
> String(encodedWithUTF8.getBytes("UTF-8"), "ISO-8859-1");
> exchange.getIn().setBody(decodedToISO88591);
> }
> Raymond
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Thu, Nov 3, 2022 at 4:00 PM Ephemeris Lappis <
> ephemeris.lappis@gmail.com>
> wrote:
>
> > Hello again.
> >
> > Well, as I said in my previous mail, I'm not sure that Camel is the
> > single component that's involved in our issues.
> > I've built a very little java project with a very simple route that
> > does the same operation as my blueprint.
> >
> > I've got strange results.
> >
> > When running with Camel 2.17.0 (the version that fails in our
> > production system), It works (yes, very strange) !
> > But when I try switching to 2.25.4, it fails again.
> >
> > In both cases I only use Camel and the logger over a Java 1.8 runtime.
> >
> > I attach the zipped project. Perhaps someone can test it too.
> >
> > This let me think that "something" elsewhere is involved that makes
> > similar routes work or fail, and, if I'm not mistaken, perhaps we can
> > control this "something"...
> >
> > Any idea from experts ???
> >
> > Thanks again.
> >
> > Regards.
> >
> >
> >
> >
> > Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
> > <an...@gmail.com> a écrit :
> > >
> > > Hi,
> > >
> > > I wouldn't know about the reason for this behavior, but my suggestion
> is
> > to
> > > try with a newer version of Camel. Camel version 2.x is not maintained
> > > anymore.
> > >
> > > Please, can you try with a newer version of Camel? Something like
> 3.18.3
> > > ... and let us know if this is still a problem?
> > >
> > > Kind regards
> > >
> > > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
> > ephemeris.lappis@gmail.com>
> > > wrote:
> > >
> > > > Hello.
> > > >
> > > > The issue happens on different versions of Camel with different
> > > > configurations :
> > > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > > > - 2.25.4 / karaf 4.4.1 / jdk 18
> > > > - 2.17.0 / fuse 6.3 / jdk 1.8
> > > >
> > > > The last one is the production environment where the issue is
> > critical...
> > > >
> > > > What's strange, is that the same little blueprint works as expected
> > > > without any error with :
> > > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> > > >
> > > > Any idea of the origin of the issue, and if a workaround exists : an
> > > > endpoint option, some configuration, anything ?
> > > >
> > > > Thanks.
> > > >
> > > > Regards.
> > > >
> > > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > > > <an...@gmail.com> a écrit :
> > > > >
> > > > > Hello,
> > > > >
> > > > > What version of Camel are you using?
> > > > >
> > > > > Kind regards
> > > > >
> > > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > > > ephemeris.lappis@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello again.
> > > > > >
> > > > > > Sorry to answer my own question...
> > > > > >
> > > > > > I've read again the file component documentation looking for any
> > > > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > > > example that it's given for using charset :
> > > > > >
> > > > > > The charset option allows for configuring an encoding of the
> files
> > on
> > > > > > both the consumer and producer endpoints. For example if you read
> > > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
> > do:
> > > > > >
> > > > > > from("file:inbox?charset=utf-8")
> > > > > > .to("file:outbox?charset=iso-8859-1")
> > > > > >
> > > > > > And this seems to be always failing with the
> > > > > > UnmappableCharacterException when a character from the input has
> no
> > > > > > mapped character in the output charset...
> > > > > >
> > > > > > Nobody has reported it yet ?
> > > > > >
> > > > > > Thanks for your feedback.
> > > > > >
> > > > > > Regards.
> > > > > >
> > > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > > > <ep...@gmail.com> a écrit :
> > > > > > >
> > > > > > > Hello.
> > > > > > >
> > > > > > > I've been using the File component for years, and discover
> today
> > a
> > > > > > > strange exception when using a charset on a producer endpoint.
> > > > > > >
> > > > > > > Example :
> > > > > > >
> > > > > > > <route id="pma-file">
> > > > > > > <from
> > > > > >
> > > >
> >
> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > > > <log message="PMA : processing ${file:name} (length
> > > > ${file:length})..."/>
> > > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > > > </route>
> > > > > > >
> > > > > > > If I put a file that contains only characters that exist in
> > > > > > > ISO-8859-1, no problem.
> > > > > > > If the file contains an € (euro) for example, an exception is
> > thrown
> > > > > > > (see at the end of the mail) :
> > > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > > > >
> > > > > > > A trivial java program, with the same jvm on the same machine,
> > that
> > > > > > > reads a file in UTF-8 and writes its content into another file
> > with
> > > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > > > >
> > > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > > > missing
> > > > > > > option on the endpoint ?
> > > > > > > I've not found any option for that...
> > > > > > >
> > > > > > > Thanks for your help.
> > > > > > >
> > > > > > > Regards.
> > > > > > >
> > > > > > > The exception :
> > > > > > >
> > org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > > [?:1.8.0_242]
> > > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > > > length =
> > > > > > 1
> > > > > > > at
> > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > > > ~[?:1.8.0_242]
> > > > > > > at
> org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > > ~[!/:2.25.4]
> > > > > > > ... 22 more
> > > > > > > Suppressed:
> > java.nio.charset.UnmappableCharacterException:
> > > > > > > Input length = 1
> > > > > > > at
> > > > > > >
> java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
> > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > > [?:1.8.0_242]
> > > > > > > at java.lang.Thread.run(Thread.java:748)
> > > > [?:1.8.0_242]
> > > > > > > 1
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Otavio R. Piske
> > > > > http://orpiske.net
> > > >
> > >
> > >
> > > --
> > > Otavio R. Piske
> > > http://orpiske.net
> >
>
--
Otavio R. Piske
http://orpiske.net
Re: File component fails with charsets
Posted by ski n <ra...@gmail.com>.
@Otavia I run the test and this also happens on Camel 3.19.0 (JDK11). The
result:
Caused by: java.nio.charset.UnmappableCharacterException: Input length = 1
I am not expert on charsets, but when a character is not available /
unmappable I would expect that it is replaced by for example a '?'.
Obviously this
does not happen. Not directly a Camel issue, but more a Java
(java.nio.charset) issue.
Some suggestions:
1. Replace the character to mappable character. Something like
.transform(simple( "${body.replace('&', '&')}"))
2. Use convertBody. Something like this:
<route id="charset_test">
<from
uri="file:./input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
<log message="PMA : processing ${file:name} (length
${file:length})..."/>
<convertBodyTo charset="ISO-8859-1" type="java.lang.String"/>
<to uri="file:./output"/>
</route>
3. Write a processor
Maybe avoiding "java.nio.charset" by using a processor:
.process(exchange -> {
String encodedWithUTF8 = exchange.getIn().getBody(String.class);
String decodedToISO88591 = new
String(encodedWithUTF8.getBytes("UTF-8"), "ISO-8859-1");
exchange.getIn().setBody(decodedToISO88591);
}
Raymond
On Thu, Nov 3, 2022 at 4:00 PM Ephemeris Lappis <ep...@gmail.com>
wrote:
> Hello again.
>
> Well, as I said in my previous mail, I'm not sure that Camel is the
> single component that's involved in our issues.
> I've built a very little java project with a very simple route that
> does the same operation as my blueprint.
>
> I've got strange results.
>
> When running with Camel 2.17.0 (the version that fails in our
> production system), It works (yes, very strange) !
> But when I try switching to 2.25.4, it fails again.
>
> In both cases I only use Camel and the logger over a Java 1.8 runtime.
>
> I attach the zipped project. Perhaps someone can test it too.
>
> This let me think that "something" elsewhere is involved that makes
> similar routes work or fail, and, if I'm not mistaken, perhaps we can
> control this "something"...
>
> Any idea from experts ???
>
> Thanks again.
>
> Regards.
>
>
>
>
> Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
> <an...@gmail.com> a écrit :
> >
> > Hi,
> >
> > I wouldn't know about the reason for this behavior, but my suggestion is
> to
> > try with a newer version of Camel. Camel version 2.x is not maintained
> > anymore.
> >
> > Please, can you try with a newer version of Camel? Something like 3.18.3
> > ... and let us know if this is still a problem?
> >
> > Kind regards
> >
> > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
> ephemeris.lappis@gmail.com>
> > wrote:
> >
> > > Hello.
> > >
> > > The issue happens on different versions of Camel with different
> > > configurations :
> > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > > - 2.25.4 / karaf 4.4.1 / jdk 18
> > > - 2.17.0 / fuse 6.3 / jdk 1.8
> > >
> > > The last one is the production environment where the issue is
> critical...
> > >
> > > What's strange, is that the same little blueprint works as expected
> > > without any error with :
> > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> > >
> > > Any idea of the origin of the issue, and if a workaround exists : an
> > > endpoint option, some configuration, anything ?
> > >
> > > Thanks.
> > >
> > > Regards.
> > >
> > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > > <an...@gmail.com> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > What version of Camel are you using?
> > > >
> > > > Kind regards
> > > >
> > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > > ephemeris.lappis@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello again.
> > > > >
> > > > > Sorry to answer my own question...
> > > > >
> > > > > I've read again the file component documentation looking for any
> > > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > > example that it's given for using charset :
> > > > >
> > > > > The charset option allows for configuring an encoding of the files
> on
> > > > > both the consumer and producer endpoints. For example if you read
> > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
> do:
> > > > >
> > > > > from("file:inbox?charset=utf-8")
> > > > > .to("file:outbox?charset=iso-8859-1")
> > > > >
> > > > > And this seems to be always failing with the
> > > > > UnmappableCharacterException when a character from the input has no
> > > > > mapped character in the output charset...
> > > > >
> > > > > Nobody has reported it yet ?
> > > > >
> > > > > Thanks for your feedback.
> > > > >
> > > > > Regards.
> > > > >
> > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > > <ep...@gmail.com> a écrit :
> > > > > >
> > > > > > Hello.
> > > > > >
> > > > > > I've been using the File component for years, and discover today
> a
> > > > > > strange exception when using a charset on a producer endpoint.
> > > > > >
> > > > > > Example :
> > > > > >
> > > > > > <route id="pma-file">
> > > > > > <from
> > > > >
> > >
> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > > <log message="PMA : processing ${file:name} (length
> > > ${file:length})..."/>
> > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > > </route>
> > > > > >
> > > > > > If I put a file that contains only characters that exist in
> > > > > > ISO-8859-1, no problem.
> > > > > > If the file contains an € (euro) for example, an exception is
> thrown
> > > > > > (see at the end of the mail) :
> > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > > >
> > > > > > A trivial java program, with the same jvm on the same machine,
> that
> > > > > > reads a file in UTF-8 and writes its content into another file
> with
> > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > > >
> > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > > missing
> > > > > > option on the endpoint ?
> > > > > > I've not found any option for that...
> > > > > >
> > > > > > Thanks for your help.
> > > > > >
> > > > > > Regards.
> > > > > >
> > > > > > The exception :
> > > > > >
> org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > > length =
> > > > > 1
> > > > > > at
> > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > > ~[?:1.8.0_242]
> > > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > ... 22 more
> > > > > > Suppressed:
> java.nio.charset.UnmappableCharacterException:
> > > > > > Input length = 1
> > > > > > at
> > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748)
> > > [?:1.8.0_242]
> > > > > > 1
> > > > >
> > > >
> > > >
> > > > --
> > > > Otavio R. Piske
> > > > http://orpiske.net
> > >
> >
> >
> > --
> > Otavio R. Piske
> > http://orpiske.net
>
Re: File component fails with charsets
Posted by Ephemeris Lappis <ep...@gmail.com>.
Hello again.
Thanks a lot for your feedback and the links to the commits. Indeed,
following your way, we've checked the source code from Red-Hat and the
camel-core 2.17.0.redhat-630516 we use has a different code, that is
not a backport of the Apache's commit, but leads to the same issue.
The code uses a FileOutputStream like the original code, but the
charset is set with a different way that probably produces the same
exception.
FileOutputStream os = new FileOutputStream(target, append);
Writer out = IOConverter.toWriter(os,
Charset.forName(charset).newEncoder());
This explains why the "original" 2.17.0 works on my machine and fails
on Fuse (that provides a divergent version).
And your explanations seem to confirm why the 2.25.4 also fails, but
for another reason.
It seems that the only way to manage the issue is to code a workaround
as Raymond said in another mail, to avoid processing "non ISO-8859-1"
characters on the file producer that must use an explicit charset.
Something like that (which is not a really pretty coding) seems to work :
from("file:./input?delete=true&charset=UTF-8&moveFailed=.BUG") //
.log("PMA : processing ${file:name} (length ${file:length}) : ${body}") //
.convertBodyTo(byte[].class, "ISO-8859-1") //
.convertBodyTo(String.class, "ISO-8859-1") //
.log("PMA : after conversion : ${body}") //
.to("file:./output?charset=ISO-8859-1");
In our case, the body should be rather small so as not to consume too
much memory. Not sure this could be a suitable solution for other
cases with bigger bodies...
Thanks a lot for your help !!!
I didn't think before that a trivial route with files and charsets
could be so hard to code :(
Regards.
Le jeu. 3 nov. 2022 à 17:07, Zheng Feng <zf...@redhat.com> a écrit :
>
> I just checked the commits history. And it seems related to this change [1]
> which switched to using Files.newBufferedWriter since camel-2.20.0. So your
> example is failing with camel > 2.20.0
> For Red Hat Fuse, it might backport this commit to their products and your
> customer should check it with them.
>
> [1]
> https://github.com/apache/camel/commit/d8c0a53b6787778661fb4cde3c59a50b5691efac#diff-47e39aad11a143d9d2c9d60998f6fd5c118118312a8efcaed65dcd0398487426
>
> On Thu, Nov 3, 2022 at 11:09 PM Ephemeris Lappis <ep...@gmail.com>
> wrote:
>
> > Hello again.
> >
> > Well, as I said in my previous mail, I'm not sure that Camel is the
> > single component that's involved in our issues.
> > I've built a very little java project with a very simple route that
> > does the same operation as my blueprint.
> >
> > I've got strange results.
> >
> > When running with Camel 2.17.0 (the version that fails in our
> > production system), It works (yes, very strange) !
> > But when I try switching to 2.25.4, it fails again.
> >
> > In both cases I only use Camel and the logger over a Java 1.8 runtime.
> >
> > I attach the zipped project. Perhaps someone can test it too.
> >
> > This let me think that "something" elsewhere is involved that makes
> > similar routes work or fail, and, if I'm not mistaken, perhaps we can
> > control this "something"...
> >
> > Any idea from experts ???
> >
> > Thanks again.
> >
> > Regards.
> >
> >
> >
> >
> > Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
> > <an...@gmail.com> a écrit :
> > >
> > > Hi,
> > >
> > > I wouldn't know about the reason for this behavior, but my suggestion is
> > to
> > > try with a newer version of Camel. Camel version 2.x is not maintained
> > > anymore.
> > >
> > > Please, can you try with a newer version of Camel? Something like 3.18.3
> > > ... and let us know if this is still a problem?
> > >
> > > Kind regards
> > >
> > > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
> > ephemeris.lappis@gmail.com>
> > > wrote:
> > >
> > > > Hello.
> > > >
> > > > The issue happens on different versions of Camel with different
> > > > configurations :
> > > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > > > - 2.25.4 / karaf 4.4.1 / jdk 18
> > > > - 2.17.0 / fuse 6.3 / jdk 1.8
> > > >
> > > > The last one is the production environment where the issue is
> > critical...
> > > >
> > > > What's strange, is that the same little blueprint works as expected
> > > > without any error with :
> > > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> > > >
> > > > Any idea of the origin of the issue, and if a workaround exists : an
> > > > endpoint option, some configuration, anything ?
> > > >
> > > > Thanks.
> > > >
> > > > Regards.
> > > >
> > > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > > > <an...@gmail.com> a écrit :
> > > > >
> > > > > Hello,
> > > > >
> > > > > What version of Camel are you using?
> > > > >
> > > > > Kind regards
> > > > >
> > > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > > > ephemeris.lappis@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello again.
> > > > > >
> > > > > > Sorry to answer my own question...
> > > > > >
> > > > > > I've read again the file component documentation looking for any
> > > > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > > > example that it's given for using charset :
> > > > > >
> > > > > > The charset option allows for configuring an encoding of the files
> > on
> > > > > > both the consumer and producer endpoints. For example if you read
> > > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
> > do:
> > > > > >
> > > > > > from("file:inbox?charset=utf-8")
> > > > > > .to("file:outbox?charset=iso-8859-1")
> > > > > >
> > > > > > And this seems to be always failing with the
> > > > > > UnmappableCharacterException when a character from the input has no
> > > > > > mapped character in the output charset...
> > > > > >
> > > > > > Nobody has reported it yet ?
> > > > > >
> > > > > > Thanks for your feedback.
> > > > > >
> > > > > > Regards.
> > > > > >
> > > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > > > <ep...@gmail.com> a écrit :
> > > > > > >
> > > > > > > Hello.
> > > > > > >
> > > > > > > I've been using the File component for years, and discover today
> > a
> > > > > > > strange exception when using a charset on a producer endpoint.
> > > > > > >
> > > > > > > Example :
> > > > > > >
> > > > > > > <route id="pma-file">
> > > > > > > <from
> > > > > >
> > > >
> > uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > > > <log message="PMA : processing ${file:name} (length
> > > > ${file:length})..."/>
> > > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > > > </route>
> > > > > > >
> > > > > > > If I put a file that contains only characters that exist in
> > > > > > > ISO-8859-1, no problem.
> > > > > > > If the file contains an € (euro) for example, an exception is
> > thrown
> > > > > > > (see at the end of the mail) :
> > > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > > > >
> > > > > > > A trivial java program, with the same jvm on the same machine,
> > that
> > > > > > > reads a file in UTF-8 and writes its content into another file
> > with
> > > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > > > >
> > > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > > > missing
> > > > > > > option on the endpoint ?
> > > > > > > I've not found any option for that...
> > > > > > >
> > > > > > > Thanks for your help.
> > > > > > >
> > > > > > > Regards.
> > > > > > >
> > > > > > > The exception :
> > > > > > >
> > org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > > [?:1.8.0_242]
> > > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > > > length =
> > > > > > 1
> > > > > > > at
> > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > > > ~[?:1.8.0_242]
> > > > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > > ~[!/:2.25.4]
> > > > > > > ... 22 more
> > > > > > > Suppressed:
> > java.nio.charset.UnmappableCharacterException:
> > > > > > > Input length = 1
> > > > > > > at
> > > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
> > ~[?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > > ~[!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > > [!/:2.25.4]
> > > > > > > at
> > > > > > >
> > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > > [?:1.8.0_242]
> > > > > > > at
> > > > > > >
> > > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > > [?:1.8.0_242]
> > > > > > > at java.lang.Thread.run(Thread.java:748)
> > > > [?:1.8.0_242]
> > > > > > > 1
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Otavio R. Piske
> > > > > http://orpiske.net
> > > >
> > >
> > >
> > > --
> > > Otavio R. Piske
> > > http://orpiske.net
> >
Re: File component fails with charsets
Posted by Zheng Feng <zf...@redhat.com>.
Hi,
For Red Hat Fuse, please check https://issues.redhat.com/browse/ENTESB-15441
and https://access.redhat.com/solutions/5649371. I think this could explain
this issue.
On Fri, Nov 4, 2022 at 12:07 AM Zheng Feng <zf...@redhat.com> wrote:
> I just checked the commits history. And it seems related to this change
> [1] which switched to using Files.newBufferedWriter since camel-2.20.0. So
> your example is failing with camel > 2.20.0
> For Red Hat Fuse, it might backport this commit to their products and your
> customer should check it with them.
>
> [1]
> https://github.com/apache/camel/commit/d8c0a53b6787778661fb4cde3c59a50b5691efac#diff-47e39aad11a143d9d2c9d60998f6fd5c118118312a8efcaed65dcd0398487426
>
> On Thu, Nov 3, 2022 at 11:09 PM Ephemeris Lappis <
> ephemeris.lappis@gmail.com> wrote:
>
>> Hello again.
>>
>> Well, as I said in my previous mail, I'm not sure that Camel is the
>> single component that's involved in our issues.
>> I've built a very little java project with a very simple route that
>> does the same operation as my blueprint.
>>
>> I've got strange results.
>>
>> When running with Camel 2.17.0 (the version that fails in our
>> production system), It works (yes, very strange) !
>> But when I try switching to 2.25.4, it fails again.
>>
>> In both cases I only use Camel and the logger over a Java 1.8 runtime.
>>
>> I attach the zipped project. Perhaps someone can test it too.
>>
>> This let me think that "something" elsewhere is involved that makes
>> similar routes work or fail, and, if I'm not mistaken, perhaps we can
>> control this "something"...
>>
>> Any idea from experts ???
>>
>> Thanks again.
>>
>> Regards.
>>
>>
>>
>>
>> Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
>> <an...@gmail.com> a écrit :
>> >
>> > Hi,
>> >
>> > I wouldn't know about the reason for this behavior, but my suggestion
>> is to
>> > try with a newer version of Camel. Camel version 2.x is not maintained
>> > anymore.
>> >
>> > Please, can you try with a newer version of Camel? Something like 3.18.3
>> > ... and let us know if this is still a problem?
>> >
>> > Kind regards
>> >
>> > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
>> ephemeris.lappis@gmail.com>
>> > wrote:
>> >
>> > > Hello.
>> > >
>> > > The issue happens on different versions of Camel with different
>> > > configurations :
>> > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
>> > > - 2.25.4 / karaf 4.4.1 / jdk 18
>> > > - 2.17.0 / fuse 6.3 / jdk 1.8
>> > >
>> > > The last one is the production environment where the issue is
>> critical...
>> > >
>> > > What's strange, is that the same little blueprint works as expected
>> > > without any error with :
>> > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
>> > >
>> > > Any idea of the origin of the issue, and if a workaround exists : an
>> > > endpoint option, some configuration, anything ?
>> > >
>> > > Thanks.
>> > >
>> > > Regards.
>> > >
>> > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
>> > > <an...@gmail.com> a écrit :
>> > > >
>> > > > Hello,
>> > > >
>> > > > What version of Camel are you using?
>> > > >
>> > > > Kind regards
>> > > >
>> > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
>> > > ephemeris.lappis@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hello again.
>> > > > >
>> > > > > Sorry to answer my own question...
>> > > > >
>> > > > > I've read again the file component documentation looking for any
>> > > > > solution to fix ou issue, and I've seen that this is exactly the
>> > > > > example that it's given for using charset :
>> > > > >
>> > > > > The charset option allows for configuring an encoding of the
>> files on
>> > > > > both the consumer and producer endpoints. For example if you read
>> > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
>> do:
>> > > > >
>> > > > > from("file:inbox?charset=utf-8")
>> > > > > .to("file:outbox?charset=iso-8859-1")
>> > > > >
>> > > > > And this seems to be always failing with the
>> > > > > UnmappableCharacterException when a character from the input has
>> no
>> > > > > mapped character in the output charset...
>> > > > >
>> > > > > Nobody has reported it yet ?
>> > > > >
>> > > > > Thanks for your feedback.
>> > > > >
>> > > > > Regards.
>> > > > >
>> > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
>> > > > > <ep...@gmail.com> a écrit :
>> > > > > >
>> > > > > > Hello.
>> > > > > >
>> > > > > > I've been using the File component for years, and discover
>> today a
>> > > > > > strange exception when using a charset on a producer endpoint.
>> > > > > >
>> > > > > > Example :
>> > > > > >
>> > > > > > <route id="pma-file">
>> > > > > > <from
>> > > > >
>> > >
>> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
>> > > > > > <log message="PMA : processing ${file:name} (length
>> > > ${file:length})..."/>
>> > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
>> > > > > > </route>
>> > > > > >
>> > > > > > If I put a file that contains only characters that exist in
>> > > > > > ISO-8859-1, no problem.
>> > > > > > If the file contains an € (euro) for example, an exception is
>> thrown
>> > > > > > (see at the end of the mail) :
>> > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
>> > > > > >
>> > > > > > A trivial java program, with the same jvm on the same machine,
>> that
>> > > > > > reads a file in UTF-8 and writes its content into another file
>> with
>> > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
>> > > > > >
>> > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
>> > > missing
>> > > > > > option on the endpoint ?
>> > > > > > I've not found any option for that...
>> > > > > >
>> > > > > > Thanks for your help.
>> > > > > >
>> > > > > > Regards.
>> > > > > >
>> > > > > > The exception :
>> > > > > >
>> org.apache.camel.component.file.GenericFileOperationFailedException:
>> > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > >
>> > >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > >
>> > >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > >
>> > >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > >
>> > >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> > > > > > [?:1.8.0_242]
>> > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
>> > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
>> > > length =
>> > > > > 1
>> > > > > > at
>> > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at
>> sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at
>> sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
>> > > > > ~[?:1.8.0_242]
>> > > > > > at
>> > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at
>> > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
>> > > > > ~[?:1.8.0_242]
>> > > > > > at
>> org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
>> > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > >
>> > >
>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
>> > > > > > ~[!/:2.25.4]
>> > > > > > ... 22 more
>> > > > > > Suppressed:
>> java.nio.charset.UnmappableCharacterException:
>> > > > > > Input length = 1
>> > > > > > at
>> > > > > >
>> java.nio.charset.CoderResult.throwException(CoderResult.java:282)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at
>> > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at
>> > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
>> ~[?:1.8.0_242]
>> > > > > > at
>> > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at
>> > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
>> > > > > > ~[?:1.8.0_242]
>> > > > > > at
>> > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
>> ~[?:1.8.0_242]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > >
>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
>> > > > > > ~[!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
>> > > > > > [!/:2.25.4]
>> > > > > > at
>> > > > > >
>> > >
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> > > > > > [?:1.8.0_242]
>> > > > > > at
>> > > > > >
>> > > > >
>> > >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> > > > > > [?:1.8.0_242]
>> > > > > > at java.lang.Thread.run(Thread.java:748)
>> > > [?:1.8.0_242]
>> > > > > > 1
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Otavio R. Piske
>> > > > http://orpiske.net
>> > >
>> >
>> >
>> > --
>> > Otavio R. Piske
>> > http://orpiske.net
>>
>
Re: File component fails with charsets
Posted by Zheng Feng <zf...@redhat.com>.
I just checked the commits history. And it seems related to this change [1]
which switched to using Files.newBufferedWriter since camel-2.20.0. So your
example is failing with camel > 2.20.0
For Red Hat Fuse, it might backport this commit to their products and your
customer should check it with them.
[1]
https://github.com/apache/camel/commit/d8c0a53b6787778661fb4cde3c59a50b5691efac#diff-47e39aad11a143d9d2c9d60998f6fd5c118118312a8efcaed65dcd0398487426
On Thu, Nov 3, 2022 at 11:09 PM Ephemeris Lappis <ep...@gmail.com>
wrote:
> Hello again.
>
> Well, as I said in my previous mail, I'm not sure that Camel is the
> single component that's involved in our issues.
> I've built a very little java project with a very simple route that
> does the same operation as my blueprint.
>
> I've got strange results.
>
> When running with Camel 2.17.0 (the version that fails in our
> production system), It works (yes, very strange) !
> But when I try switching to 2.25.4, it fails again.
>
> In both cases I only use Camel and the logger over a Java 1.8 runtime.
>
> I attach the zipped project. Perhaps someone can test it too.
>
> This let me think that "something" elsewhere is involved that makes
> similar routes work or fail, and, if I'm not mistaken, perhaps we can
> control this "something"...
>
> Any idea from experts ???
>
> Thanks again.
>
> Regards.
>
>
>
>
> Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
> <an...@gmail.com> a écrit :
> >
> > Hi,
> >
> > I wouldn't know about the reason for this behavior, but my suggestion is
> to
> > try with a newer version of Camel. Camel version 2.x is not maintained
> > anymore.
> >
> > Please, can you try with a newer version of Camel? Something like 3.18.3
> > ... and let us know if this is still a problem?
> >
> > Kind regards
> >
> > On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <
> ephemeris.lappis@gmail.com>
> > wrote:
> >
> > > Hello.
> > >
> > > The issue happens on different versions of Camel with different
> > > configurations :
> > > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > > - 2.25.4 / karaf 4.4.1 / jdk 18
> > > - 2.17.0 / fuse 6.3 / jdk 1.8
> > >
> > > The last one is the production environment where the issue is
> critical...
> > >
> > > What's strange, is that the same little blueprint works as expected
> > > without any error with :
> > > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> > >
> > > Any idea of the origin of the issue, and if a workaround exists : an
> > > endpoint option, some configuration, anything ?
> > >
> > > Thanks.
> > >
> > > Regards.
> > >
> > > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > > <an...@gmail.com> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > What version of Camel are you using?
> > > >
> > > > Kind regards
> > > >
> > > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > > ephemeris.lappis@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello again.
> > > > >
> > > > > Sorry to answer my own question...
> > > > >
> > > > > I've read again the file component documentation looking for any
> > > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > > example that it's given for using charset :
> > > > >
> > > > > The charset option allows for configuring an encoding of the files
> on
> > > > > both the consumer and producer endpoints. For example if you read
> > > > > utf-8 files, and want to convert the files to iso-8859-1, you can
> do:
> > > > >
> > > > > from("file:inbox?charset=utf-8")
> > > > > .to("file:outbox?charset=iso-8859-1")
> > > > >
> > > > > And this seems to be always failing with the
> > > > > UnmappableCharacterException when a character from the input has no
> > > > > mapped character in the output charset...
> > > > >
> > > > > Nobody has reported it yet ?
> > > > >
> > > > > Thanks for your feedback.
> > > > >
> > > > > Regards.
> > > > >
> > > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > > <ep...@gmail.com> a écrit :
> > > > > >
> > > > > > Hello.
> > > > > >
> > > > > > I've been using the File component for years, and discover today
> a
> > > > > > strange exception when using a charset on a producer endpoint.
> > > > > >
> > > > > > Example :
> > > > > >
> > > > > > <route id="pma-file">
> > > > > > <from
> > > > >
> > >
> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > > <log message="PMA : processing ${file:name} (length
> > > ${file:length})..."/>
> > > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > > </route>
> > > > > >
> > > > > > If I put a file that contains only characters that exist in
> > > > > > ISO-8859-1, no problem.
> > > > > > If the file contains an € (euro) for example, an exception is
> thrown
> > > > > > (see at the end of the mail) :
> > > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > > >
> > > > > > A trivial java program, with the same jvm on the same machine,
> that
> > > > > > reads a file in UTF-8 and writes its content into another file
> with
> > > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > > >
> > > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > > missing
> > > > > > option on the endpoint ?
> > > > > > I've not found any option for that...
> > > > > >
> > > > > > Thanks for your help.
> > > > > >
> > > > > > Regards.
> > > > > >
> > > > > > The exception :
> > > > > >
> org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > > length =
> > > > > 1
> > > > > > at
> > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > > ~[?:1.8.0_242]
> > > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > ... 22 more
> > > > > > Suppressed:
> java.nio.charset.UnmappableCharacterException:
> > > > > > Input length = 1
> > > > > > at
> > > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > > ~[?:1.8.0_242]
> > > > > > at
> > > > > > java.io.BufferedWriter.close(BufferedWriter.java:265)
> ~[?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > > ~[!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > > [!/:2.25.4]
> > > > > > at
> > > > > >
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > > [?:1.8.0_242]
> > > > > > at
> > > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > > [?:1.8.0_242]
> > > > > > at java.lang.Thread.run(Thread.java:748)
> > > [?:1.8.0_242]
> > > > > > 1
> > > > >
> > > >
> > > >
> > > > --
> > > > Otavio R. Piske
> > > > http://orpiske.net
> > >
> >
> >
> > --
> > Otavio R. Piske
> > http://orpiske.net
>
Re: File component fails with charsets
Posted by Ephemeris Lappis <ep...@gmail.com>.
Hello again.
Well, as I said in my previous mail, I'm not sure that Camel is the
single component that's involved in our issues.
I've built a very little java project with a very simple route that
does the same operation as my blueprint.
I've got strange results.
When running with Camel 2.17.0 (the version that fails in our
production system), It works (yes, very strange) !
But when I try switching to 2.25.4, it fails again.
In both cases I only use Camel and the logger over a Java 1.8 runtime.
I attach the zipped project. Perhaps someone can test it too.
This let me think that "something" elsewhere is involved that makes
similar routes work or fail, and, if I'm not mistaken, perhaps we can
control this "something"...
Any idea from experts ???
Thanks again.
Regards.
Le jeu. 3 nov. 2022 à 12:30, Otavio Rodolfo Piske
<an...@gmail.com> a écrit :
>
> Hi,
>
> I wouldn't know about the reason for this behavior, but my suggestion is to
> try with a newer version of Camel. Camel version 2.x is not maintained
> anymore.
>
> Please, can you try with a newer version of Camel? Something like 3.18.3
> ... and let us know if this is still a problem?
>
> Kind regards
>
> On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <ep...@gmail.com>
> wrote:
>
> > Hello.
> >
> > The issue happens on different versions of Camel with different
> > configurations :
> > - 2.25.4 / karaf 4.3.6 / jdk 1.8
> > - 2.25.4 / karaf 4.4.1 / jdk 18
> > - 2.17.0 / fuse 6.3 / jdk 1.8
> >
> > The last one is the production environment where the issue is critical...
> >
> > What's strange, is that the same little blueprint works as expected
> > without any error with :
> > - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
> >
> > Any idea of the origin of the issue, and if a workaround exists : an
> > endpoint option, some configuration, anything ?
> >
> > Thanks.
> >
> > Regards.
> >
> > Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> > <an...@gmail.com> a écrit :
> > >
> > > Hello,
> > >
> > > What version of Camel are you using?
> > >
> > > Kind regards
> > >
> > > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> > ephemeris.lappis@gmail.com>
> > > wrote:
> > >
> > > > Hello again.
> > > >
> > > > Sorry to answer my own question...
> > > >
> > > > I've read again the file component documentation looking for any
> > > > solution to fix ou issue, and I've seen that this is exactly the
> > > > example that it's given for using charset :
> > > >
> > > > The charset option allows for configuring an encoding of the files on
> > > > both the consumer and producer endpoints. For example if you read
> > > > utf-8 files, and want to convert the files to iso-8859-1, you can do:
> > > >
> > > > from("file:inbox?charset=utf-8")
> > > > .to("file:outbox?charset=iso-8859-1")
> > > >
> > > > And this seems to be always failing with the
> > > > UnmappableCharacterException when a character from the input has no
> > > > mapped character in the output charset...
> > > >
> > > > Nobody has reported it yet ?
> > > >
> > > > Thanks for your feedback.
> > > >
> > > > Regards.
> > > >
> > > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > > <ep...@gmail.com> a écrit :
> > > > >
> > > > > Hello.
> > > > >
> > > > > I've been using the File component for years, and discover today a
> > > > > strange exception when using a charset on a producer endpoint.
> > > > >
> > > > > Example :
> > > > >
> > > > > <route id="pma-file">
> > > > > <from
> > > >
> > uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > > <log message="PMA : processing ${file:name} (length
> > ${file:length})..."/>
> > > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > > </route>
> > > > >
> > > > > If I put a file that contains only characters that exist in
> > > > > ISO-8859-1, no problem.
> > > > > If the file contains an € (euro) for example, an exception is thrown
> > > > > (see at the end of the mail) :
> > > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > > >
> > > > > A trivial java program, with the same jvm on the same machine, that
> > > > > reads a file in UTF-8 and writes its content into another file with
> > > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > > >
> > > > > Is it a bug (I can't imagine nobody saw that before) or just a
> > missing
> > > > > option on the endpoint ?
> > > > > I've not found any option for that...
> > > > >
> > > > > Thanks for your help.
> > > > >
> > > > > Regards.
> > > > >
> > > > > The exception :
> > > > > org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > > at
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > [!/:2.25.4]
> > > > > at
> > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > [!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > [?:1.8.0_242]
> > > > > at
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > [?:1.8.0_242]
> > > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> > length =
> > > > 1
> > > > > at
> > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > ~[?:1.8.0_242]
> > > > > at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > ~[?:1.8.0_242]
> > > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > > ~[?:1.8.0_242]
> > > > > at
> > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > ~[?:1.8.0_242]
> > > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > > ~[?:1.8.0_242]
> > > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > ~[!/:2.25.4]
> > > > > ... 22 more
> > > > > Suppressed: java.nio.charset.UnmappableCharacterException:
> > > > > Input length = 1
> > > > > at
> > > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_242]
> > > > > at
> > > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > > ~[?:1.8.0_242]
> > > > > at
> > > > > java.io.BufferedWriter.close(BufferedWriter.java:265) ~[?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > > ~[!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > > [!/:2.25.4]
> > > > > at
> > > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > > [!/:2.25.4]
> > > > > at
> > > > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > [?:1.8.0_242]
> > > > > at
> > > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > [?:1.8.0_242]
> > > > > at java.lang.Thread.run(Thread.java:748)
> > [?:1.8.0_242]
> > > > > 1
> > > >
> > >
> > >
> > > --
> > > Otavio R. Piske
> > > http://orpiske.net
> >
>
>
> --
> Otavio R. Piske
> http://orpiske.net
Re: File component fails with charsets
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
Hi,
I wouldn't know about the reason for this behavior, but my suggestion is to
try with a newer version of Camel. Camel version 2.x is not maintained
anymore.
Please, can you try with a newer version of Camel? Something like 3.18.3
... and let us know if this is still a problem?
Kind regards
On Thu, Nov 3, 2022 at 9:30 AM Ephemeris Lappis <ep...@gmail.com>
wrote:
> Hello.
>
> The issue happens on different versions of Camel with different
> configurations :
> - 2.25.4 / karaf 4.3.6 / jdk 1.8
> - 2.25.4 / karaf 4.4.1 / jdk 18
> - 2.17.0 / fuse 6.3 / jdk 1.8
>
> The last one is the production environment where the issue is critical...
>
> What's strange, is that the same little blueprint works as expected
> without any error with :
> - 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
>
> Any idea of the origin of the issue, and if a workaround exists : an
> endpoint option, some configuration, anything ?
>
> Thanks.
>
> Regards.
>
> Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
> <an...@gmail.com> a écrit :
> >
> > Hello,
> >
> > What version of Camel are you using?
> >
> > Kind regards
> >
> > On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <
> ephemeris.lappis@gmail.com>
> > wrote:
> >
> > > Hello again.
> > >
> > > Sorry to answer my own question...
> > >
> > > I've read again the file component documentation looking for any
> > > solution to fix ou issue, and I've seen that this is exactly the
> > > example that it's given for using charset :
> > >
> > > The charset option allows for configuring an encoding of the files on
> > > both the consumer and producer endpoints. For example if you read
> > > utf-8 files, and want to convert the files to iso-8859-1, you can do:
> > >
> > > from("file:inbox?charset=utf-8")
> > > .to("file:outbox?charset=iso-8859-1")
> > >
> > > And this seems to be always failing with the
> > > UnmappableCharacterException when a character from the input has no
> > > mapped character in the output charset...
> > >
> > > Nobody has reported it yet ?
> > >
> > > Thanks for your feedback.
> > >
> > > Regards.
> > >
> > > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > > <ep...@gmail.com> a écrit :
> > > >
> > > > Hello.
> > > >
> > > > I've been using the File component for years, and discover today a
> > > > strange exception when using a charset on a producer endpoint.
> > > >
> > > > Example :
> > > >
> > > > <route id="pma-file">
> > > > <from
> > >
> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > > <log message="PMA : processing ${file:name} (length
> ${file:length})..."/>
> > > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > > </route>
> > > >
> > > > If I put a file that contains only characters that exist in
> > > > ISO-8859-1, no problem.
> > > > If the file contains an € (euro) for example, an exception is thrown
> > > > (see at the end of the mail) :
> > > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > > >
> > > > A trivial java program, with the same jvm on the same machine, that
> > > > reads a file in UTF-8 and writes its content into another file with
> > > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > > >
> > > > Is it a bug (I can't imagine nobody saw that before) or just a
> missing
> > > > option on the endpoint ?
> > > > I've not found any option for that...
> > > >
> > > > Thanks for your help.
> > > >
> > > > Regards.
> > > >
> > > > The exception :
> > > > org.apache.camel.component.file.GenericFileOperationFailedException:
> > > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > > at
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > [!/:2.25.4]
> > > > at
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > [!/:2.25.4]
> > > > at
> org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > [!/:2.25.4]
> > > > at
> org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > [!/:2.25.4]
> > > > at
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > [!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > [!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > [!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > [!/:2.25.4]
> > > > at
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > [!/:2.25.4]
> > > > at
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > [!/:2.25.4]
> > > > at
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > [?:1.8.0_242]
> > > > at
> > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > [?:1.8.0_242]
> > > > at
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > [?:1.8.0_242]
> > > > at
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > [?:1.8.0_242]
> > > > at
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > [?:1.8.0_242]
> > > > at
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > [?:1.8.0_242]
> > > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > > Caused by: java.nio.charset.UnmappableCharacterException: Input
> length =
> > > 1
> > > > at
> > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > ~[?:1.8.0_242]
> > > > at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > ~[?:1.8.0_242]
> > > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > > ~[?:1.8.0_242]
> > > > at
> java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > ~[?:1.8.0_242]
> > > > at
> java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > ~[?:1.8.0_242]
> > > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > > ~[?:1.8.0_242]
> > > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > > ~[!/:2.25.4]
> > > > at
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > ~[!/:2.25.4]
> > > > ... 22 more
> > > > Suppressed: java.nio.charset.UnmappableCharacterException:
> > > > Input length = 1
> > > > at
> > > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > > ~[?:1.8.0_242]
> > > > at
> > > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > > ~[?:1.8.0_242]
> > > > at
> > > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_242]
> > > > at
> > > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > > ~[?:1.8.0_242]
> > > > at
> > > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > > ~[?:1.8.0_242]
> > > > at
> > > > java.io.BufferedWriter.close(BufferedWriter.java:265) ~[?:1.8.0_242]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > > ~[!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > > ~[!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > > ~[!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > > ~[!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > > ~[!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > > ~[!/:2.25.4]
> > > > at
> > > >
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > > ~[!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > > [!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > [!/:2.25.4]
> > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > > [!/:2.25.4]
> > > > at
> > > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > > [!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > > [!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > > [!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > > [!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > > [!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > > [!/:2.25.4]
> > > > at
> > > >
> > >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > > [!/:2.25.4]
> > > > at
> > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > [?:1.8.0_242]
> > > > at
> > > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > > [?:1.8.0_242]
> > > > at
> > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > > [?:1.8.0_242]
> > > > at
> > > >
> > >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > > [?:1.8.0_242]
> > > > at
> > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > [?:1.8.0_242]
> > > > at
> > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > [?:1.8.0_242]
> > > > at java.lang.Thread.run(Thread.java:748)
> [?:1.8.0_242]
> > > > 1
> > >
> >
> >
> > --
> > Otavio R. Piske
> > http://orpiske.net
>
--
Otavio R. Piske
http://orpiske.net
Re: File component fails with charsets
Posted by Ephemeris Lappis <ep...@gmail.com>.
Hello.
The issue happens on different versions of Camel with different configurations :
- 2.25.4 / karaf 4.3.6 / jdk 1.8
- 2.25.4 / karaf 4.4.1 / jdk 18
- 2.17.0 / fuse 6.3 / jdk 1.8
The last one is the production environment where the issue is critical...
What's strange, is that the same little blueprint works as expected
without any error with :
- 2.16.5 / karaf 4.0.9 (smx 7.0.1) / jdk 1.8
Any idea of the origin of the issue, and if a workaround exists : an
endpoint option, some configuration, anything ?
Thanks.
Regards.
Le mer. 2 nov. 2022 à 18:51, Otavio Rodolfo Piske
<an...@gmail.com> a écrit :
>
> Hello,
>
> What version of Camel are you using?
>
> Kind regards
>
> On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <ep...@gmail.com>
> wrote:
>
> > Hello again.
> >
> > Sorry to answer my own question...
> >
> > I've read again the file component documentation looking for any
> > solution to fix ou issue, and I've seen that this is exactly the
> > example that it's given for using charset :
> >
> > The charset option allows for configuring an encoding of the files on
> > both the consumer and producer endpoints. For example if you read
> > utf-8 files, and want to convert the files to iso-8859-1, you can do:
> >
> > from("file:inbox?charset=utf-8")
> > .to("file:outbox?charset=iso-8859-1")
> >
> > And this seems to be always failing with the
> > UnmappableCharacterException when a character from the input has no
> > mapped character in the output charset...
> >
> > Nobody has reported it yet ?
> >
> > Thanks for your feedback.
> >
> > Regards.
> >
> > Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> > <ep...@gmail.com> a écrit :
> > >
> > > Hello.
> > >
> > > I've been using the File component for years, and discover today a
> > > strange exception when using a charset on a producer endpoint.
> > >
> > > Example :
> > >
> > > <route id="pma-file">
> > > <from
> > uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > > <log message="PMA : processing ${file:name} (length ${file:length})..."/>
> > > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > > </route>
> > >
> > > If I put a file that contains only characters that exist in
> > > ISO-8859-1, no problem.
> > > If the file contains an € (euro) for example, an exception is thrown
> > > (see at the end of the mail) :
> > > java.nio.charset.UnmappableCharacterException: Input length = 1
> > >
> > > A trivial java program, with the same jvm on the same machine, that
> > > reads a file in UTF-8 and writes its content into another file with
> > > ISO-8859-1 only replaces the missing unmapped characters with ?
> > >
> > > Is it a bug (I can't imagine nobody saw that before) or just a missing
> > > option on the endpoint ?
> > > I've not found any option for that...
> > >
> > > Thanks for your help.
> > >
> > > Regards.
> > >
> > > The exception :
> > > org.apache.camel.component.file.GenericFileOperationFailedException:
> > > Cannot store file: C:\TMP\PMA\output\F1.txt
> > > at
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > > ~[!/:2.25.4]
> > > at
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > ~[!/:2.25.4]
> > > at
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > ~[!/:2.25.4]
> > > at
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > ~[!/:2.25.4]
> > > at
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > ~[!/:2.25.4]
> > > at
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > ~[!/:2.25.4]
> > > at
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > [!/:2.25.4]
> > > at
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > [!/:2.25.4]
> > > at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > [!/:2.25.4]
> > > at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > [!/:2.25.4]
> > > at
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > [!/:2.25.4]
> > > at
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > [!/:2.25.4]
> > > at
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > [!/:2.25.4]
> > > at
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > [!/:2.25.4]
> > > at
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > [!/:2.25.4]
> > > at
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > [!/:2.25.4]
> > > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > [?:1.8.0_242]
> > > at
> > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > [?:1.8.0_242]
> > > at
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > [?:1.8.0_242]
> > > at
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > [?:1.8.0_242]
> > > at
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > [?:1.8.0_242]
> > > at
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > [?:1.8.0_242]
> > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > Caused by: java.nio.charset.UnmappableCharacterException: Input length =
> > 1
> > > at
> > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > ~[?:1.8.0_242]
> > > at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > ~[?:1.8.0_242]
> > > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> > ~[?:1.8.0_242]
> > > at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > ~[?:1.8.0_242]
> > > at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > ~[?:1.8.0_242]
> > > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> > ~[?:1.8.0_242]
> > > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> > ~[!/:2.25.4]
> > > at
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > > ~[!/:2.25.4]
> > > at
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > ~[!/:2.25.4]
> > > ... 22 more
> > > Suppressed: java.nio.charset.UnmappableCharacterException:
> > > Input length = 1
> > > at
> > > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > > ~[?:1.8.0_242]
> > > at
> > > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > > ~[?:1.8.0_242]
> > > at
> > > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_242]
> > > at
> > > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > > ~[?:1.8.0_242]
> > > at
> > > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > > ~[?:1.8.0_242]
> > > at
> > > java.io.BufferedWriter.close(BufferedWriter.java:265) ~[?:1.8.0_242]
> > > at
> > >
> > org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > > ~[!/:2.25.4]
> > > at
> > >
> > org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > > ~[!/:2.25.4]
> > > at
> > >
> > org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > > ~[!/:2.25.4]
> > > at
> > >
> > org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > > ~[!/:2.25.4]
> > > at
> > >
> > org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > > ~[!/:2.25.4]
> > > at
> > >
> > org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > > ~[!/:2.25.4]
> > > at
> > > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > > ~[!/:2.25.4]
> > > at
> > >
> > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > > [!/:2.25.4]
> > > at
> > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > [!/:2.25.4]
> > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > > [!/:2.25.4]
> > > at
> > > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > > [!/:2.25.4]
> > > at
> > >
> > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > > [!/:2.25.4]
> > > at
> > >
> > org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > > [!/:2.25.4]
> > > at
> > >
> > org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > > [!/:2.25.4]
> > > at
> > >
> > org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > > [!/:2.25.4]
> > > at
> > >
> > org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > > [!/:2.25.4]
> > > at
> > >
> > org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > > [!/:2.25.4]
> > > at
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > [?:1.8.0_242]
> > > at
> > > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > > [?:1.8.0_242]
> > > at
> > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > > [?:1.8.0_242]
> > > at
> > >
> > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > > [?:1.8.0_242]
> > > at
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > [?:1.8.0_242]
> > > at
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > [?:1.8.0_242]
> > > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > > 1
> >
>
>
> --
> Otavio R. Piske
> http://orpiske.net
Re: File component fails with charsets
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
Hello,
What version of Camel are you using?
Kind regards
On Wed, Nov 2, 2022 at 3:22 PM Ephemeris Lappis <ep...@gmail.com>
wrote:
> Hello again.
>
> Sorry to answer my own question...
>
> I've read again the file component documentation looking for any
> solution to fix ou issue, and I've seen that this is exactly the
> example that it's given for using charset :
>
> The charset option allows for configuring an encoding of the files on
> both the consumer and producer endpoints. For example if you read
> utf-8 files, and want to convert the files to iso-8859-1, you can do:
>
> from("file:inbox?charset=utf-8")
> .to("file:outbox?charset=iso-8859-1")
>
> And this seems to be always failing with the
> UnmappableCharacterException when a character from the input has no
> mapped character in the output charset...
>
> Nobody has reported it yet ?
>
> Thanks for your feedback.
>
> Regards.
>
> Le ven. 28 oct. 2022 à 16:59, Ephemeris Lappis
> <ep...@gmail.com> a écrit :
> >
> > Hello.
> >
> > I've been using the File component for years, and discover today a
> > strange exception when using a charset on a producer endpoint.
> >
> > Example :
> >
> > <route id="pma-file">
> > <from
> uri="file:/TMP/PMA/input?delete=true&charset=UTF-8&moveFailed=.BUG"/>
> > <log message="PMA : processing ${file:name} (length ${file:length})..."/>
> > <to uri="file:/TMP/PMA/output?charset=ISO-8859-1"/>
> > </route>
> >
> > If I put a file that contains only characters that exist in
> > ISO-8859-1, no problem.
> > If the file contains an € (euro) for example, an exception is thrown
> > (see at the end of the mail) :
> > java.nio.charset.UnmappableCharacterException: Input length = 1
> >
> > A trivial java program, with the same jvm on the same machine, that
> > reads a file in UTF-8 and writes its content into another file with
> > ISO-8859-1 only replaces the missing unmapped characters with ?
> >
> > Is it a bug (I can't imagine nobody saw that before) or just a missing
> > option on the endpoint ?
> > I've not found any option for that...
> >
> > Thanks for your help.
> >
> > Regards.
> >
> > The exception :
> > org.apache.camel.component.file.GenericFileOperationFailedException:
> > Cannot store file: C:\TMP\PMA\output\F1.txt
> > at
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:359)
> > ~[!/:2.25.4]
> > at
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > ~[!/:2.25.4]
> > at
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > ~[!/:2.25.4]
> > at
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > ~[!/:2.25.4]
> > at
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > ~[!/:2.25.4]
> > at
> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > ~[!/:2.25.4]
> > at
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > [!/:2.25.4]
> > at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > [!/:2.25.4]
> > at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > [!/:2.25.4]
> > at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > [!/:2.25.4]
> > at
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > [!/:2.25.4]
> > at
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > [!/:2.25.4]
> > at
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > [!/:2.25.4]
> > at
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > [!/:2.25.4]
> > at
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > [!/:2.25.4]
> > at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > [!/:2.25.4]
> > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > [?:1.8.0_242]
> > at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > [?:1.8.0_242]
> > at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > [?:1.8.0_242]
> > at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > [?:1.8.0_242]
> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > [?:1.8.0_242]
> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > [?:1.8.0_242]
> > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > Caused by: java.nio.charset.UnmappableCharacterException: Input length =
> 1
> > at
> java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > ~[?:1.8.0_242]
> > at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > ~[?:1.8.0_242]
> > at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
> ~[?:1.8.0_242]
> > at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > ~[?:1.8.0_242]
> > at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > ~[?:1.8.0_242]
> > at java.io.BufferedWriter.flush(BufferedWriter.java:253)
> ~[?:1.8.0_242]
> > at org.apache.camel.util.IOHelper.copy(IOHelper.java:250)
> ~[!/:2.25.4]
> > at
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:421)
> > ~[!/:2.25.4]
> > at
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > ~[!/:2.25.4]
> > ... 22 more
> > Suppressed: java.nio.charset.UnmappableCharacterException:
> > Input length = 1
> > at
> > java.nio.charset.CoderResult.throwException(CoderResult.java:282)
> > ~[?:1.8.0_242]
> > at
> > sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:285)
> > ~[?:1.8.0_242]
> > at
> > sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[?:1.8.0_242]
> > at
> > java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
> > ~[?:1.8.0_242]
> > at
> > java.io.BufferedWriter.flushBuffer(BufferedWriter.java:129)
> > ~[?:1.8.0_242]
> > at
> > java.io.BufferedWriter.close(BufferedWriter.java:265) ~[?:1.8.0_242]
> > at
> >
> org.apache.camel.component.file.FileOperations.writeFileByReaderWithCharset(FileOperations.java:422)
> > ~[!/:2.25.4]
> > at
> >
> org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:337)
> > ~[!/:2.25.4]
> > at
> >
> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:305)
> > ~[!/:2.25.4]
> > at
> >
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:169)
> > ~[!/:2.25.4]
> > at
> >
> org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:80)
> > ~[!/:2.25.4]
> > at
> >
> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
> > ~[!/:2.25.4]
> > at
> > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
> > ~[!/:2.25.4]
> > at
> >
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
> > [!/:2.25.4]
> > at
> >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > [!/:2.25.4]
> > at
> > org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
> > [!/:2.25.4]
> > at
> > org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
> > [!/:2.25.4]
> > at
> >
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
> > [!/:2.25.4]
> > at
> >
> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:452)
> > [!/:2.25.4]
> > at
> >
> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:221)
> > [!/:2.25.4]
> > at
> >
> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:185)
> > [!/:2.25.4]
> > at
> >
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
> > [!/:2.25.4]
> > at
> >
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
> > [!/:2.25.4]
> > at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > [?:1.8.0_242]
> > at
> > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> > [?:1.8.0_242]
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> > [?:1.8.0_242]
> > at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> > [?:1.8.0_242]
> > at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > [?:1.8.0_242]
> > at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > [?:1.8.0_242]
> > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]
> > 1
>
--
Otavio R. Piske
http://orpiske.net