You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Dinesh (Jira)" <ji...@apache.org> on 2020/09/13 00:23:00 UTC
[jira] [Commented] (CAMEL-15526) Camel-AWS2-S3: Consume Gzip file
from S3 not working.
[ https://issues.apache.org/jira/browse/CAMEL-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194911#comment-17194911 ]
Dinesh commented on CAMEL-15526:
--------------------------------
Updated Stack trace.
> Camel-AWS2-S3: Consume Gzip file from S3 not working.
> -------------------------------------------------------
>
> Key: CAMEL-15526
> URL: https://issues.apache.org/jira/browse/CAMEL-15526
> Project: Camel
> Issue Type: Bug
> Affects Versions: 3.4.2
> Environment: camel Aws2-s3
>
> Reporter: Dinesh
> Assignee: Andrea Cosentino
> Priority: Minor
>
> Hi,
> When consuming Gzip file from S3 using AWS2-S3 component, file not downlaoding properly;
> Same code working for AWS-S3, after upgrading to AWS2-S3, this code not working.
>
> Task:
> 1. Download "myfile.csv.gzip" file from S3.
> 2. GUnzip file and send to processor.
> 3. Processor will send file to http end point using multipart upload.
> Here is the code used.
>
> {{}}
> {{from("aws2-s3://mybucket?amazonS3Client=#amazonS3Client&deleteAfterRead=false&delay=5000&synchronous=true&includeBody=true&autocloseBody=false&prefix=myfile.csv.gz")}}
> {\{ .unmarshal().gzipDeflater()}}
> {\{ .process(@Override public void process(Exchange exchange) throws Exception {}}
> {{ MultipartEntityBuilder multipartEntityBuilder = MultipartEntityBuilder.create();
> String filename = exchange.getIn().getHeader(AWS2S3Constants.KEY, String.class);
> String body = exchange.getIn().getBody(String.class);
> ContentBody cd = new InputStreamBody(new ByteArrayInputStream(body.getBytes()), ContentType.MULTIPART_FORM_DATA, "temp.csv");
> multipartEntityBuilder.addPart("file", cd);
> System.out.println(cd);
> exchange.getOut().setBody(multipartEntityBuilder.build());)
> .to("direct:httproute");}}
> {{}}
> Stack Trace:
> {panel:title=Stack Trace}
> [main] WARN software.amazon.awssdk.profiles.internal.ProfileFileReader - Ignoring profile 'plugins' on line 6 because it did not start with 'profile ' and it was not 'default'.
> [main] INFO org.apache.camel.support.LRUCacheFactory - Detected and using LRUCacheFactory: camel-caffeine-lrucache
> [main] INFO org.apache.camel.component.http.HttpComponent - Created ClientConnectionManager org.apache.http.impl.conn.PoolingHttpClientConnectionManager@1b73be9f
> [main] INFO org.apache.camel.component.http.HttpComponent - Created ClientConnectionManager org.apache.http.impl.conn.PoolingHttpClientConnectionManager@35a9782c
> [main] INFO org.apache.camel.impl.engine.AbstractCamelContext - Apache Camel 3.4.2 (camel-1) is starting
> [main] INFO org.apache.camel.impl.engine.AbstractCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
> [main] INFO org.apache.camel.impl.engine.AbstractCamelContext - Using HealthCheck: camel-health
> [main] INFO org.apache.camel.impl.engine.InternalRouteStartupManager - Route: route1 started and consuming from: aws2-s3://mybucket
> [main] INFO org.apache.camel.impl.engine.InternalRouteStartupManager - Route: route2 started and consuming from: direct://httproute
> [main] INFO org.apache.camel.impl.engine.AbstractCamelContext - Total 3 routes, of which 3 are started
> [main] INFO org.apache.camel.impl.engine.AbstractCamelContext - Apache Camel 3.4.2 (camel-1) started in 1.496 seconds
> [Camel (camel-1) thread #1 - aws2-s3://mybucket] INFO consuming - Consumer Fired!
> [Camel (camel-1) thread #1 - aws2-s3://mybucket] INFO route1 - Replay Message Sent to file:s3out input_2020-07-01-07-20-31-796_0.csv.gz
> [Camel (camel-1) thread #1 - aws2-s3://mybucket] ERROR org.apache.camel.processor.errorhandler.DefaultErrorHandler - Failed delivery for (MessageId: ID-DGHANTA-Mac-1599955541230-0-1 on ExchangeId: ID-DGHANTA-Mac-1599955541230-0-1). Exhausted after delivery attempt: 1 caught: java.util.zip.ZipException: Not in GZIP format
> Message History (complete message history is disabled)
> ---------------------------------------------------------------------------------------------------------------------------------------
> RouteId ProcessorId Processor Elapsed (ms)
> [route1 ] [route1 ] [from[aws2-s3://mybucket?amazonS3Client=%23] [ 250]
> ...
> [route1 ] [unmarshal1 ] [unmarshal[org.apache.camel.model.dataformat.GzipDataFormat@d41f816] ] [ 0]
> Stacktrace
> ---------------------------------------------------------------------------------------------------------------------------------------
> java.util.zip.ZipException: Not in GZIP format
> at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)
> at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
> at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
> at org.apache.camel.dataformat.deflater.GzipDeflaterDataFormat.unmarshal(GzipDeflaterDataFormat.java:62)
> at org.apache.camel.support.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:64)
> at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:395)
> at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
> at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60)
> at org.apache.camel.processor.Pipeline.process(Pipeline.java:147)
> at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:286)
> at org.apache.camel.component.aws2.s3.AWS2S3Consumer.processBatch(AWS2S3Consumer.java:200)
> at org.apache.camel.component.aws2.s3.AWS2S3Consumer.poll(AWS2S3Consumer.java:113)
> at org.apache.camel.support.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187)
> at org.apache.camel.support.ScheduledPollConsumer.run(ScheduledPollConsumer.java:106)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {panel}
> Regards,
> Dinesh
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)