You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2021/03/05 09:01:32 UTC
[GitHub] [camel-quarkus] tidemyr opened a new issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
tidemyr opened a new issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318
**Description:**
We intend to use Quarkus Camel for some integration patterns. One of these needs is to be able to upload files to Azure
from different types of incoming endpoints. In this specific microservice by moving files from an SFTP-server to Azure.
The code is limited to include only Quarkus supported methods for native builds and of course runs perfectly when using normal Java. But as native build there are runtime errors during the Azure upload, the SFPT and the mail-relay sending is working fine in both native and normal Java runtime.
**Expected behavior**
This limited code working exactly the same way running as native build as with normal Java.
Can be limited to just the Azure upload functionality to recreate the problem.
**Actual behavior**
Got runtime error (see additional context)
**Configuration**
Java 11.0.8.j9-adpt (as for normal Java-runtime)
GraalVM 20.2.0
Quarkus 1.11.1.Final
Camel Quarkus: 1.6.0
Maven 3.6.3
**Additional context**
Camel DSL code:
``
@Override
public void configure() throws Exception {
StorageSharedKeyCredential credential = new StorageSharedKeyCredential(azureAccountName, azureAccessKey);
String uri = String.format("https://%s.blob.core.windows.net", azureAccountName);
BlobServiceClient client = new BlobServiceClientBuilder()
.endpoint(uri)
.credential(credential)
.buildClient();
getContext().getRegistry().bind("client", client);
from(String.format("sftp:%s/%s?binary=true&username=%s&password=%s&%s",
sftpServer,
sftpPath,
sftpUsername,
sftpPassword,
sftpConfigOptions
))
.routeId("sftpToAzure")
.errorHandler(deadLetterChannel("seda:error"))
.convertBodyTo(byte[].class)
.log(LoggingLevel.INFO, "File ${file:name} (${file:size} bytes) is fetched from SFTP.")
.process(setHeaders)
.to(String.format("azure-storage-blob://%s/%s?blobName=blob&operation=uploadBlockBlob&serviceClient=#client",
azureAccountName,
azureContainer
))
.log(LoggingLevel.INFO, "Uploaded to Azure.");
from("seda:error")
.routeId("mailError")
.log(LoggingLevel.ERROR, exceptionMessage().toString())
.doTry()
.setHeader("from", simple(mailFrom))
.setHeader("to", simple(mailTo))
.setHeader("subject", simple("ERROR - " + appName + " in " + appProfile.toUpperCase()))
.setBody(exceptionMessage())
.to(String.format("smtp://%s:%s?username=%s&password=%s",
mailServer,
mailPort,
mailUsername,
mailPassword
));
}
pom.xml extract:
...
<properties>
<compiler-plugin.version>3.8.1</compiler-plugin.version>
<maven.compiler.parameters>true</maven.compiler.parameters>
<maven.compiler.source>11</maven.compiler.source>
<maven.compiler.target>11</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<quarkus-plugin.version>1.11.1.Final</quarkus-plugin.version>
<quarkus.platform.artifact-id>quarkus-universe-bom</quarkus.platform.artifact-id>
<quarkus.platform.group-id>io.quarkus</quarkus.platform.group-id>
<quarkus.platform.version>1.11.1.Final</quarkus.platform.version>
<camel-quarkus.version>1.6.0</camel-quarkus.version>
<surefire-plugin.version>2.22.1</surefire-plugin.version>
<docker-maven-plugin.version>0.33.0</docker-maven-plugin.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-bom</artifactId>
<version>${camel-quarkus.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-arc</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-junit5</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>rest-assured</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-openapi</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-core</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-ftp</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-azure-storage-blob</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-seda</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-mail</artifactId>
</dependency>
<dependency>
<groupId>org.apache.camel.quarkus</groupId>
<artifactId>camel-quarkus-log</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-config-yaml</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-health</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-micrometer</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-opentracing</artifactId>
</dependency>
<dependency>
<groupId>se.vt.integration.quarkus</groupId>
<artifactId>vt-jasypt-quarkus</artifactId>
<version>2.0.2</version>
<scope>provided</scope>
</dependency>
</dependencies>
...
Runtime error as native build:
2021-03-05 09:45:05,077 INFO [org.apa.cam.com.sed.SedaEndpoint] (main) Endpoint seda://error is using shared queue: seda://error with size: 1000
2021-03-05 09:45:05,080 INFO [org.apa.cam.qua.cor.CamelBootstrapRecorder] (main) bootstrap runtime: org.apache.camel.quarkus.core.CamelContextRuntime
2021-03-05 09:45:05,081 INFO [org.apa.cam.com.azu.sto.blo.BlobComponent] (main) BlobServiceClient instance is already set at endpoint level: skipping the check in the registry
2021-03-05 09:45:05,082 INFO [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.7.0 (camel-1) is starting
2021-03-05 09:45:05,082 INFO [org.apa.cam.imp.eng.AbstractCamelContext] (main) StreamCaching is not in use. If using streams then it's recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
2021-03-05 09:45:05,091 INFO [org.apa.cam.com.fil.rem.SftpOperations] (main) Known host file not configured, using user known host file: ?/.ssh/known_hosts
2021-03-05 09:45:05,192 WARN [org.apa.cam.com.fil.rem.SftpOperations] (main) JSCH -> Permanently added 'someSftpServer' (RSA) to the list of known hosts.
2021-03-05 09:45:05,193 WARN [org.apa.cam.com.fil.rem.SftpOperations] (main) Server asks for confirmation (yes|no): ?/.ssh/known_hosts does not exist.
Are you sure you want to create it?. Camel will answer no.
2021-03-05 09:45:05,340 INFO [org.apa.cam.imp.eng.InternalRouteStartupManager] (main) Route: sftpToAzure started and consuming from: sftp://someSftpServer/../../Integrations/someDirectory
2021-03-05 09:45:05,340 INFO [org.apa.cam.imp.eng.InternalRouteStartupManager] (main) Route: mailError started and consuming from: seda://error
2021-03-05 09:45:05,340 INFO [org.apa.cam.imp.eng.AbstractCamelContext] (main) Total 2 routes, of which 2 are started
2021-03-05 09:45:05,340 INFO [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 3.7.0 (camel-1) started in 258ms
2021-03-05 09:45:05,342 INFO [io.quarkus] (main) srv0210-someApplicationName 1.0.1-SNAPSHOT native (powered by Quarkus 1.11.0.Final) started in 0.303s. Listening on: http://0.0.0.0:8080
2021-03-05 09:45:05,342 INFO [io.quarkus] (main) Profile test activated.
2021-03-05 09:45:05,342 INFO [io.quarkus] (main) Installed features: [camel-azure-storage-blob, camel-core, camel-file, camel-ftp, camel-log, camel-mail, camel-seda, camel-support-common, camel-support-mail, cdi, config-yaml, jaeger, micrometer, smallrye-health, smallrye-openapi, smallrye-opentracing, swagger-ui]
2021-03-05 09:47:04,810 INFO [sftpToAzure] (Camel (camel-1) thread #0 - sftp://someSftpServer/../../Integrations/someDirectióry) File 20210111150626x.csv (5380 bytes) is fetched from SFTP.
2021-03-05 09:47:04,815 ERROR [rea.cor.sch.Schedulers] (parallel-1) Scheduler worker in group main failed with an uncaught exception: java.lang.UnsatisfiedLinkError: io.netty.
internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference()I [symbol: Java_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipher
ServerPreference or Java_io_netty_internal_tcnative_NativeStaticallyReferencedJniMethods_sslOpCipherServerPreference__]
at com.oracle.svm.jni.access.JNINativeLinkage.getOrFindEntryPoint(JNINativeLinkage.java:153)
at com.oracle.svm.jni.JNIGeneratedMethodSupport.nativeCallAddress(JNIGeneratedMethodSupport.java:57)
at io.netty.internal.tcnative.NativeStaticallyReferencedJniMethods.sslOpCipherServerPreference(NativeStaticallyReferencedJniMethods.java)
at io.netty.internal.tcnative.SSL.<clinit>(SSL.java:67)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
at io.netty.handler.ssl.OpenSsl.<clinit>(OpenSsl.java:121)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
at reactor.netty.tcp.SslProvider.updateDefaultConfiguration(SslProvider.java:388)
at reactor.netty.tcp.SslProvider.<init>(SslProvider.java:309)
at reactor.netty.tcp.SslProvider$Build.build(SslProvider.java:569)
at reactor.netty.tcp.TcpClientSecure.<clinit>(TcpClientSecure.java:71)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
at reactor.netty.tcp.SslProvider.defaultClientProvider(SslProvider.java:100)
at reactor.netty.http.client.HttpClientSecure.<clinit>(HttpClientSecure.java:72)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga edited a comment on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
ppalaga edited a comment on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791399904
I adapted our SFTP test to send to a file to Azure blob service https://github.com/apache/camel-quarkus/pull/2320 and it is passing on the current Camel Quarkus master and GraalVM 21.0.0. The issue might have been fixed by various tweaks in Azure extensions we have done recently.
Anyway, please check whether I have not omitted some important detail in my reproducer.
Let my try to reproduce with CQ 1.6.0 and GraalVM 20.2.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
ppalaga commented on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791311414
Thanks for the report, let me try to reproduce.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] tidemyr commented on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
tidemyr commented on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791460868
Thank you very much for looking at the problem. This can definitely be a solution. :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] tidemyr commented on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
tidemyr commented on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791325245
It's just sets the Azure blob name:
` private final Processor setHeaders = new Processor() {
@Override
public void process(Exchange exchange) throws Exception {
Message message = exchange.getIn();
String datetime = OffsetDateTime.now(ZoneOffset.UTC).format(dtf);
message.setHeader(BlobConstants.BLOB_NAME, "missingName-" + datetime + ".csv");
Map<String, Object> headers = message.getHeaders();
Iterator iterator = headers.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry pair = (Map.Entry)iterator.next();
if (pair.getKey().equals("CamelFileNameOnly")) {
message.setHeader(BlobConstants.BLOB_NAME, pair.getValue().toString());
break;
}
}
}
};
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
ppalaga commented on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791414940
OK, I can confirm, the issue is reproduce with the following combinations:
* CQ 1.6.0 and GraalVM 20.2.0
* CQ 1.6.0 and GraalVM 20.3.1.
But it works with CQ 1.6.0 and GraalVM 21.0.0. Is GraalVM 21.0.0 an option for you @tidemyr?
Thanks for the report, BTW!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
ppalaga commented on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791399904
I adapted our SFTP test to send to a file to Azure blob service https://github.com/apache/camel-quarkus/pull/2320 and I cannot reproduce the issue on the current Camel Quarkus master and GraalVM 21.0.0. The issue might have been fixed caused by various tweaks in Azure extensions we have done recently.
Anyway, please check whether I have not omitted some important detail in my reproducer.
Let my try to reproduce with CQ 1.6.0 and GraalVM 20.2.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga closed issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
ppalaga closed issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] tidemyr edited a comment on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
tidemyr edited a comment on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791325245
It's just sets the Azure blob name (which overrides the required 'blobName' (?blobName=blob&...) option in the Azure URL):
` private final Processor setHeaders = new Processor() {
@Override
public void process(Exchange exchange) throws Exception {
Message message = exchange.getIn();
String datetime = OffsetDateTime.now(ZoneOffset.UTC).format(dtf);
message.setHeader(BlobConstants.BLOB_NAME, "missingName-" + datetime + ".csv");
Map<String, Object> headers = message.getHeaders();
Iterator iterator = headers.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry pair = (Map.Entry)iterator.next();
if (pair.getKey().equals("CamelFileNameOnly")) {
message.setHeader(BlobConstants.BLOB_NAME, pair.getValue().toString());
break;
}
}
}
};
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] ppalaga commented on issue #2318: Camel DSL Azure - Storage Blob Service doesn't work as native build.
Posted by GitBox <gi...@apache.org>.
ppalaga commented on issue #2318:
URL: https://github.com/apache/camel-quarkus/issues/2318#issuecomment-791324263
Could you please give a hint what your `setHeaders` method is doing?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org