You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2021/04/22 15:37:54 UTC
[GitHub] [camel-quarkus] JiriOndrusek opened a new pull request #2505: Google Storage support #2421
JiriOndrusek opened a new pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505
fixes https://github.com/apache/camel-quarkus/issues/2421
[ ] An issue should be filed for the change unless this is a trivial change (fixing a typo or similar). One issue should ideally be fixed by not more than one commit and the other way round, each commit should fix just one issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful and properly spelled subject line and body. Copying the title of the associated issue is typically enough. Please include the issue number in the commit message prefixed by #.
[ ] The pull request description should explain what the pull request does, how, and why. If the info is available in the associated issue or some other external document, a link is enough.
[ ] Phrases like Fix #<issueNumber> or Fixes #<issueNumber> will auto-close the named issue upon merging the pull request. Using them is typically a good idea.
[ ] Please run mvn process-resources -Pformat (and amend the changes if necessary) before sending the pull request.
[ ] Contributor guide is your good friend: https://camel.apache.org/camel-quarkus/latest/contributor-guide.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621312380
##########
File path: extensions/google-storage/deployment/src/main/java/org/apache/camel/quarkus/component/google/storage/deployment/GoogleStorageProcessor.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.deployment;
+
+import com.google.api.client.googleapis.json.GoogleJsonError;
+import com.google.api.client.http.HttpHeaders;
+import com.google.api.client.json.GenericJson;
+import com.google.api.client.json.webtoken.JsonWebSignature;
+import com.google.api.client.json.webtoken.JsonWebToken;
+import com.google.api.client.util.GenericData;
+import com.google.api.services.storage.Storage;
+import com.google.api.services.storage.StorageRequest;
+import com.google.cloud.storage.Bucket;
+import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
+import io.quarkus.deployment.annotations.BuildProducer;
+import io.quarkus.deployment.annotations.BuildStep;
+import io.quarkus.deployment.builditem.CombinedIndexBuildItem;
+import io.quarkus.deployment.builditem.ExtensionSslNativeSupportBuildItem;
+import io.quarkus.deployment.builditem.FeatureBuildItem;
+import io.quarkus.deployment.builditem.IndexDependencyBuildItem;
+import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;
+import org.apache.camel.quarkus.component.google.storage.GoogleStorageEnforcer;
+import org.jboss.jandex.DotName;
+import org.jboss.jandex.IndexView;
+
+class GoogleStorageProcessor {
+
+ private static final String FEATURE = "camel-google-storage";
+
+ @BuildStep
+ FeatureBuildItem feature() {
+ return new FeatureBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ ExtensionSslNativeSupportBuildItem activateSslNativeSupport() {
+ return new ExtensionSslNativeSupportBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ public AdditionalBeanBuildItem storageEnforcer() {
+ return new AdditionalBeanBuildItem(GoogleStorageEnforcer.class);
+ }
+
+ @BuildStep
+ void registerForReflection(CombinedIndexBuildItem combinedIndex,
Review comment:
@jamesnetherton I'll report an issue to quarkiverse to upgrade their reference for `com.google.cloud:google-cloud-storage:jar`. Once this fix is provided, we can remove exclusion from camel component.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r618975319
##########
File path: tooling/scripts/test-categories.yaml
##########
@@ -32,6 +32,7 @@ group-02:
- elasticsearch-rest
- google-bigquery
- google-pubsub
+ - google-storage
Review comment:
Sounds correct, thanks for the link
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-827700973
Please do not merge this PR yet. I need to look into a way of cleaning data after this test (which was removed during last refactor)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] jamesnetherton commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
jamesnetherton commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-825419628
Shouldn't we use the Quarkiverse google cloud bits given that it should have already solved the native part:
https://github.com/quarkiverse/quarkus-google-cloud-services/tree/main/storage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-829878873
@aldettinger Thanks, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r623646358
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.quarkus.test.mock.backend.MockBackendUtils;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.DEST_BUCKET;
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.TEST_BUCKET1;
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.TEST_BUCKET2;
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.TEST_BUCKET3;
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @AfterEach
+ public void afterEach() {
+ //clean after test (only in real environment)
+ if (!GoogleStorageHelper.usingMockBackend()) {
+ RestAssured.given()
+ .get("/google-storage/deleteBuckets")
+ .then()
+ .statusCode(200);
+ }
+ }
+
+ @Test
+ public void testConsumer() throws InterruptedException {
+ try {
+ //producer - putObject
+ putObject("Sheldon", TEST_BUCKET3, FILE_NAME_007);
+
+ //get result from direct (for pooling) with timeout
+ RestAssured.given()
+ .post("/google-storage/getFromDirect")
+ .then()
+ .statusCode(200)
+ .body(is("Sheldon"));
+
+ //producer - getObject
+ executeOperation(DEST_BUCKET, GoogleCloudStorageOperations.getObject,
+ Collections.singletonMap(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007),
+ is("Sheldon"));
+
+ } finally {
+ //stop route to allow bucket deletion without errors in real environment
+ if (!GoogleStorageHelper.usingMockBackend()) {
+ RestAssured.given()
+ .get("/google-storage/stopRoute")
+ .then()
+ .statusCode(200);
+ }
+ }
+ }
+
+ @Test
+ public void testProducer() {
+ //delete existing buckets t - only on real account - Deleting buckets is not (yet) supported by fsouza/fake-gcs-server.
+ if (!MockBackendUtils.startMockBackend()) {
+ String buckets = executeOperation(GoogleCloudStorageOperations.listBuckets, Collections.emptyMap(),
+ null);
+ List<String> bucketsToDelete = Arrays.stream(buckets.split(","))
+ .filter(b -> b.equals(TEST_BUCKET1) || b.equals(TEST_BUCKET2))
+ .collect(Collectors.toList());
+ if (!bucketsToDelete.isEmpty()) {
+ bucketsToDelete.forEach(
+ b -> executeOperation(b, GoogleCloudStorageOperations.deleteBucket, Collections.emptyMap(),
+ is(Boolean.toString(true))));
+ }
+ }
+
+ //create object in testBucket
+ putObject("Sheldon", TEST_BUCKET1, FILE_NAME_007);
+
+ putObject("Irma", TEST_BUCKET2, FILE_NAME_006);
+
+ //copy object to test_bucket2
+ executeOperation(GoogleCloudStorageOperations.copyObject,
+ CollectionHelper.mapOf(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007,
+ GoogleCloudStorageConstants.DESTINATION_BUCKET_NAME, TEST_BUCKET2,
+ GoogleCloudStorageConstants.DESTINATION_OBJECT_NAME, FILE_NAME_007 + "_copy"),
+ is("Sheldon"));
+
+ //GetObject
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.getObject,
+ Collections.singletonMap(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007 + "_copy"),
+ is("Sheldon"));
+
+ //list buckets
+ executeOperation(GoogleCloudStorageOperations.listBuckets, Collections.emptyMap(),
+ both(containsString(TEST_BUCKET1)).and(containsString(TEST_BUCKET2)));
+
+ //deleteObject
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.deleteObject,
+ CollectionHelper.mapOf(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_006),
+ is(Boolean.toString(true)));
+
+ //ListObjects
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.listObjects, Collections.emptyMap(),
+ containsString(FILE_NAME_007 + "_copy"));
+
+ //CreateDownloadLink - requires authentication
+ if (!GoogleStorageHelper.usingMockBackend()) {
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.createDownloadLink,
+ Collections.singletonMap(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007 + "_copy"),
+ startsWith("http"));
+ }
+ }
+
+ private void putObject(String sheldon, String testBucket1, String fileName007) {
Review comment:
Fixed, probably generated method and I didn't notice wrong param names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621181875
##########
File path: extensions/google-storage/deployment/src/main/java/org/apache/camel/quarkus/component/google/storage/deployment/GoogleStorageProcessor.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.deployment;
+
+import com.google.api.client.googleapis.json.GoogleJsonError;
+import com.google.api.client.http.HttpHeaders;
+import com.google.api.client.json.GenericJson;
+import com.google.api.client.json.webtoken.JsonWebSignature;
+import com.google.api.client.json.webtoken.JsonWebToken;
+import com.google.api.client.util.GenericData;
+import com.google.api.services.storage.Storage;
+import com.google.api.services.storage.StorageRequest;
+import com.google.cloud.storage.Bucket;
+import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
+import io.quarkus.deployment.annotations.BuildProducer;
+import io.quarkus.deployment.annotations.BuildStep;
+import io.quarkus.deployment.builditem.CombinedIndexBuildItem;
+import io.quarkus.deployment.builditem.ExtensionSslNativeSupportBuildItem;
+import io.quarkus.deployment.builditem.FeatureBuildItem;
+import io.quarkus.deployment.builditem.IndexDependencyBuildItem;
+import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;
+import org.apache.camel.quarkus.component.google.storage.GoogleStorageEnforcer;
+import org.jboss.jandex.DotName;
+import org.jboss.jandex.IndexView;
+
+class GoogleStorageProcessor {
+
+ private static final String FEATURE = "camel-google-storage";
+
+ @BuildStep
+ FeatureBuildItem feature() {
+ return new FeatureBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ ExtensionSslNativeSupportBuildItem activateSslNativeSupport() {
+ return new ExtensionSslNativeSupportBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ public AdditionalBeanBuildItem storageEnforcer() {
+ return new AdditionalBeanBuildItem(GoogleStorageEnforcer.class);
+ }
+
+ @BuildStep
+ void registerForReflection(CombinedIndexBuildItem combinedIndex,
Review comment:
yes that is right it should be covered by quarkiverse. I'll recheck it again. I removed native things and did not cover build items.
As for upgrade of the version, my problem was caused by local error. Version of quarkiverse is using `1.113.4`, camel is using `1.113.4`. And it looks there is a newer version of quarkiverse, which is using much higher version of google cloud libraries, So I would say that current state is acceptable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622453557
##########
File path: integration-tests/google-storage/src/main/java/org/apache/camel/quarkus/component/google/storage/it/RealAccountPropertyProducer.java
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.eclipse.microprofile.config.spi.ConfigSource;
+import org.eclipse.microprofile.config.spi.ConfigSourceProvider;
+
+public class RealAccountPropertyProducer implements ConfigSourceProvider {
+
Review comment:
Can't we simply use env var from properties file, something like https://quarkus.io/guides/config-reference#combine-property-env-var ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r618671079
##########
File path: extensions/google-storage/runtime/src/main/java/org/apache/camel/quarkus/component/google/storage/UrlFetchTransportSubstitute.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage;
+
+import java.io.IOException;
+
+import com.google.api.client.extensions.appengine.http.UrlFetchTransport;
+import com.oracle.svm.core.annotate.Substitute;
+import com.oracle.svm.core.annotate.TargetClass;
+
+@TargetClass(value = UrlFetchTransport.class)
+public final class UrlFetchTransportSubstitute {
+
+ @Substitute
+ protected Object buildRequest(String method, String url) throws IOException {
+ throw new IllegalStateException("Not used by camel extension");
Review comment:
I would say the common usage in such situations is to throw NotSupportedException ? Also do you think "camel-quarkus extension" in the message would be more explicit ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger merged pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger merged pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-836433923
@aldettinger yes, I resolved the last conversation. I forgot to state that PR is ready to be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622775341
##########
File path: docs/modules/ROOT/pages/reference/extensions/google-storage.adoc
##########
@@ -0,0 +1,60 @@
+// Do not edit directly!
+// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
+= Google Storage
+:linkattrs:
+:cq-artifact-id: camel-quarkus-google-storage
+:cq-native-supported: true
+:cq-status: Stable
+:cq-status-deprecation: Stable
+:cq-description: Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+:cq-deprecated: false
+:cq-jvm-since: 1.9.0
+:cq-native-since: 1.9.0
+
+[.badges]
+[.badge-key]##JVM since##[.badge-supported]##1.9.0## [.badge-key]##Native since##[.badge-supported]##1.9.0##
+
+Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+
+== What's inside
+
+* xref:{cq-camel-components}::google-storage-component.adoc[Google Storage component], URI syntax: `google-storage:bucketName`
+
+Please refer to the above link for usage and configuration details.
+
+== Maven coordinates
+
+https://code.quarkus.io/?extension-search=camel-quarkus-google-storage[Create a new project with this extension on code.quarkus.io, window="_blank"]
+
+Or add the coordinates to your existing project:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel.quarkus</groupId>
+ <artifactId>camel-quarkus-google-storage</artifactId>
+</dependency>
+----
+
+Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.
+
+== SSL in native mode
+
+This extension auto-enables SSL support in native mode. Hence you do not need to add
+`quarkus.ssl.native=true` to your `application.properties` yourself. See also
+https://quarkus.io/guides/native-and-ssl[Quarkus SSL guide].
+
+== Additional Camel Quarkus configuration
+
+There are two different configuration approaches:
+
+* Google storage client can be defined via quarkus properties leveraging the Quarkiverse - Google Cloud Services - Storage (see https://github.com/quarkiverse/quarkus-google-cloud-services/blob/main/storage[extension git page]).
+Camel will autowire client into the Google-storage component.
+This configuration allows definition of only one storage client, therefore it isn't possible to define several different endpoints, which run together.
+
+* Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint.
Review comment:
@lburgazzoli I plan to open an issue to make configuration of client optional (see previous comment), but registration of multiple clients as you suggested makes even better sense. I'll create it in that way.
##########
File path: extensions/google-storage/runtime/src/main/java/org/apache/camel/quarkus/component/google/storage/GoogleStorageEnforcer.java
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage;
+
+import javax.enterprise.context.ApplicationScoped;
+import javax.enterprise.inject.Instance;
+import javax.inject.Inject;
+
+import com.google.cloud.storage.Storage;
+
+@ApplicationScoped
+public class GoogleStorageEnforcer {
+
+ //Quarkiverse StorageProducer creates storage based on properties.
+ //Because of https://github.com/apache/camel-quarkus/issues/1387, storage is removed and not applied to component.
+ //UnremovableBeanBuildItem does not fix this.
+ //Injecting all storages makes Quarkiverse StorageProducer to be used.
+ //We are injecting all storages, because client can define more storages. (so injecting only one could fail|)
Review comment:
I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-825429575
@jamesnetherton It makes totally sense. When I assigned this issue, there was no such extension (or I searched it badly). I'll rework it, thanks for noticing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622817351
##########
File path: extensions/google-storage/runtime/src/main/java/org/apache/camel/quarkus/component/google/storage/GoogleStorageEnforcer.java
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage;
+
+import javax.enterprise.context.ApplicationScoped;
+import javax.enterprise.inject.Instance;
+import javax.inject.Inject;
+
+import com.google.cloud.storage.Storage;
+
+@ApplicationScoped
+public class GoogleStorageEnforcer {
+
+ //Quarkiverse StorageProducer creates storage based on properties.
+ //Because of https://github.com/apache/camel-quarkus/issues/1387, storage is removed and not applied to component.
+ //UnremovableBeanBuildItem does not fix this.
+ //Injecting all storages makes Quarkiverse StorageProducer to be used.
+ //We are injecting all storages, because client can define more storages. (so injecting only one could fail|)
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r619015372
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String DEST_BUCKET = "camel_quarkus_test_dest_bucket";
+ private static final String TEST_BUCKET1 = "camel_quarkus_test_bucket1";
+ private static final String TEST_BUCKET2 = "camel_quarkus_test_bucket2";
+ private static final String TEST_BUCKET3 = "camel_quarkus_test_bucket3";
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @BeforeEach
+ public void beforeEach() {
+ RestAssured.given().get("/google-storage/loadComponent").then().statusCode(200);
Review comment:
So, it's a bit like we would need to initialize the component for tests once before all tests are run. The only idea I have is `@BeforeAll`. static should not be an issue, but from your line I deduce that the quarkus junit extension would not have started enough at this stage. If that's the case, I don't have better idea. Thanks for explanations :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r619198640
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String DEST_BUCKET = "camel_quarkus_test_dest_bucket";
+ private static final String TEST_BUCKET1 = "camel_quarkus_test_bucket1";
+ private static final String TEST_BUCKET2 = "camel_quarkus_test_bucket2";
+ private static final String TEST_BUCKET3 = "camel_quarkus_test_bucket3";
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @BeforeEach
+ public void beforeEach() {
+ RestAssured.given().get("/google-storage/loadComponent").then().statusCode(200);
Review comment:
@aldettinger Exactly as you said, problem is not caused by the static call, but because it was called even sooner then Resource was prepared. But example from @zbendhiba looks very promising. I'll try it (with the bigger refactor to use quarkiverse extension)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622817638
##########
File path: integration-tests/google-storage/src/main/java/org/apache/camel/quarkus/component/google/storage/it/RealAccountPropertyProducer.java
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.eclipse.microprofile.config.spi.ConfigSource;
+import org.eclipse.microprofile.config.spi.ConfigSourceProvider;
+
+public class RealAccountPropertyProducer implements ConfigSourceProvider {
+
Review comment:
@aldettinger it works now, thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621108407
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String DEST_BUCKET = "camel_quarkus_test_dest_bucket";
+ private static final String TEST_BUCKET1 = "camel_quarkus_test_bucket1";
+ private static final String TEST_BUCKET2 = "camel_quarkus_test_bucket2";
+ private static final String TEST_BUCKET3 = "camel_quarkus_test_bucket3";
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @BeforeEach
+ public void beforeEach() {
+ RestAssured.given().get("/google-storage/loadComponent").then().statusCode(200);
Review comment:
@aldettinger I used approach according to @zbendhiba
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622776782
##########
File path: integration-tests/google-storage/src/main/java/org/apache/camel/quarkus/component/google/storage/it/RealAccountPropertyProducer.java
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.eclipse.microprofile.config.spi.ConfigSource;
+import org.eclipse.microprofile.config.spi.ConfigSourceProvider;
+
+public class RealAccountPropertyProducer implements ConfigSourceProvider {
+
Review comment:
@aldettinger this is a nice idea. I already tried it and I was facing an issue with it. But now I think that it should be possible to figure it out. I'll try it once again. Thanks for suggestion!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r619196044
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String DEST_BUCKET = "camel_quarkus_test_dest_bucket";
+ private static final String TEST_BUCKET1 = "camel_quarkus_test_bucket1";
+ private static final String TEST_BUCKET2 = "camel_quarkus_test_bucket2";
+ private static final String TEST_BUCKET3 = "camel_quarkus_test_bucket3";
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @BeforeEach
+ public void beforeEach() {
+ RestAssured.given().get("/google-storage/loadComponent").then().statusCode(200);
Review comment:
thanks @zbendhiba for the example. Using `@Produce` sounds great for this case. Exactly the way I was looking for. Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622833075
##########
File path: docs/modules/ROOT/pages/reference/extensions/google-storage.adoc
##########
@@ -0,0 +1,60 @@
+// Do not edit directly!
+// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
+= Google Storage
+:linkattrs:
+:cq-artifact-id: camel-quarkus-google-storage
+:cq-native-supported: true
+:cq-status: Stable
+:cq-status-deprecation: Stable
+:cq-description: Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+:cq-deprecated: false
+:cq-jvm-since: 1.9.0
+:cq-native-since: 1.9.0
+
+[.badges]
+[.badge-key]##JVM since##[.badge-supported]##1.9.0## [.badge-key]##Native since##[.badge-supported]##1.9.0##
+
+Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+
+== What's inside
+
+* xref:{cq-camel-components}::google-storage-component.adoc[Google Storage component], URI syntax: `google-storage:bucketName`
+
+Please refer to the above link for usage and configuration details.
+
+== Maven coordinates
+
+https://code.quarkus.io/?extension-search=camel-quarkus-google-storage[Create a new project with this extension on code.quarkus.io, window="_blank"]
+
+Or add the coordinates to your existing project:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel.quarkus</groupId>
+ <artifactId>camel-quarkus-google-storage</artifactId>
+</dependency>
+----
+
+Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.
+
+== SSL in native mode
+
+This extension auto-enables SSL support in native mode. Hence you do not need to add
+`quarkus.ssl.native=true` to your `application.properties` yourself. See also
+https://quarkus.io/guides/native-and-ssl[Quarkus SSL guide].
+
+== Additional Camel Quarkus configuration
+
+There are two different configuration approaches:
+
+* Google storage client can be defined via quarkus properties leveraging the Quarkiverse - Google Cloud Services - Storage (see https://github.com/quarkiverse/quarkus-google-cloud-services/blob/main/storage[extension git page]).
+Camel will autowire client into the Google-storage component.
+This configuration allows definition of only one storage client, therefore it isn't possible to define several different endpoints, which run together.
+
+* Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint.
Review comment:
Doc is changed as requested.
Issue about multiple clients registration is created - https://github.com/quarkiverse/quarkus-google-cloud-services/issues/114
Follow-up isuue in camel-quarkus - https://github.com/apache/camel-quarkus/issues/2534
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621181875
##########
File path: extensions/google-storage/deployment/src/main/java/org/apache/camel/quarkus/component/google/storage/deployment/GoogleStorageProcessor.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.deployment;
+
+import com.google.api.client.googleapis.json.GoogleJsonError;
+import com.google.api.client.http.HttpHeaders;
+import com.google.api.client.json.GenericJson;
+import com.google.api.client.json.webtoken.JsonWebSignature;
+import com.google.api.client.json.webtoken.JsonWebToken;
+import com.google.api.client.util.GenericData;
+import com.google.api.services.storage.Storage;
+import com.google.api.services.storage.StorageRequest;
+import com.google.cloud.storage.Bucket;
+import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
+import io.quarkus.deployment.annotations.BuildProducer;
+import io.quarkus.deployment.annotations.BuildStep;
+import io.quarkus.deployment.builditem.CombinedIndexBuildItem;
+import io.quarkus.deployment.builditem.ExtensionSslNativeSupportBuildItem;
+import io.quarkus.deployment.builditem.FeatureBuildItem;
+import io.quarkus.deployment.builditem.IndexDependencyBuildItem;
+import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;
+import org.apache.camel.quarkus.component.google.storage.GoogleStorageEnforcer;
+import org.jboss.jandex.DotName;
+import org.jboss.jandex.IndexView;
+
+class GoogleStorageProcessor {
+
+ private static final String FEATURE = "camel-google-storage";
+
+ @BuildStep
+ FeatureBuildItem feature() {
+ return new FeatureBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ ExtensionSslNativeSupportBuildItem activateSslNativeSupport() {
+ return new ExtensionSslNativeSupportBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ public AdditionalBeanBuildItem storageEnforcer() {
+ return new AdditionalBeanBuildItem(GoogleStorageEnforcer.class);
+ }
+
+ @BuildStep
+ void registerForReflection(CombinedIndexBuildItem combinedIndex,
Review comment:
yes that is right it should be covered by quarkiverse. I'll recheck it again. I removed native things and did not cover build items.
As for upgrade of the version, my problem was caused by local error. Version of quarkiverse is using `1.113.4`, camel is using `1.113.8`. And it looks there is a newer version of quarkiverse, ~~which is using much higher version of google cloud libraries, So I would say that current state is acceptable~~, but it uses the same version (`1.113.4`) -I verified it on current `0.7.0-SNAPSHOT` version of quarkiverse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621181875
##########
File path: extensions/google-storage/deployment/src/main/java/org/apache/camel/quarkus/component/google/storage/deployment/GoogleStorageProcessor.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.deployment;
+
+import com.google.api.client.googleapis.json.GoogleJsonError;
+import com.google.api.client.http.HttpHeaders;
+import com.google.api.client.json.GenericJson;
+import com.google.api.client.json.webtoken.JsonWebSignature;
+import com.google.api.client.json.webtoken.JsonWebToken;
+import com.google.api.client.util.GenericData;
+import com.google.api.services.storage.Storage;
+import com.google.api.services.storage.StorageRequest;
+import com.google.cloud.storage.Bucket;
+import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
+import io.quarkus.deployment.annotations.BuildProducer;
+import io.quarkus.deployment.annotations.BuildStep;
+import io.quarkus.deployment.builditem.CombinedIndexBuildItem;
+import io.quarkus.deployment.builditem.ExtensionSslNativeSupportBuildItem;
+import io.quarkus.deployment.builditem.FeatureBuildItem;
+import io.quarkus.deployment.builditem.IndexDependencyBuildItem;
+import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;
+import org.apache.camel.quarkus.component.google.storage.GoogleStorageEnforcer;
+import org.jboss.jandex.DotName;
+import org.jboss.jandex.IndexView;
+
+class GoogleStorageProcessor {
+
+ private static final String FEATURE = "camel-google-storage";
+
+ @BuildStep
+ FeatureBuildItem feature() {
+ return new FeatureBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ ExtensionSslNativeSupportBuildItem activateSslNativeSupport() {
+ return new ExtensionSslNativeSupportBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ public AdditionalBeanBuildItem storageEnforcer() {
+ return new AdditionalBeanBuildItem(GoogleStorageEnforcer.class);
+ }
+
+ @BuildStep
+ void registerForReflection(CombinedIndexBuildItem combinedIndex,
Review comment:
yes that is right it should be covered by quarkiverse. I'll recheck it again. I removed native things and did not cover build items.
As for upgrade of the version, my problem was caused by local error. Version of quarkiverse is using `1.113.4`, camel is using `1.113.8`. And it looks there is a newer version of quarkiverse, which is using much higher version of google cloud libraries, So I would say that current state is acceptable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r618679346
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String DEST_BUCKET = "camel_quarkus_test_dest_bucket";
+ private static final String TEST_BUCKET1 = "camel_quarkus_test_bucket1";
+ private static final String TEST_BUCKET2 = "camel_quarkus_test_bucket2";
+ private static final String TEST_BUCKET3 = "camel_quarkus_test_bucket3";
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @BeforeEach
+ public void beforeEach() {
+ RestAssured.given().get("/google-storage/loadComponent").then().statusCode(200);
Review comment:
Do we really need do this before each test ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] lburgazzoli commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
lburgazzoli commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622448354
##########
File path: docs/modules/ROOT/pages/reference/extensions/google-storage.adoc
##########
@@ -0,0 +1,60 @@
+// Do not edit directly!
+// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
+= Google Storage
+:linkattrs:
+:cq-artifact-id: camel-quarkus-google-storage
+:cq-native-supported: true
+:cq-status: Stable
+:cq-status-deprecation: Stable
+:cq-description: Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+:cq-deprecated: false
+:cq-jvm-since: 1.9.0
+:cq-native-since: 1.9.0
+
+[.badges]
+[.badge-key]##JVM since##[.badge-supported]##1.9.0## [.badge-key]##Native since##[.badge-supported]##1.9.0##
+
+Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+
+== What's inside
+
+* xref:{cq-camel-components}::google-storage-component.adoc[Google Storage component], URI syntax: `google-storage:bucketName`
+
+Please refer to the above link for usage and configuration details.
+
+== Maven coordinates
+
+https://code.quarkus.io/?extension-search=camel-quarkus-google-storage[Create a new project with this extension on code.quarkus.io, window="_blank"]
+
+Or add the coordinates to your existing project:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel.quarkus</groupId>
+ <artifactId>camel-quarkus-google-storage</artifactId>
+</dependency>
+----
+
+Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.
+
+== SSL in native mode
+
+This extension auto-enables SSL support in native mode. Hence you do not need to add
+`quarkus.ssl.native=true` to your `application.properties` yourself. See also
+https://quarkus.io/guides/native-and-ssl[Quarkus SSL guide].
+
+== Additional Camel Quarkus configuration
+
+There are two different configuration approaches:
+
+* Google storage client can be defined via quarkus properties leveraging the Quarkiverse - Google Cloud Services - Storage (see https://github.com/quarkiverse/quarkus-google-cloud-services/blob/main/storage[extension git page]).
+Camel will autowire client into the Google-storage component.
+This configuration allows definition of only one storage client, therefore it isn't possible to define several different endpoints, which run together.
+
+* Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint.
Review comment:
can we open an issue on the extension to check if configuring multiple client would be an option in the future ?
quarkus has such option for jdbc, maybe it could be possible to have the same logic here too (and probably for other extensions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-836382916
Hey @JiriOndrusek , were all the comments addressed ? Could this PR be merged ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622774302
##########
File path: docs/modules/ROOT/pages/reference/extensions/google-storage.adoc
##########
@@ -0,0 +1,60 @@
+// Do not edit directly!
+// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
+= Google Storage
+:linkattrs:
+:cq-artifact-id: camel-quarkus-google-storage
+:cq-native-supported: true
+:cq-status: Stable
+:cq-status-deprecation: Stable
+:cq-description: Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+:cq-deprecated: false
+:cq-jvm-since: 1.9.0
+:cq-native-since: 1.9.0
+
+[.badges]
+[.badge-key]##JVM since##[.badge-supported]##1.9.0## [.badge-key]##Native since##[.badge-supported]##1.9.0##
+
+Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+
+== What's inside
+
+* xref:{cq-camel-components}::google-storage-component.adoc[Google Storage component], URI syntax: `google-storage:bucketName`
+
+Please refer to the above link for usage and configuration details.
+
+== Maven coordinates
+
+https://code.quarkus.io/?extension-search=camel-quarkus-google-storage[Create a new project with this extension on code.quarkus.io, window="_blank"]
+
+Or add the coordinates to your existing project:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel.quarkus</groupId>
+ <artifactId>camel-quarkus-google-storage</artifactId>
+</dependency>
+----
+
+Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.
+
+== SSL in native mode
+
+This extension auto-enables SSL support in native mode. Hence you do not need to add
+`quarkus.ssl.native=true` to your `application.properties` yourself. See also
+https://quarkus.io/guides/native-and-ssl[Quarkus SSL guide].
+
+== Additional Camel Quarkus configuration
+
+There are two different configuration approaches:
+
+* Google storage client can be defined via quarkus properties leveraging the Quarkiverse - Google Cloud Services - Storage (see https://github.com/quarkiverse/quarkus-google-cloud-services/blob/main/storage[extension git page]).
+Camel will autowire client into the Google-storage component.
+This configuration allows definition of only one storage client, therefore it isn't possible to define several different endpoints, which run together.
+
+* Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint.
Review comment:
@aldettinger Problem is unfortunately a little bit bigger here. Application properties has to be used always. Even if you define you own multiple clients. Reason is that quarkiverse extension always creates client and if there are no properties, creation of client fails with
>java.io.IOException: The Application Default Credentials are not available
It is unfortunate to require app properties to be filled even if it is not required, but I haven't found a way of making it worl (and to keep autowiring from camel component working) I plan to raise a ticket to quarkiverse extension asking for optional configuration of client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r618700554
##########
File path: tooling/scripts/test-categories.yaml
##########
@@ -32,6 +32,7 @@ group-02:
- elasticsearch-rest
- google-bigquery
- google-pubsub
+ - google-storage
Review comment:
group-01 or group-08 may achieve a greater balance https://github.com/apache/camel-quarkus/actions/runs/773244432
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621108086
##########
File path: extensions/google-storage/runtime/src/main/java/org/apache/camel/quarkus/component/google/storage/UrlFetchTransportSubstitute.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage;
+
+import java.io.IOException;
+
+import com.google.api.client.extensions.appengine.http.UrlFetchTransport;
+import com.oracle.svm.core.annotate.Substitute;
+import com.oracle.svm.core.annotate.TargetClass;
+
+@TargetClass(value = UrlFetchTransport.class)
+public final class UrlFetchTransportSubstitute {
+
+ @Substitute
+ protected Object buildRequest(String method, String url) throws IOException {
+ throw new IllegalStateException("Not used by camel extension");
Review comment:
@aldettinger Class was removed during re-work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621108662
##########
File path: tooling/scripts/test-categories.yaml
##########
@@ -32,6 +32,7 @@ group-02:
- elasticsearch-rest
- google-bigquery
- google-pubsub
+ - google-storage
Review comment:
@aldettinger I moved test into group-01
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621183262
##########
File path: integration-tests/google-storage/src/main/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageResource.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.stream.Collectors;
+
+import javax.enterprise.context.ApplicationScoped;
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+import com.google.cloud.storage.Blob;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.CopyWriter;
+import com.google.cloud.storage.StorageOptions;
+import io.quarkiverse.googlecloudservices.storage.runtime.StorageProducer;
+import io.quarkus.arc.Unremovable;
+import org.apache.camel.ConsumerTemplate;
+import org.apache.camel.ProducerTemplate;
+import org.apache.camel.component.google.storage.GoogleCloudStorageComponent;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.jboss.logging.Logger;
+
+@Path("/google-storage")
+@ApplicationScoped
+public class GoogleStorageResource {
+
+ public static final String DIRECT_POLLING = "direct:polling";
+
+ public static final String PARAM_PORT = "org.apache.camel.quarkus.component.googlr.storage.it.GoogleStorageClientProducer_port";
+
+ public static final String QUERY_OBJECT_NAME = "objectName";
+ public static final String QUERY_BUCKET = "bucketName";
+ public static final String QUERY_OPERATION = "operation";
+ public static final String QUERY_DESTINATION_BUCKET = "destinationBucket";
+ public static final String QUERY_DIRECT = "fromDirect";
+ public static final String QUERY_POLLING_ACTION = "pollingAction";
+
+ private static final Logger LOG = Logger.getLogger(GoogleStorageResource.class);
+ private static final String COMPONENT_GOOGLE_STORAGE = "google-storage";
+
+ @Inject
+ ProducerTemplate producerTemplate;
+
+ @Inject
+ ConsumerTemplate consumerTemplate;
+
+ @Inject
+ StorageProducer sp;
+
+ @Produces
+ @ApplicationScoped
+ @Unremovable
+ @Named(COMPONENT_GOOGLE_STORAGE)
+ GoogleCloudStorageComponent produceComponent() throws IOException {
+ GoogleCloudStorageComponent gsc = new GoogleCloudStorageComponent();
+ if (!GoogleStorageHelper.isRealAccount()) {
+ String port = System.getProperty(GoogleStorageResource.PARAM_PORT);
+ gsc.getConfiguration().setStorageClient(StorageOptions.newBuilder()
+ .setHost("http://localhost:" + port)
+ .setProjectId("dummy-project-for-testing")
+ .build()
+ .getService());
+ } else {
+ gsc.getConfiguration().setStorageClient(sp.storage());
+ }
+ return gsc;
+ }
+
+ @Path("/operation")
+ @POST
+ @Consumes(MediaType.APPLICATION_JSON)
+ @Produces(MediaType.TEXT_PLAIN)
+ public String operation(Map<String, Object> parameters,
+ @QueryParam(QUERY_OPERATION) String operation,
+ @QueryParam(QUERY_BUCKET) String bucketName) throws Exception {
+ GoogleCloudStorageOperations op = GoogleCloudStorageOperations.valueOf(operation);
+ String url = getBaseUrl(bucketName, "operation=" + op.toString());
+ final Object response = producerTemplate.requestBodyAndHeaders(url, null, parameters, Object.class);
+ if (response instanceof Blob) {
+ return new String(((Blob) response).getContent());
+ }
+ if (response instanceof CopyWriter) {
+ return new String(((CopyWriter) response).getResult().getContent());
+ }
+ if (response instanceof List) {
+ List l = (List) response;
+ return (String) l.stream().map(o -> {
+ if (o instanceof Bucket) {
+ return ((Bucket) o).getName();
+ }
+ if (o instanceof Blob) {
+ return ((Blob) o).getName();
+ }
+ return "null";
+ }).collect(Collectors.joining(","));
+ }
+ return String.valueOf(response);
+ }
+
+ @Path("/putObject")
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ @Produces(MediaType.TEXT_PLAIN)
+ public Response putObject(String body,
+ @QueryParam(QUERY_BUCKET) String bucketName,
+ @QueryParam(QUERY_OBJECT_NAME) String objectName) throws Exception {
+ String url = getBaseUrl(bucketName, "autoCreateBucket=true");
+ final Blob response = producerTemplate.requestBodyAndHeader(url,
+ body,
+ GoogleCloudStorageConstants.OBJECT_NAME, objectName, Blob.class);
+ return Response
+ .created(new URI("https://camel.apache.org/"))
+ .entity(response.getName())
+ .build();
+ }
+
+ @Path("/startPolling")
+ @POST
+ public void startPolling(@QueryParam(QUERY_BUCKET) String bucketName,
+ @QueryParam(QUERY_POLLING_ACTION) String pollingAction,
+ @QueryParam(QUERY_DESTINATION_BUCKET) String destinationBucket) {
+ // use another thread for polling consumer to demonstrate that we can wait before
+ // the message is sent to the queue
+ Executors.newSingleThreadExecutor().execute(() -> {
Review comment:
No problem with it. I thought that not using seda|mock|direct libraries is better solution. I'll use route then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-827535359
@jamesnetherton I reworked extension to use quarkiverse .
There a few points I was facing:
1. Quarkiverse extension uses quarkus properties to create a storage client. This client was not autowired into component (because of https://github.com/apache/camel-quarkus/issues/1387). I tried to use `UnremovableBeanBuildItem`, but it had no effect. So I created class `GoogleStorageEnforcer`, which injects all storage clients, which marks them as used.
2. To allow more different clients in one context, there is a way of doing it via producing more storage clients - it is described in doc.
3. There is a warning during native build
> WARNING: Could not resolve io.grpc.netty.shaded.io.netty.channel.socket.nio.NioServerSocketChannel for reflection configuration. Reason: java.lang.ClassNotFoundException: io.grpc.netty.shaded.io.netty.channel.socket.nio.NioServerSocketChannel.
, but it doesn't make extension fail.
4. There is a requirement to export `GOOGLE_PROJECT_ID` and `GOOGLE_APPLICATION_CREDENTIALS` to run tests with real account. It is described in README.adoc in the test project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r618981647
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String DEST_BUCKET = "camel_quarkus_test_dest_bucket";
+ private static final String TEST_BUCKET1 = "camel_quarkus_test_bucket1";
+ private static final String TEST_BUCKET2 = "camel_quarkus_test_bucket2";
+ private static final String TEST_BUCKET3 = "camel_quarkus_test_bucket3";
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @BeforeEach
+ public void beforeEach() {
+ RestAssured.given().get("/google-storage/loadComponent").then().statusCode(200);
Review comment:
To be honest I don't like it either, but I need to configure component before usage. Better approach would be to use `@BeforeAll`, but than the method has to be static, which won't allow me to call `/google-storage/loadComponent`.
The other approach I see is to allow order of the tests and then call it only once in the first test. But this approach seems like a lesser evil to me. (there is an if statement, which skips second initialization) WDYT? @aldettinger
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621307789
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTestResource.java
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Collections;
+import java.util.Map;
+
+import io.quarkus.test.common.QuarkusTestResourceLifecycleManager;
+import org.apache.camel.util.CollectionHelper;
+import org.testcontainers.containers.GenericContainer;
+
+public class GoogleStorageTestResource implements QuarkusTestResourceLifecycleManager {
+
+ public static final int PORT = 4443;
+ public static final String CONTAINER_NAME = "fsouza/fake-gcs-server";
+
+ private GenericContainer<?> container;
+
+ @Override
+ public Map<String, String> start() {
+ //if there is a real configuration, don't start container
Review comment:
I used `MockBackendUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-829037506
@aldettinger all requested changes are fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621307569
##########
File path: integration-tests/google-storage/src/main/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageResource.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.stream.Collectors;
+
+import javax.enterprise.context.ApplicationScoped;
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+import com.google.cloud.storage.Blob;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.CopyWriter;
+import com.google.cloud.storage.StorageOptions;
+import io.quarkiverse.googlecloudservices.storage.runtime.StorageProducer;
+import io.quarkus.arc.Unremovable;
+import org.apache.camel.ConsumerTemplate;
+import org.apache.camel.ProducerTemplate;
+import org.apache.camel.component.google.storage.GoogleCloudStorageComponent;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.jboss.logging.Logger;
+
+@Path("/google-storage")
+@ApplicationScoped
+public class GoogleStorageResource {
+
+ public static final String DIRECT_POLLING = "direct:polling";
+
+ public static final String PARAM_PORT = "org.apache.camel.quarkus.component.googlr.storage.it.GoogleStorageClientProducer_port";
+
+ public static final String QUERY_OBJECT_NAME = "objectName";
+ public static final String QUERY_BUCKET = "bucketName";
+ public static final String QUERY_OPERATION = "operation";
+ public static final String QUERY_DESTINATION_BUCKET = "destinationBucket";
+ public static final String QUERY_DIRECT = "fromDirect";
+ public static final String QUERY_POLLING_ACTION = "pollingAction";
+
+ private static final Logger LOG = Logger.getLogger(GoogleStorageResource.class);
+ private static final String COMPONENT_GOOGLE_STORAGE = "google-storage";
+
+ @Inject
+ ProducerTemplate producerTemplate;
+
+ @Inject
+ ConsumerTemplate consumerTemplate;
+
+ @Inject
+ StorageProducer sp;
+
+ @Produces
+ @ApplicationScoped
+ @Unremovable
+ @Named(COMPONENT_GOOGLE_STORAGE)
+ GoogleCloudStorageComponent produceComponent() throws IOException {
+ GoogleCloudStorageComponent gsc = new GoogleCloudStorageComponent();
+ if (!GoogleStorageHelper.isRealAccount()) {
+ String port = System.getProperty(GoogleStorageResource.PARAM_PORT);
+ gsc.getConfiguration().setStorageClient(StorageOptions.newBuilder()
+ .setHost("http://localhost:" + port)
+ .setProjectId("dummy-project-for-testing")
+ .build()
+ .getService());
+ } else {
+ gsc.getConfiguration().setStorageClient(sp.storage());
+ }
+ return gsc;
+ }
+
+ @Path("/operation")
+ @POST
+ @Consumes(MediaType.APPLICATION_JSON)
+ @Produces(MediaType.TEXT_PLAIN)
+ public String operation(Map<String, Object> parameters,
+ @QueryParam(QUERY_OPERATION) String operation,
+ @QueryParam(QUERY_BUCKET) String bucketName) throws Exception {
+ GoogleCloudStorageOperations op = GoogleCloudStorageOperations.valueOf(operation);
+ String url = getBaseUrl(bucketName, "operation=" + op.toString());
+ final Object response = producerTemplate.requestBodyAndHeaders(url, null, parameters, Object.class);
+ if (response instanceof Blob) {
+ return new String(((Blob) response).getContent());
+ }
+ if (response instanceof CopyWriter) {
+ return new String(((CopyWriter) response).getResult().getContent());
+ }
+ if (response instanceof List) {
+ List l = (List) response;
+ return (String) l.stream().map(o -> {
+ if (o instanceof Bucket) {
+ return ((Bucket) o).getName();
+ }
+ if (o instanceof Blob) {
+ return ((Blob) o).getName();
+ }
+ return "null";
+ }).collect(Collectors.joining(","));
+ }
+ return String.valueOf(response);
+ }
+
+ @Path("/putObject")
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ @Produces(MediaType.TEXT_PLAIN)
+ public Response putObject(String body,
+ @QueryParam(QUERY_BUCKET) String bucketName,
+ @QueryParam(QUERY_OBJECT_NAME) String objectName) throws Exception {
+ String url = getBaseUrl(bucketName, "autoCreateBucket=true");
+ final Blob response = producerTemplate.requestBodyAndHeader(url,
+ body,
+ GoogleCloudStorageConstants.OBJECT_NAME, objectName, Blob.class);
+ return Response
+ .created(new URI("https://camel.apache.org/"))
+ .entity(response.getName())
+ .build();
+ }
+
+ @Path("/startPolling")
+ @POST
+ public void startPolling(@QueryParam(QUERY_BUCKET) String bucketName,
+ @QueryParam(QUERY_POLLING_ACTION) String pollingAction,
+ @QueryParam(QUERY_DESTINATION_BUCKET) String destinationBucket) {
+ // use another thread for polling consumer to demonstrate that we can wait before
+ // the message is sent to the queue
+ Executors.newSingleThreadExecutor().execute(() -> {
Review comment:
I used a route builder to poll into direct and then receive message from it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622444133
##########
File path: docs/modules/ROOT/pages/reference/extensions/google-storage.adoc
##########
@@ -0,0 +1,60 @@
+// Do not edit directly!
+// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
+= Google Storage
+:linkattrs:
+:cq-artifact-id: camel-quarkus-google-storage
+:cq-native-supported: true
+:cq-status: Stable
+:cq-status-deprecation: Stable
+:cq-description: Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+:cq-deprecated: false
+:cq-jvm-since: 1.9.0
+:cq-native-since: 1.9.0
+
+[.badges]
+[.badge-key]##JVM since##[.badge-supported]##1.9.0## [.badge-key]##Native since##[.badge-supported]##1.9.0##
+
+Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+
+== What's inside
+
+* xref:{cq-camel-components}::google-storage-component.adoc[Google Storage component], URI syntax: `google-storage:bucketName`
+
+Please refer to the above link for usage and configuration details.
+
+== Maven coordinates
+
+https://code.quarkus.io/?extension-search=camel-quarkus-google-storage[Create a new project with this extension on code.quarkus.io, window="_blank"]
+
+Or add the coordinates to your existing project:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel.quarkus</groupId>
+ <artifactId>camel-quarkus-google-storage</artifactId>
+</dependency>
+----
+
+Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.
+
+== SSL in native mode
+
+This extension auto-enables SSL support in native mode. Hence you do not need to add
+`quarkus.ssl.native=true` to your `application.properties` yourself. See also
+https://quarkus.io/guides/native-and-ssl[Quarkus SSL guide].
+
+== Additional Camel Quarkus configuration
+
+There are two different configuration approaches:
+
+* Google storage client can be defined via quarkus properties leveraging the Quarkiverse - Google Cloud Services - Storage (see https://github.com/quarkiverse/quarkus-google-cloud-services/blob/main/storage[extension git page]).
+Camel will autowire client into the Google-storage component.
+This configuration allows definition of only one storage client, therefore it isn't possible to define several different endpoints, which run together.
+
+* Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint.
Review comment:
I would stress the point here like (if I understand correctly):
When multiple storage clients are needed, quarkus properties should not be used. Instead, one or more storage client bean need to be registered in the registry (e.g. via a CDI bean producer). Each storage client bean could then be referenced from distinct endpoints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] zbendhiba commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
zbendhiba commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r619193070
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String DEST_BUCKET = "camel_quarkus_test_dest_bucket";
+ private static final String TEST_BUCKET1 = "camel_quarkus_test_bucket1";
+ private static final String TEST_BUCKET2 = "camel_quarkus_test_bucket2";
+ private static final String TEST_BUCKET3 = "camel_quarkus_test_bucket3";
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @BeforeEach
+ public void beforeEach() {
+ RestAssured.given().get("/google-storage/loadComponent").then().statusCode(200);
Review comment:
@JiriOndrusek can you load the configured component while the app starts?
something [like that ](https://github.com/apache/camel-quarkus/blob/e39f071c41353787308f2d8484b102445890ce16/integration-tests/hazelcast/src/main/java/org/apache/camel/quarkus/component/hazelcast/it/HazelcastRoutes.java#L80)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621183601
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTestResource.java
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Collections;
+import java.util.Map;
+
+import io.quarkus.test.common.QuarkusTestResourceLifecycleManager;
+import org.apache.camel.util.CollectionHelper;
+import org.testcontainers.containers.GenericContainer;
+
+public class GoogleStorageTestResource implements QuarkusTestResourceLifecycleManager {
+
+ public static final int PORT = 4443;
+ public static final String CONTAINER_NAME = "fsouza/fake-gcs-server";
+
+ private GenericContainer<?> container;
+
+ @Override
+ public Map<String, String> start() {
+ //if there is a real configuration, don't start container
Review comment:
Ok, I'll look into it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r623269225
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTest.java
##########
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import io.quarkus.test.common.QuarkusTestResource;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.RestAssured;
+import io.restassured.http.ContentType;
+import io.restassured.response.ValidatableResponse;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.apache.camel.quarkus.test.mock.backend.MockBackendUtils;
+import org.apache.camel.util.CollectionHelper;
+import org.hamcrest.Matcher;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.DEST_BUCKET;
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.TEST_BUCKET1;
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.TEST_BUCKET2;
+import static org.apache.camel.quarkus.component.google.storage.it.GoogleStorageResource.TEST_BUCKET3;
+import static org.hamcrest.Matchers.both;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.startsWith;
+
+@QuarkusTest
+@QuarkusTestResource(GoogleStorageTestResource.class)
+class GoogleStorageTest {
+
+ private static final String FILE_NAME_007 = "file007";
+ private static final String FILE_NAME_006 = "file006";
+
+ @AfterEach
+ public void afterEach() {
+ //clean after test (only in real environment)
+ if (!GoogleStorageHelper.usingMockBackend()) {
+ RestAssured.given()
+ .get("/google-storage/deleteBuckets")
+ .then()
+ .statusCode(200);
+ }
+ }
+
+ @Test
+ public void testConsumer() throws InterruptedException {
+ try {
+ //producer - putObject
+ putObject("Sheldon", TEST_BUCKET3, FILE_NAME_007);
+
+ //get result from direct (for pooling) with timeout
+ RestAssured.given()
+ .post("/google-storage/getFromDirect")
+ .then()
+ .statusCode(200)
+ .body(is("Sheldon"));
+
+ //producer - getObject
+ executeOperation(DEST_BUCKET, GoogleCloudStorageOperations.getObject,
+ Collections.singletonMap(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007),
+ is("Sheldon"));
+
+ } finally {
+ //stop route to allow bucket deletion without errors in real environment
+ if (!GoogleStorageHelper.usingMockBackend()) {
+ RestAssured.given()
+ .get("/google-storage/stopRoute")
+ .then()
+ .statusCode(200);
+ }
+ }
+ }
+
+ @Test
+ public void testProducer() {
+ //delete existing buckets t - only on real account - Deleting buckets is not (yet) supported by fsouza/fake-gcs-server.
+ if (!MockBackendUtils.startMockBackend()) {
+ String buckets = executeOperation(GoogleCloudStorageOperations.listBuckets, Collections.emptyMap(),
+ null);
+ List<String> bucketsToDelete = Arrays.stream(buckets.split(","))
+ .filter(b -> b.equals(TEST_BUCKET1) || b.equals(TEST_BUCKET2))
+ .collect(Collectors.toList());
+ if (!bucketsToDelete.isEmpty()) {
+ bucketsToDelete.forEach(
+ b -> executeOperation(b, GoogleCloudStorageOperations.deleteBucket, Collections.emptyMap(),
+ is(Boolean.toString(true))));
+ }
+ }
+
+ //create object in testBucket
+ putObject("Sheldon", TEST_BUCKET1, FILE_NAME_007);
+
+ putObject("Irma", TEST_BUCKET2, FILE_NAME_006);
+
+ //copy object to test_bucket2
+ executeOperation(GoogleCloudStorageOperations.copyObject,
+ CollectionHelper.mapOf(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007,
+ GoogleCloudStorageConstants.DESTINATION_BUCKET_NAME, TEST_BUCKET2,
+ GoogleCloudStorageConstants.DESTINATION_OBJECT_NAME, FILE_NAME_007 + "_copy"),
+ is("Sheldon"));
+
+ //GetObject
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.getObject,
+ Collections.singletonMap(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007 + "_copy"),
+ is("Sheldon"));
+
+ //list buckets
+ executeOperation(GoogleCloudStorageOperations.listBuckets, Collections.emptyMap(),
+ both(containsString(TEST_BUCKET1)).and(containsString(TEST_BUCKET2)));
+
+ //deleteObject
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.deleteObject,
+ CollectionHelper.mapOf(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_006),
+ is(Boolean.toString(true)));
+
+ //ListObjects
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.listObjects, Collections.emptyMap(),
+ containsString(FILE_NAME_007 + "_copy"));
+
+ //CreateDownloadLink - requires authentication
+ if (!GoogleStorageHelper.usingMockBackend()) {
+ executeOperation(TEST_BUCKET2, GoogleCloudStorageOperations.createDownloadLink,
+ Collections.singletonMap(GoogleCloudStorageConstants.OBJECT_NAME, FILE_NAME_007 + "_copy"),
+ startsWith("http"));
+ }
+ }
+
+ private void putObject(String sheldon, String testBucket1, String fileName007) {
Review comment:
Maybe the param names could be more generic here. testBucket ? filename ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621312380
##########
File path: extensions/google-storage/deployment/src/main/java/org/apache/camel/quarkus/component/google/storage/deployment/GoogleStorageProcessor.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.deployment;
+
+import com.google.api.client.googleapis.json.GoogleJsonError;
+import com.google.api.client.http.HttpHeaders;
+import com.google.api.client.json.GenericJson;
+import com.google.api.client.json.webtoken.JsonWebSignature;
+import com.google.api.client.json.webtoken.JsonWebToken;
+import com.google.api.client.util.GenericData;
+import com.google.api.services.storage.Storage;
+import com.google.api.services.storage.StorageRequest;
+import com.google.cloud.storage.Bucket;
+import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
+import io.quarkus.deployment.annotations.BuildProducer;
+import io.quarkus.deployment.annotations.BuildStep;
+import io.quarkus.deployment.builditem.CombinedIndexBuildItem;
+import io.quarkus.deployment.builditem.ExtensionSslNativeSupportBuildItem;
+import io.quarkus.deployment.builditem.FeatureBuildItem;
+import io.quarkus.deployment.builditem.IndexDependencyBuildItem;
+import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;
+import org.apache.camel.quarkus.component.google.storage.GoogleStorageEnforcer;
+import org.jboss.jandex.DotName;
+import org.jboss.jandex.IndexView;
+
+class GoogleStorageProcessor {
+
+ private static final String FEATURE = "camel-google-storage";
+
+ @BuildStep
+ FeatureBuildItem feature() {
+ return new FeatureBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ ExtensionSslNativeSupportBuildItem activateSslNativeSupport() {
+ return new ExtensionSslNativeSupportBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ public AdditionalBeanBuildItem storageEnforcer() {
+ return new AdditionalBeanBuildItem(GoogleStorageEnforcer.class);
+ }
+
+ @BuildStep
+ void registerForReflection(CombinedIndexBuildItem combinedIndex,
Review comment:
@jamesnetherton I'll report an issue to quarkiverse to upgrade their reference for `com.google.cloud:google-cloud-storage:jar`. Once this fix is provided, we can remove exclusion from camel component. But IMHO extension could be merged before this version is aligned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622792763
##########
File path: docs/modules/ROOT/pages/reference/extensions/google-storage.adoc
##########
@@ -0,0 +1,60 @@
+// Do not edit directly!
+// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
+= Google Storage
+:linkattrs:
+:cq-artifact-id: camel-quarkus-google-storage
+:cq-native-supported: true
+:cq-status: Stable
+:cq-status-deprecation: Stable
+:cq-description: Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+:cq-deprecated: false
+:cq-jvm-since: 1.9.0
+:cq-native-since: 1.9.0
+
+[.badges]
+[.badge-key]##JVM since##[.badge-supported]##1.9.0## [.badge-key]##Native since##[.badge-supported]##1.9.0##
+
+Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+
+== What's inside
+
+* xref:{cq-camel-components}::google-storage-component.adoc[Google Storage component], URI syntax: `google-storage:bucketName`
+
+Please refer to the above link for usage and configuration details.
+
+== Maven coordinates
+
+https://code.quarkus.io/?extension-search=camel-quarkus-google-storage[Create a new project with this extension on code.quarkus.io, window="_blank"]
+
+Or add the coordinates to your existing project:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel.quarkus</groupId>
+ <artifactId>camel-quarkus-google-storage</artifactId>
+</dependency>
+----
+
+Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.
+
+== SSL in native mode
+
+This extension auto-enables SSL support in native mode. Hence you do not need to add
+`quarkus.ssl.native=true` to your `application.properties` yourself. See also
+https://quarkus.io/guides/native-and-ssl[Quarkus SSL guide].
+
+== Additional Camel Quarkus configuration
+
+There are two different configuration approaches:
+
+* Google storage client can be defined via quarkus properties leveraging the Quarkiverse - Google Cloud Services - Storage (see https://github.com/quarkiverse/quarkus-google-cloud-services/blob/main/storage[extension git page]).
+Camel will autowire client into the Google-storage component.
+This configuration allows definition of only one storage client, therefore it isn't possible to define several different endpoints, which run together.
+
+* Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint.
Review comment:
Thanks for sharing. No big deal for the properties always needed as we'll have a separate issue.
However, I would stress that when reading the doc first, I did not understood why/when "Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint" is needed. Would you think changing the doc as below has value?
When multiple storage clients are needed, one or more storage client bean need to be registered in the registry (e.g. via a CDI bean producer). Each storage client bean could then be referenced from distinct endpoints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] aldettinger commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
aldettinger commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622445737
##########
File path: extensions/google-storage/runtime/src/main/java/org/apache/camel/quarkus/component/google/storage/GoogleStorageEnforcer.java
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage;
+
+import javax.enterprise.context.ApplicationScoped;
+import javax.enterprise.inject.Instance;
+import javax.inject.Inject;
+
+import com.google.cloud.storage.Storage;
+
+@ApplicationScoped
+public class GoogleStorageEnforcer {
+
+ //Quarkiverse StorageProducer creates storage based on properties.
+ //Because of https://github.com/apache/camel-quarkus/issues/1387, storage is removed and not applied to component.
+ //UnremovableBeanBuildItem does not fix this.
+ //Injecting all storages makes Quarkiverse StorageProducer to be used.
+ //We are injecting all storages, because client can define more storages. (so injecting only one could fail|)
Review comment:
small typo here `are need, It is possible could be used `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#issuecomment-828376502
PR is ready to review/merge.
Issue for update of `google-cloud-storage` in quarkiverse extension is reported as https://github.com/quarkiverse/quarkus-google-cloud-services/issues/113
Issue for camel-quarkus to remove exclusion is reported as https://github.com/apache/camel-quarkus/issues/2526
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] jamesnetherton commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
jamesnetherton commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r621163948
##########
File path: integration-tests/google-storage/src/test/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageTestResource.java
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.util.Collections;
+import java.util.Map;
+
+import io.quarkus.test.common.QuarkusTestResourceLifecycleManager;
+import org.apache.camel.util.CollectionHelper;
+import org.testcontainers.containers.GenericContainer;
+
+public class GoogleStorageTestResource implements QuarkusTestResourceLifecycleManager {
+
+ public static final int PORT = 4443;
+ public static final String CONTAINER_NAME = "fsouza/fake-gcs-server";
+
+ private GenericContainer<?> container;
+
+ @Override
+ public Map<String, String> start() {
+ //if there is a real configuration, don't start container
Review comment:
We should probably add a call to `MockBackendUtils`. See example from AWS tests here:
https://github.com/apache/camel-quarkus/blob/c50a7ae86148f34e286e218bba8fbff48b4c0d68/integration-tests-support/aws2/src/main/java/org/apache/camel/quarkus/test/support/aws2/Aws2TestResource.java#L47-L48
##########
File path: integration-tests/google-storage/src/main/java/org/apache/camel/quarkus/component/google/storage/it/GoogleStorageResource.java
##########
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.it;
+
+import java.io.IOException;
+import java.net.URI;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.stream.Collectors;
+
+import javax.enterprise.context.ApplicationScoped;
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+
+import com.google.cloud.storage.Blob;
+import com.google.cloud.storage.Bucket;
+import com.google.cloud.storage.CopyWriter;
+import com.google.cloud.storage.StorageOptions;
+import io.quarkiverse.googlecloudservices.storage.runtime.StorageProducer;
+import io.quarkus.arc.Unremovable;
+import org.apache.camel.ConsumerTemplate;
+import org.apache.camel.ProducerTemplate;
+import org.apache.camel.component.google.storage.GoogleCloudStorageComponent;
+import org.apache.camel.component.google.storage.GoogleCloudStorageConstants;
+import org.apache.camel.component.google.storage.GoogleCloudStorageOperations;
+import org.jboss.logging.Logger;
+
+@Path("/google-storage")
+@ApplicationScoped
+public class GoogleStorageResource {
+
+ public static final String DIRECT_POLLING = "direct:polling";
+
+ public static final String PARAM_PORT = "org.apache.camel.quarkus.component.googlr.storage.it.GoogleStorageClientProducer_port";
+
+ public static final String QUERY_OBJECT_NAME = "objectName";
+ public static final String QUERY_BUCKET = "bucketName";
+ public static final String QUERY_OPERATION = "operation";
+ public static final String QUERY_DESTINATION_BUCKET = "destinationBucket";
+ public static final String QUERY_DIRECT = "fromDirect";
+ public static final String QUERY_POLLING_ACTION = "pollingAction";
+
+ private static final Logger LOG = Logger.getLogger(GoogleStorageResource.class);
+ private static final String COMPONENT_GOOGLE_STORAGE = "google-storage";
+
+ @Inject
+ ProducerTemplate producerTemplate;
+
+ @Inject
+ ConsumerTemplate consumerTemplate;
+
+ @Inject
+ StorageProducer sp;
+
+ @Produces
+ @ApplicationScoped
+ @Unremovable
+ @Named(COMPONENT_GOOGLE_STORAGE)
+ GoogleCloudStorageComponent produceComponent() throws IOException {
+ GoogleCloudStorageComponent gsc = new GoogleCloudStorageComponent();
+ if (!GoogleStorageHelper.isRealAccount()) {
+ String port = System.getProperty(GoogleStorageResource.PARAM_PORT);
+ gsc.getConfiguration().setStorageClient(StorageOptions.newBuilder()
+ .setHost("http://localhost:" + port)
+ .setProjectId("dummy-project-for-testing")
+ .build()
+ .getService());
+ } else {
+ gsc.getConfiguration().setStorageClient(sp.storage());
+ }
+ return gsc;
+ }
+
+ @Path("/operation")
+ @POST
+ @Consumes(MediaType.APPLICATION_JSON)
+ @Produces(MediaType.TEXT_PLAIN)
+ public String operation(Map<String, Object> parameters,
+ @QueryParam(QUERY_OPERATION) String operation,
+ @QueryParam(QUERY_BUCKET) String bucketName) throws Exception {
+ GoogleCloudStorageOperations op = GoogleCloudStorageOperations.valueOf(operation);
+ String url = getBaseUrl(bucketName, "operation=" + op.toString());
+ final Object response = producerTemplate.requestBodyAndHeaders(url, null, parameters, Object.class);
+ if (response instanceof Blob) {
+ return new String(((Blob) response).getContent());
+ }
+ if (response instanceof CopyWriter) {
+ return new String(((CopyWriter) response).getResult().getContent());
+ }
+ if (response instanceof List) {
+ List l = (List) response;
+ return (String) l.stream().map(o -> {
+ if (o instanceof Bucket) {
+ return ((Bucket) o).getName();
+ }
+ if (o instanceof Blob) {
+ return ((Blob) o).getName();
+ }
+ return "null";
+ }).collect(Collectors.joining(","));
+ }
+ return String.valueOf(response);
+ }
+
+ @Path("/putObject")
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ @Produces(MediaType.TEXT_PLAIN)
+ public Response putObject(String body,
+ @QueryParam(QUERY_BUCKET) String bucketName,
+ @QueryParam(QUERY_OBJECT_NAME) String objectName) throws Exception {
+ String url = getBaseUrl(bucketName, "autoCreateBucket=true");
+ final Blob response = producerTemplate.requestBodyAndHeader(url,
+ body,
+ GoogleCloudStorageConstants.OBJECT_NAME, objectName, Blob.class);
+ return Response
+ .created(new URI("https://camel.apache.org/"))
+ .entity(response.getName())
+ .build();
+ }
+
+ @Path("/startPolling")
+ @POST
+ public void startPolling(@QueryParam(QUERY_BUCKET) String bucketName,
+ @QueryParam(QUERY_POLLING_ACTION) String pollingAction,
+ @QueryParam(QUERY_DESTINATION_BUCKET) String destinationBucket) {
+ // use another thread for polling consumer to demonstrate that we can wait before
+ // the message is sent to the queue
+ Executors.newSingleThreadExecutor().execute(() -> {
Review comment:
For this kind of thing, it's far easier to create a `RouteBuilder`, configure the `google-storage` consumer and direct any messages to a `seda` or `mock` endpoint which you can then retrieve via a `ConsumerTemplate` in the JAX-RS resource class.
It makes the test flow much better...
1. Create some storage object
2. Retrieve storage object
There's no need for executors, polling etc.
##########
File path: extensions/google-storage/deployment/src/main/java/org/apache/camel/quarkus/component/google/storage/deployment/GoogleStorageProcessor.java
##########
@@ -0,0 +1,87 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage.deployment;
+
+import com.google.api.client.googleapis.json.GoogleJsonError;
+import com.google.api.client.http.HttpHeaders;
+import com.google.api.client.json.GenericJson;
+import com.google.api.client.json.webtoken.JsonWebSignature;
+import com.google.api.client.json.webtoken.JsonWebToken;
+import com.google.api.client.util.GenericData;
+import com.google.api.services.storage.Storage;
+import com.google.api.services.storage.StorageRequest;
+import com.google.cloud.storage.Bucket;
+import io.quarkus.arc.deployment.AdditionalBeanBuildItem;
+import io.quarkus.deployment.annotations.BuildProducer;
+import io.quarkus.deployment.annotations.BuildStep;
+import io.quarkus.deployment.builditem.CombinedIndexBuildItem;
+import io.quarkus.deployment.builditem.ExtensionSslNativeSupportBuildItem;
+import io.quarkus.deployment.builditem.FeatureBuildItem;
+import io.quarkus.deployment.builditem.IndexDependencyBuildItem;
+import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;
+import org.apache.camel.quarkus.component.google.storage.GoogleStorageEnforcer;
+import org.jboss.jandex.DotName;
+import org.jboss.jandex.IndexView;
+
+class GoogleStorageProcessor {
+
+ private static final String FEATURE = "camel-google-storage";
+
+ @BuildStep
+ FeatureBuildItem feature() {
+ return new FeatureBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ ExtensionSslNativeSupportBuildItem activateSslNativeSupport() {
+ return new ExtensionSslNativeSupportBuildItem(FEATURE);
+ }
+
+ @BuildStep
+ public AdditionalBeanBuildItem storageEnforcer() {
+ return new AdditionalBeanBuildItem(GoogleStorageEnforcer.class);
+ }
+
+ @BuildStep
+ void registerForReflection(CombinedIndexBuildItem combinedIndex,
Review comment:
Is this needed anymore? I was hoping the quarkiverse extension would take care of this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r622807912
##########
File path: docs/modules/ROOT/pages/reference/extensions/google-storage.adoc
##########
@@ -0,0 +1,60 @@
+// Do not edit directly!
+// This file was generated by camel-quarkus-maven-plugin:update-extension-doc-page
+= Google Storage
+:linkattrs:
+:cq-artifact-id: camel-quarkus-google-storage
+:cq-native-supported: true
+:cq-status: Stable
+:cq-status-deprecation: Stable
+:cq-description: Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+:cq-deprecated: false
+:cq-jvm-since: 1.9.0
+:cq-native-since: 1.9.0
+
+[.badges]
+[.badge-key]##JVM since##[.badge-supported]##1.9.0## [.badge-key]##Native since##[.badge-supported]##1.9.0##
+
+Store and retrieve objects from Google Cloud Storage Service using the google-cloud-storage library.
+
+== What's inside
+
+* xref:{cq-camel-components}::google-storage-component.adoc[Google Storage component], URI syntax: `google-storage:bucketName`
+
+Please refer to the above link for usage and configuration details.
+
+== Maven coordinates
+
+https://code.quarkus.io/?extension-search=camel-quarkus-google-storage[Create a new project with this extension on code.quarkus.io, window="_blank"]
+
+Or add the coordinates to your existing project:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel.quarkus</groupId>
+ <artifactId>camel-quarkus-google-storage</artifactId>
+</dependency>
+----
+
+Check the xref:user-guide/index.adoc[User guide] for more information about writing Camel Quarkus applications.
+
+== SSL in native mode
+
+This extension auto-enables SSL support in native mode. Hence you do not need to add
+`quarkus.ssl.native=true` to your `application.properties` yourself. See also
+https://quarkus.io/guides/native-and-ssl[Quarkus SSL guide].
+
+== Additional Camel Quarkus configuration
+
+There are two different configuration approaches:
+
+* Google storage client can be defined via quarkus properties leveraging the Quarkiverse - Google Cloud Services - Storage (see https://github.com/quarkiverse/quarkus-google-cloud-services/blob/main/storage[extension git page]).
+Camel will autowire client into the Google-storage component.
+This configuration allows definition of only one storage client, therefore it isn't possible to define several different endpoints, which run together.
+
+* Provide client/clients for camel registry (e.g. CDI producer/bean) and reference them from endpoint.
Review comment:
I'll rephrase the doc to provide a better explanation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-quarkus] JiriOndrusek commented on a change in pull request #2505: Google Storage support #2421
Posted by GitBox <gi...@apache.org>.
JiriOndrusek commented on a change in pull request #2505:
URL: https://github.com/apache/camel-quarkus/pull/2505#discussion_r618974878
##########
File path: extensions/google-storage/runtime/src/main/java/org/apache/camel/quarkus/component/google/storage/UrlFetchTransportSubstitute.java
##########
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.quarkus.component.google.storage;
+
+import java.io.IOException;
+
+import com.google.api.client.extensions.appengine.http.UrlFetchTransport;
+import com.oracle.svm.core.annotate.Substitute;
+import com.oracle.svm.core.annotate.TargetClass;
+
+@TargetClass(value = UrlFetchTransport.class)
+public final class UrlFetchTransportSubstitute {
+
+ @Substitute
+ protected Object buildRequest(String method, String url) throws IOException {
+ throw new IllegalStateException("Not used by camel extension");
Review comment:
You are right, I'll fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org