You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by da...@apache.org on 2017/03/14 13:31:54 UTC
[2/3] camel git commit: CAMEL-11004: camel-connector - Allow to
define what the data type is as output/input.
CAMEL-11004: camel-connector - Allow to define what the data type is as output/input.
And polished a bit some other minor code.
Project: http://git-wip-us.apache.org/repos/asf/camel/repo
Commit: http://git-wip-us.apache.org/repos/asf/camel/commit/a57bc99e
Tree: http://git-wip-us.apache.org/repos/asf/camel/tree/a57bc99e
Diff: http://git-wip-us.apache.org/repos/asf/camel/diff/a57bc99e
Branch: refs/heads/master
Commit: a57bc99eb37ace23a484f3d062311be6a5a523ed
Parents: 910b678
Author: Claus Ibsen <da...@apache.org>
Authored: Tue Mar 14 14:12:32 2017 +0100
Committer: Claus Ibsen <da...@apache.org>
Committed: Tue Mar 14 14:14:20 2017 +0100
----------------------------------------------------------------------
.../java/org/apache/camel/CamelContext.java | 2 +-
.../src/main/docs/connector-component.adoc | 28 ++++++-
.../component/connector/ConnectorModel.java | 46 +++++++++++
.../camel/component/connector/DataType.java | 81 ++++++++++++++++++++
.../connector/DefaultConnectorComponent.java | 14 +++-
.../connector/DefaultConnectorEndpoint.java | 17 +++-
.../src/main/resources/camel-connector.json | 2 +
.../src/main/resources/camel-connector.json | 2 +
.../src/main/resources/camel-connector.json | 2 +
.../src/main/resources/camel-connector.json | 2 +
.../src/main/resources/camel-connector.json | 2 +
11 files changed, 191 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/camel-core/src/main/java/org/apache/camel/CamelContext.java
----------------------------------------------------------------------
diff --git a/camel-core/src/main/java/org/apache/camel/CamelContext.java b/camel-core/src/main/java/org/apache/camel/CamelContext.java
index aadd886..a8bc8ef 100644
--- a/camel-core/src/main/java/org/apache/camel/CamelContext.java
+++ b/camel-core/src/main/java/org/apache/camel/CamelContext.java
@@ -1089,7 +1089,7 @@ public interface CamelContext extends SuspendableService, RuntimeConfiguration {
* See this FAQ before use: <a href="http://camel.apache.org/why-does-camel-use-too-many-threads-with-producertemplate.html">
* Why does Camel use too many threads with ProducerTemplate?</a>
* <p/>
- * <b>Important:</b> Make sure to call {@link org.apache.camel.ProducerTemplate#stop()} when you are done using the template,
+ * <b>Important:</b> Make sure to call {@link org.apache.camel.FluentProducerTemplate#stop()} when you are done using the template,
* to clean up any resources.
* <p/>
* Will use cache size defined in Camel property with key {@link Exchange#MAXIMUM_CACHE_POOL_SIZE}.
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/camel-connector/src/main/docs/connector-component.adoc
----------------------------------------------------------------------
diff --git a/connectors/camel-connector/src/main/docs/connector-component.adoc b/connectors/camel-connector/src/main/docs/connector-component.adoc
index be13741..27e3504 100644
--- a/connectors/camel-connector/src/main/docs/connector-component.adoc
+++ b/connectors/camel-connector/src/main/docs/connector-component.adoc
@@ -47,6 +47,31 @@ You can also copy the existing examples from the `connectors` directory where th
You can find an example using these connectors in the `foo-bar-wine-example` in the `connectors` directory.
+### Input and Output Data Type
+
+Every connector *must* define which input and output data type are in use.
+
+The following data types are in use:
+
+[width="100%",cols="2m,8",options="header"]
+|=========================================
+| Data Type | Description
+| none | No data
+| any | Supports any kind of data. You can also use `*` instead of `any`
+| java | Java data. An optional sub type can define the fully qualified class name such as `java:com.foo.MyCustomer`.
+| text | Text based data
+| xml | XML based data. An option sub type can define the XML namespace of the XML root element.
+| json | JSon based data. An option sub type can define the fully qualified class name of a Java POJO that maps to this JSon structure.
+|=========================================
+
+For example to accept any incoming data type and output Java as `com.foo.MyCustomer` you would
+configure the the `camel-connector.json` file:
+
+```
+ "inputDataType": "any",
+ "outputDataType": "java:com.foo.MyCustomer",
+```
+
### The connectors schema file
A connector has a schema file `camel-connector.json` located in `src/main/resources` directory.
@@ -56,4 +81,5 @@ This schema holds the information where you can pre-configure and specify which
The options the connector can provide is a limited set of all the existing options that comes from the Camel component
its based upon. Each option can then also be pre-configured with a default-value.
-To understand this schema file, its easier to study those existing connectors from the `connectors` directory.
\ No newline at end of file
+To understand this schema file, its easier to study those existing connectors from the `connectors` directory.
+
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/ConnectorModel.java
----------------------------------------------------------------------
diff --git a/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/ConnectorModel.java b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/ConnectorModel.java
index c5ef6ef..0b2b6b2 100644
--- a/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/ConnectorModel.java
+++ b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/ConnectorModel.java
@@ -44,6 +44,8 @@ final class ConnectorModel {
private static final Pattern JAVA_TYPE_PATTERN = Pattern.compile("\"javaType\"\\s?:\\s?\"([\\w|.]+)\".*");
private static final Pattern BASE_JAVA_TYPE_PATTERN = Pattern.compile("\"baseJavaType\"\\s?:\\s?\"([\\w|.]+)\".*");
private static final Pattern BASE_SCHEME_PATTERN = Pattern.compile("\"baseScheme\"\\s?:\\s?\"([\\w|.]+)\".*");
+ private static final Pattern INPUT_DATA_TYPE_PATTERN = Pattern.compile("\"inputDataType\"\\s?:\\s?\"(\\*|[\\w|.:]+)\".*");
+ private static final Pattern OUTPUT_DATA_TYPE_PATTERN = Pattern.compile("\"outputDataType\"\\s?:\\s?\"([\\w|.:]+)\".*");
private final String componentName;
private final String className;
@@ -53,6 +55,8 @@ final class ConnectorModel {
private String baseJavaType;
private String connectorJSon;
private String connectorName;
+ private DataType inputDataType;
+ private DataType outputDataType;
private Map<String, String> defaultComponentOptions;
private Map<String, String> defaultEndpointOptions;
@@ -118,6 +122,26 @@ final class ConnectorModel {
return defaultEndpointOptions;
}
+ public DataType getInputDataType() {
+ if (inputDataType == null) {
+ String line = extractInputDataType(lines.get());
+ if (line != null) {
+ inputDataType = new DataType(line);
+ }
+ }
+ return inputDataType;
+ }
+
+ public DataType getOutputDataType() {
+ if (outputDataType == null) {
+ String line = extractOutputDataType(lines.get());
+ if (line != null) {
+ outputDataType = new DataType(line);
+ }
+ }
+ return outputDataType;
+ }
+
// ***************************************
// Helpers
// ***************************************
@@ -210,6 +234,28 @@ final class ConnectorModel {
return null;
}
+ private static String extractInputDataType(List<String> json) {
+ for (String line : json) {
+ line = line.trim();
+ Matcher matcher = INPUT_DATA_TYPE_PATTERN.matcher(line);
+ if (matcher.matches()) {
+ return matcher.group(1);
+ }
+ }
+ return null;
+ }
+
+ private static String extractOutputDataType(List<String> json) {
+ for (String line : json) {
+ line = line.trim();
+ Matcher matcher = OUTPUT_DATA_TYPE_PATTERN.matcher(line);
+ if (matcher.matches()) {
+ return matcher.group(1);
+ }
+ }
+ return null;
+ }
+
private Map<String, String> extractComponentDefaultValues(List<String> lines) {
Map<String, String> answer = new LinkedHashMap<>();
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DataType.java
----------------------------------------------------------------------
diff --git a/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DataType.java b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DataType.java
new file mode 100644
index 0000000..7c9650b
--- /dev/null
+++ b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DataType.java
@@ -0,0 +1,81 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.connector;
+
+/**
+ * Data types supported by Camel connectors.
+ * <p/>
+ * A connector is more strict that a regular Camel component and as such the connector
+ * is limited to supporting one data type as input and output.
+ */
+public final class DataType {
+
+ /**
+ * The supported data types.
+ */
+ public enum Type {
+ none, any, java, text, xml, json;
+ }
+
+ private final Type type;
+ private final String subType;
+
+ DataType(String text) {
+ String[] parts = text.split(":");
+
+ String name = parts[0].toLowerCase();
+ // allow * as shorthand for any kind
+ if ("*".equals(name)) {
+ name = "any";
+ }
+
+ type = Type.valueOf(name);
+ if (parts.length == 2) {
+ subType = parts[1];
+ } else {
+ subType = null;
+ }
+ }
+
+ DataType(Type type, String subType) {
+ this.type = type;
+ this.subType = subType;
+ }
+
+ /**
+ * The type one of <tt>none</tt>, <tt>any</tt> (you can also use <tt>*</tt> as any), <tt>java</tt>, <tt>text</tt>, <tt>xml</tt>, or <tt>json</tt>.
+ */
+ public Type getType() {
+ return type;
+ }
+
+ /**
+ * Optional sub type to qualify the data type such as a java fully qualified class name, or a xml namespace etc
+ */
+ public String getSubType() {
+ return subType;
+ }
+
+ @Override
+ public String toString() {
+ if (subType != null) {
+ return type.name() + ":" + subType;
+ } else {
+ return type.name();
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorComponent.java
----------------------------------------------------------------------
diff --git a/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorComponent.java b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorComponent.java
index e8506e9..536a224 100644
--- a/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorComponent.java
+++ b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorComponent.java
@@ -63,10 +63,8 @@ public abstract class DefaultConnectorComponent extends DefaultComponent impleme
String delegateUri = createEndpointUri(scheme, options);
log.debug("Connector resolved: {} -> {}", uri, delegateUri);
-
Endpoint delegate = getCamelContext().getEndpoint(delegateUri);
-
- return new DefaultConnectorEndpoint(uri, this, delegate);
+ return new DefaultConnectorEndpoint(uri, this, delegate, model.getInputDataType(), model.getOutputDataType());
}
@Override
@@ -143,6 +141,15 @@ public abstract class DefaultConnectorComponent extends DefaultComponent impleme
@Override
protected void doStart() throws Exception {
+ // lets enforce that every connector must have an input and output data type
+
+ if (model.getInputDataType() == null) {
+ throw new IllegalArgumentException("Camel connector must have inputDataType defined in camel-connector.json file");
+ }
+ if (model.getOutputDataType() == null) {
+ throw new IllegalArgumentException("Camel connector must have outputDataType defined in camel-connector.json file");
+ }
+
// it may be a custom component so we need to register this in the camel catalog also
String scheme = model.getBaseScheme();
if (!catalog.findComponentNames().contains(scheme)) {
@@ -170,7 +177,6 @@ public abstract class DefaultConnectorComponent extends DefaultComponent impleme
}
log.debug("Starting connector: {}", componentName);
-
super.doStart();
}
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorEndpoint.java
----------------------------------------------------------------------
diff --git a/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorEndpoint.java b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorEndpoint.java
index f8743f2..6550907 100644
--- a/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorEndpoint.java
+++ b/connectors/camel-connector/src/main/java/org/apache/camel/component/connector/DefaultConnectorEndpoint.java
@@ -31,10 +31,15 @@ import org.apache.camel.util.ServiceHelper;
public class DefaultConnectorEndpoint extends DefaultEndpoint implements DelegateEndpoint {
private final Endpoint endpoint;
+ private final DataType inputDataType;
+ private final DataType outputDataType;
- public DefaultConnectorEndpoint(String endpointUri, Component component, Endpoint endpoint) {
+ public DefaultConnectorEndpoint(String endpointUri, ConnectorComponent component, Endpoint endpoint,
+ DataType inputDataType, DataType outputDataType) {
super(endpointUri, component);
this.endpoint = endpoint;
+ this.inputDataType = inputDataType;
+ this.outputDataType = outputDataType;
}
@Override
@@ -62,6 +67,16 @@ public class DefaultConnectorEndpoint extends DefaultEndpoint implements Delegat
return endpoint.getEndpointUri();
}
+ @ManagedAttribute(description = "Input data type")
+ public DataType getInputDataType() {
+ return inputDataType;
+ }
+
+ @ManagedAttribute(description = "Output data type")
+ public DataType getOutputDataType() {
+ return outputDataType;
+ }
+
@Override
protected void doStart() throws Exception {
super.doStart();
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/examples/bar-connector/src/main/resources/camel-connector.json
----------------------------------------------------------------------
diff --git a/connectors/examples/bar-connector/src/main/resources/camel-connector.json b/connectors/examples/bar-connector/src/main/resources/camel-connector.json
index 46a1740..4c1acd2 100644
--- a/connectors/examples/bar-connector/src/main/resources/camel-connector.json
+++ b/connectors/examples/bar-connector/src/main/resources/camel-connector.json
@@ -13,6 +13,8 @@
"description": "To order drinks from the bar",
"labels": [ "bar" ],
"pattern": "To",
+ "inputDataType": "*",
+ "outputDataType": "text",
"endpointOptions": [ "drink", "amount", "celebrity" ],
"endpointValues": {
"amount": 2
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/examples/foo-connector/src/main/resources/camel-connector.json
----------------------------------------------------------------------
diff --git a/connectors/examples/foo-connector/src/main/resources/camel-connector.json b/connectors/examples/foo-connector/src/main/resources/camel-connector.json
index deeb90d..13f196f 100644
--- a/connectors/examples/foo-connector/src/main/resources/camel-connector.json
+++ b/connectors/examples/foo-connector/src/main/resources/camel-connector.json
@@ -13,6 +13,8 @@
"description": "Something cool",
"labels": [ "foo", "timer" ],
"pattern": "From",
+ "inputDataType": "none",
+ "outputDataType": "none",
"endpointOptions": [ "timerName", "period", "repeatCount" ],
"endpointValues": {
"fixedRate": true,
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/examples/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json
----------------------------------------------------------------------
diff --git a/connectors/examples/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json b/connectors/examples/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json
index ff515ce..5153000 100644
--- a/connectors/examples/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json
+++ b/connectors/examples/salesforce-upsert-contact-connector/src/main/resources/camel-connector.json
@@ -12,6 +12,8 @@
"description": "Create or update Salesforce Contact SObject",
"labels": [ "salesforce" ],
"pattern": "To",
+ "inputDataType": "java:org.foo.salesforce.upsert.contact.Contact",
+ "outputDataType": "java:org.apache.camel.component.salesforce.api.dto.CreateSObjectResult",
"componentOptions" : [ "loginUrl", "clientId", "clientSecret", "refreshToken" ],
"endpointValues" : {
"operationName": "upsertSObject"
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/examples/twitter-mention-connector/src/main/resources/camel-connector.json
----------------------------------------------------------------------
diff --git a/connectors/examples/twitter-mention-connector/src/main/resources/camel-connector.json b/connectors/examples/twitter-mention-connector/src/main/resources/camel-connector.json
index 30523cb..53087f5 100644
--- a/connectors/examples/twitter-mention-connector/src/main/resources/camel-connector.json
+++ b/connectors/examples/twitter-mention-connector/src/main/resources/camel-connector.json
@@ -12,6 +12,8 @@
"description": "Connection from twitter when anyone mention you",
"labels": [ "twitter" ],
"pattern": "From",
+ "inputDataType": "none",
+ "outputDataType": "java:twitter4j.Status",
"componentOptions" : [ "accessToken", "accessTokenSecret", "consumerKey", "consumerSecret" ],
"endpointValues" : {
"kind" : "timeline/mentions"
http://git-wip-us.apache.org/repos/asf/camel/blob/a57bc99e/connectors/examples/wine-connector/src/main/resources/camel-connector.json
----------------------------------------------------------------------
diff --git a/connectors/examples/wine-connector/src/main/resources/camel-connector.json b/connectors/examples/wine-connector/src/main/resources/camel-connector.json
index 6c14a9f..683a1ab 100644
--- a/connectors/examples/wine-connector/src/main/resources/camel-connector.json
+++ b/connectors/examples/wine-connector/src/main/resources/camel-connector.json
@@ -13,6 +13,8 @@
"description": "To order wine from the bar",
"labels": [ "bar" ],
"pattern": "To",
+ "inputDataType": "*",
+ "outputDataType": "text",
"endpointOptions": [ "drink", "amount" ],
"endpointValues": {
"drink": "Wine"