You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2020/11/21 08:54:19 UTC
[GitHub] [camel] davsclaus commented on a change in pull request #4561: [CAMEL-15804] - DataSonnet Expression Language Support
davsclaus commented on a change in pull request #4561:
URL: https://github.com/apache/camel/pull/4561#discussion_r528168958
##########
File path: components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
##########
@@ -0,0 +1,193 @@
+[[datasonnet-language]]
+= DataSonnet Language
+:docTitle: DataSonnet
+:artifactId: camel-datasonnet
+:description: To use DataSonnet scripts in Camel expressions or predicates.
+:since: 3.7
+:supportLevel: Preview
+include::{cq-version}@camel-quarkus:ROOT:partial$reference/languages/datasonnet.adoc[opts=optional]
+
+*Since Camel {since}*
+
+Camel supports https://datasonnet.com/[DataSonnet] transformations to allow an Expression or Predicate to be used in the Java DSL or xref:manual::xml-configuration.adoc[XML
+Configuration].
+
+To use a DataSonnet expression use the following Java code:
+[source,java]
+---------------------------------------
+... datasonnet("someDSExpression") ...
+---------------------------------------
+
+== Example
+
+Here is a simple example using a DataSonnet expression as a predicate in a Message Filter:
+
+[source,java]
+------------------------------------------------------------------------------------------------
+// lets route if a line item is over $100
+from("queue:foo")
+ .filter(datasonnet("ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null"))
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet>ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null</datasonnet>
Review comment:
In XML I think > needs to be escaped
##########
File path: components/camel-datasonnet/pom.xml
##########
@@ -0,0 +1,134 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>components</artifactId>
+ <version>3.7.0-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>camel-datasonnet</artifactId>
+ <packaging>jar</packaging>
+
+ <name>Camel :: DataSonnet</name>
+ <description>Camel DataSonnet support</description>
+
+ <properties>
+ <maven.compiler.source>1.8</maven.compiler.source>
+ <maven.compiler.target>1.8</maven.compiler.target>
+ </properties>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>net.alchim31.maven</groupId>
+ <artifactId>scala-maven-plugin</artifactId>
+ <version>4.4.0</version>
+ <executions>
+ <execution>
+ <id>scala-compile-first</id>
+ <phase>process-resources</phase>
+ <goals>
+ <goal>add-source</goal>
+ <goal>compile</goal>
+ </goals>
+ </execution>
+ <execution>
+ <id>scala-test-compile</id>
+ <phase>process-test-resources</phase>
+ <goals>
+ <goal>testCompile</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <additionalClasspathElements>
+ <additionalClasspathElement>${project.basedir}/src/test/resources/dslibs.jar</additionalClasspathElement>
+ </additionalClasspathElements>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.scala-lang</groupId>
+ <artifactId>scala-library</artifactId>
+ <version>${scala-datasonnet-version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-support</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.ms3-inc</groupId>
+ <artifactId>datasonnet-mapper</artifactId>
+ <version>${datasonnet-mapper-version}</version>
+ </dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>io.github.classgraph</groupId>
+ <artifactId>classgraph</artifactId>
+ <version>${classgraph-version}</version>
+ </dependency>
+
+ <!-- testing -->
+ <dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-test-spring-junit5</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.logging.log4j</groupId>
+ <artifactId>log4j-slf4j-impl</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.skyscreamer</groupId>
+ <artifactId>jsonassert</artifactId>
+ <version>${jsonassert-version}</version>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+
+ <repositories>
+ <repository>
+ <id>ms3-nexus</id>
Review comment:
I assume this 3rd party maven repo can be removed
##########
File path: components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
##########
@@ -0,0 +1,193 @@
+[[datasonnet-language]]
+= DataSonnet Language
+:docTitle: DataSonnet
+:artifactId: camel-datasonnet
+:description: To use DataSonnet scripts in Camel expressions or predicates.
+:since: 3.7
+:supportLevel: Preview
+include::{cq-version}@camel-quarkus:ROOT:partial$reference/languages/datasonnet.adoc[opts=optional]
+
+*Since Camel {since}*
+
+Camel supports https://datasonnet.com/[DataSonnet] transformations to allow an Expression or Predicate to be used in the Java DSL or xref:manual::xml-configuration.adoc[XML
+Configuration].
+
+To use a DataSonnet expression use the following Java code:
+[source,java]
+---------------------------------------
+... datasonnet("someDSExpression") ...
+---------------------------------------
+
+== Example
+
+Here is a simple example using a DataSonnet expression as a predicate in a Message Filter:
+
+[source,java]
+------------------------------------------------------------------------------------------------
+// lets route if a line item is over $100
+from("queue:foo")
+ .filter(datasonnet("ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null"))
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet>ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null</datasonnet>
+ <to uri="queue:bar"/>
+ </filter>
+</route>
+-----------------------------------------------------------------------------
+
+Here is an example of a simple DataSonnet expression as a transformation EIP. This example will transform an XML body with
+`lineItems` into JSON while filtering out lines that are under 100.
+
+[source,java]
+------------------------------------------------------------------------------------------------
+from("queue:foo")
+ .transform(datasonnet("ds.filter(body.lineItems, function(item) item > 100)", String.class)
+ .bodyMediaType("application/xml").outputMediaType("application/json")
+ )
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet bodyMediaType="application/xml" outputMediaType="application/json" resultTypeName="java.lang.String" >
+ ds.filter(body.lineItems, function(item) item > 100)
+ </datasonnet>
+ <to uri="queue:bar"/>
+ </filter>
+</route>
+-----------------------------------------------------------------------------
+
+== Setting result type
+
+The xref:datasonnet-language.adoc[DataSonnet] expression will return a `com.datasonnet.document.Document` by default. The
+document preserves the content type metadata along with the contents of the result of the transformation. In predicates,
+however, the Document will be automatically unwrapped and the boolean content will be returned. Similarly any times you
+want the content in a specific result type like a String. To do this you have to instruct the
+xref:datasonnet-language.adoc[DataSonnet] which result type to return.
+
+In Java DSL:
+
+[source,java]
+----
+datasonnet("body.foo", String.class)
+----
+
+In Spring DSL you use the *resultType* attribute to provide a fully
+qualified classname:
+
+[source,xml]
+----
+<datasonnet resultType="java.lang.String">body.foo</datasonnet>
+----
+
+If the expression results in an array, or an object, you can instruct the expression to return you `List.class`
+or `Map.class`, respectively. However, you must also set the output media type to `application/x-java-object`.
+
+NOTE: The default `Document` object is useful in situations where there are intermediate transformation steps, and so
+retaining the content metadata through a route execution is valuable.
+
+== Specifying Media Types
+
+Traditionally the input and output media types are specified through the
+https://datasonnet.s3-us-west-2.amazonaws.com/docs-ci/primary/master/datasonnet/1.0-SNAPSHOT/headers.html[DataSonnet Header]
+The xref:datasonnet-language.adoc[DataSonnet] expression provides convenience options for specifying the body and output
+media types without the need for a Header, this is useful if the transformation is a one-liner, for example.
+
+The DataSonnet expression will look for a body media type in the following order:
+
+1. If the body is a `Document` it will use the metadata in the object
+2. If the convenience bodyMediaType method was used, it will use its value
+3. A "CamelDatasonnetBodyMediaType" exchange property
+4. A "Content-Type" message header
+5. The DataSonnet Header payload media type directive
+6. `application/x-java-object`
+
+And for output media type:
+
+1. If the convenience outputMediaType method was used, it will use its value
+2. A "CamelDatasonnetOutputMediaType" exchange property
+3. A "CamelDatasonnetOutputMediaType" message header
+4. The DataSonnet Header output media type directive
+5. `application/x-java-object`
+
+== Functions
+
+Camel adds the following DataSonnet functions that can be used to access the
+exchange:
+
+[width="100%",cols="10%,10%,10%,70%",options="header",]
+|===
+|Function |Argument |Type |Description
+
+|cml.properties |key for property |String |To lookup a property using the
+xref:ROOT:properties-component.adoc[Properties] component (property placeholders).
+
+|cml.header |the header name |String |Will return the message header.
+
+|cml.exchangeProperty |key for property |String |Will return the exchange property.
+|===
+
+Here's an example showing some of these functions in use:
Review comment:
I wonder if we can make a better example as using datasonnet to set the body via a property is not something you would do. I assume you can use these functions in those previous example. For example the value of 100 can be a property
##########
File path: components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/CamelDatasonnetTest.java
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.language.datasonnet;
+
+import java.io.InputStream;
+import java.text.SimpleDateFormat;
+import java.util.Arrays;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.test.spring.junit5.CamelSpringTestSupport;
+import org.apache.commons.io.IOUtils;
+import org.junit.jupiter.api.Disabled;
+import org.junit.jupiter.api.Test;
+import org.skyscreamer.jsonassert.JSONAssert;
+import org.springframework.context.support.AbstractApplicationContext;
+import org.springframework.context.support.ClassPathXmlApplicationContext;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+public class CamelDatasonnetTest extends CamelSpringTestSupport {
+ private MockEndpoint mock;// = getMockEndpoint("mock:direct:end");
+
+ @Override
+ protected AbstractApplicationContext createApplicationContext() {
+ return new ClassPathXmlApplicationContext("org/apache/camel/language.datasonnet/camel-context.xml");
+ }
+
+ @Test
+ public void testTransform() throws Exception {
+ runCamelTest(loadResourceAsString("simpleMapping_payload.json"),
+ loadResourceAsString("simpleMapping_result.json"),
+ "direct:basicTransform");
+ }
+
+ @Test
+ public void testTransformXML() throws Exception {
+ runCamelTest(loadResourceAsString("payload.xml"),
+ loadResourceAsString("readXMLExtTest.json"),
+ "direct:transformXML");
+ }
+
+ @Test
+ public void testTransformCSV() throws Exception {
+ runCamelTest(loadResourceAsString("payload.csv"),
+ "{\"account\":\"123\"}",
+ "direct:transformCSV");
+ }
+
+ @Test
+ public void testDatasonnetScript() throws Exception {
+ runCamelTest(loadResourceAsString("simpleMapping_payload.json"),
+ loadResourceAsString("simpleMapping_result.json"),
+ "direct:datasonnetScript");
+ }
+
+ @Test
+ public void testNamedImports() throws Exception {
+ runCamelTest("{}",
+ loadResourceAsString("namedImports_result.json"),
+ "direct:namedImports");
+ }
+
+ // TODO: 9/8/20 need to pass result type param to language
+ @Disabled
+ @Test
+ public void testExpressionLanguage() throws Exception {
+ runCamelTest("World",
+ "{ \"test\":\"Hello, World\"}",
+ "direct:expressionLanguage");
+ }
+
+ @Test
+ public void testNullInput() throws Exception {
+ runCamelTest("",
+ "{ \"test\":\"Hello, World\"}",
+ "direct:nullInput");
+ runCamelTest(null,
+ "{ \"test\":\"Hello, World\"}",
+ "direct:nullInput");
+ }
+
+ @Test
+ public void testReadJava() throws Exception {
+ Gizmo theGizmo = new Gizmo();
+ theGizmo.setName("gizmo");
+ theGizmo.setQuantity(123);
+ theGizmo.setInStock(true);
+ theGizmo.setColors(Arrays.asList("red", "white", "blue"));
+
+ Manufacturer manufacturer = new Manufacturer();
+ manufacturer.setManufacturerName("ACME Corp.");
+ manufacturer.setManufacturerCode("ACME123");
+ theGizmo.setManufacturer(manufacturer);
+
+ SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd");
+ theGizmo.setDate(df.parse("2020-01-06"));
+
+ runCamelTest(theGizmo,
+ loadResourceAsString("javaTest.json"),
+ "direct:readJava");
+ }
+
+ @Test
+ public void testWriteJava() throws Exception {
+ Gizmo theGizmo = new Gizmo();
+ theGizmo.setName("gizmo");
+ theGizmo.setQuantity(123);
+ theGizmo.setInStock(true);
+ theGizmo.setColors(Arrays.asList("red", "white", "blue"));
+
+ Manufacturer manufacturer = new Manufacturer();
+ manufacturer.setManufacturerName("ACME Corp.");
+ manufacturer.setManufacturerCode("ACME123");
+ theGizmo.setManufacturer(manufacturer);
+
+ SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd");
+ theGizmo.setDate(df.parse("2020-01-06"));
+
+ String payload = loadResourceAsString("javaTest.json");
+
+ template.sendBody("direct:writeJava", payload);
+ mock = getMockEndpoint("mock:direct:end");
+ Exchange exchange = mock.assertExchangeReceived(mock.getReceivedCounter() - 1);
+ Object response = exchange.getIn().getBody();
+
+ assertEquals(theGizmo, response);
+ }
+
+ private void runCamelTest(Object payload, String expectedJson, String uri) throws Exception {
+ template.sendBody(uri, payload);
+ mock = getMockEndpoint("mock:direct:end");
+ Exchange exchange = mock.assertExchangeReceived(mock.getReceivedCounter() - 1);
+ String response = exchange.getIn().getBody().toString();
+ System.out.println("RESPONSE IS " + response);
Review comment:
Remove system out or do a LOG
##########
File path: components/camel-datasonnet/pom.xml
##########
@@ -0,0 +1,134 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>components</artifactId>
+ <version>3.7.0-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>camel-datasonnet</artifactId>
+ <packaging>jar</packaging>
+
+ <name>Camel :: DataSonnet</name>
+ <description>Camel DataSonnet support</description>
+
+ <properties>
Review comment:
Remove these as this is configured in some parent/root pom file
##########
File path: components/camel-datasonnet/pom.xml
##########
@@ -0,0 +1,134 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>components</artifactId>
+ <version>3.7.0-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>camel-datasonnet</artifactId>
+ <packaging>jar</packaging>
+
+ <name>Camel :: DataSonnet</name>
+ <description>Camel DataSonnet support</description>
+
+ <properties>
+ <maven.compiler.source>1.8</maven.compiler.source>
+ <maven.compiler.target>1.8</maven.compiler.target>
+ </properties>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>net.alchim31.maven</groupId>
Review comment:
Can we avoid having scala source code. We want everyone to be able to maintain the code and for that we prefer Java
##########
File path: components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
##########
@@ -0,0 +1,193 @@
+[[datasonnet-language]]
+= DataSonnet Language
+:docTitle: DataSonnet
+:artifactId: camel-datasonnet
+:description: To use DataSonnet scripts in Camel expressions or predicates.
+:since: 3.7
+:supportLevel: Preview
+include::{cq-version}@camel-quarkus:ROOT:partial$reference/languages/datasonnet.adoc[opts=optional]
+
+*Since Camel {since}*
+
+Camel supports https://datasonnet.com/[DataSonnet] transformations to allow an Expression or Predicate to be used in the Java DSL or xref:manual::xml-configuration.adoc[XML
+Configuration].
+
+To use a DataSonnet expression use the following Java code:
+[source,java]
+---------------------------------------
+... datasonnet("someDSExpression") ...
+---------------------------------------
+
+== Example
+
+Here is a simple example using a DataSonnet expression as a predicate in a Message Filter:
+
+[source,java]
+------------------------------------------------------------------------------------------------
+// lets route if a line item is over $100
+from("queue:foo")
+ .filter(datasonnet("ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null"))
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet>ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null</datasonnet>
+ <to uri="queue:bar"/>
+ </filter>
+</route>
+-----------------------------------------------------------------------------
+
+Here is an example of a simple DataSonnet expression as a transformation EIP. This example will transform an XML body with
+`lineItems` into JSON while filtering out lines that are under 100.
+
+[source,java]
+------------------------------------------------------------------------------------------------
+from("queue:foo")
+ .transform(datasonnet("ds.filter(body.lineItems, function(item) item > 100)", String.class)
+ .bodyMediaType("application/xml").outputMediaType("application/json")
+ )
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet bodyMediaType="application/xml" outputMediaType="application/json" resultTypeName="java.lang.String" >
+ ds.filter(body.lineItems, function(item) item > 100)
+ </datasonnet>
+ <to uri="queue:bar"/>
+ </filter>
+</route>
+-----------------------------------------------------------------------------
+
+== Setting result type
+
+The xref:datasonnet-language.adoc[DataSonnet] expression will return a `com.datasonnet.document.Document` by default. The
+document preserves the content type metadata along with the contents of the result of the transformation. In predicates,
+however, the Document will be automatically unwrapped and the boolean content will be returned. Similarly any times you
+want the content in a specific result type like a String. To do this you have to instruct the
+xref:datasonnet-language.adoc[DataSonnet] which result type to return.
+
+In Java DSL:
+
+[source,java]
+----
+datasonnet("body.foo", String.class)
+----
+
+In Spring DSL you use the *resultType* attribute to provide a fully
+qualified classname:
+
+[source,xml]
+----
+<datasonnet resultType="java.lang.String">body.foo</datasonnet>
+----
+
+If the expression results in an array, or an object, you can instruct the expression to return you `List.class`
+or `Map.class`, respectively. However, you must also set the output media type to `application/x-java-object`.
+
+NOTE: The default `Document` object is useful in situations where there are intermediate transformation steps, and so
+retaining the content metadata through a route execution is valuable.
+
+== Specifying Media Types
+
+Traditionally the input and output media types are specified through the
+https://datasonnet.s3-us-west-2.amazonaws.com/docs-ci/primary/master/datasonnet/1.0-SNAPSHOT/headers.html[DataSonnet Header]
+The xref:datasonnet-language.adoc[DataSonnet] expression provides convenience options for specifying the body and output
+media types without the need for a Header, this is useful if the transformation is a one-liner, for example.
+
+The DataSonnet expression will look for a body media type in the following order:
+
+1. If the body is a `Document` it will use the metadata in the object
+2. If the convenience bodyMediaType method was used, it will use its value
+3. A "CamelDatasonnetBodyMediaType" exchange property
+4. A "Content-Type" message header
+5. The DataSonnet Header payload media type directive
+6. `application/x-java-object`
+
+And for output media type:
+
+1. If the convenience outputMediaType method was used, it will use its value
+2. A "CamelDatasonnetOutputMediaType" exchange property
+3. A "CamelDatasonnetOutputMediaType" message header
+4. The DataSonnet Header output media type directive
+5. `application/x-java-object`
+
+== Functions
+
+Camel adds the following DataSonnet functions that can be used to access the
+exchange:
+
+[width="100%",cols="10%,10%,10%,70%",options="header",]
+|===
+|Function |Argument |Type |Description
+
+|cml.properties |key for property |String |To lookup a property using the
+xref:ROOT:properties-component.adoc[Properties] component (property placeholders).
+
+|cml.header |the header name |String |Will return the message header.
+
+|cml.exchangeProperty |key for property |String |Will return the exchange property.
+|===
+
+Here's an example showing some of these functions in use:
+
+[source,java]
+------------------------------------------------------------------------------------------------
+from("direct:in")
+ .setBody(datasonnet("cml.properties('foo')", String.class))
+ .to("mock:camel");
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="direct:in"/>
+ <setBody>
+ <datasonnet resultTypeName="java.lang.String">cml.properties('foo')</datasonnet>
+ </setBody>
+ <to uri="mock:camel"/>
+</route>
+-----------------------------------------------------------------------------
+
+== Loading script from external resource
+
+You can externalize the script and have Camel load it from a resource
+such as `"classpath:"`, `"file:"`, or `"http:"`. +
+This is done using the following syntax: `"resource:scheme:location"`,
+eg to refer to a file on the classpath you can do:
+
+[source,java]
+-------------------------------------------------------------------
+.setHeader("myHeader").datasonnet("resource:classpath:mydatasonnet.ds")
+-------------------------------------------------------------------
+
+== Dependencies
+
+To use scripting languages in your camel routes you need to add a
+dependency on *camel-datasonnet*.
+
+If you use Maven you could just add the following to your `pom.xml`,
+substituting the version number for the latest and greatest release (see
+the download page for the latest versions).
+
Review comment:
Very good documentation - thank you
##########
File path: components/camel-datasonnet/src/main/java/org/apache/camel/language/datasonnet/DatasonnetExpression.java
##########
@@ -0,0 +1,255 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.language.datasonnet;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.FileVisitResult;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.SimpleFileVisitor;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+import com.datasonnet.Mapper;
+import com.datasonnet.MapperBuilder;
+import com.datasonnet.document.DefaultDocument;
+import com.datasonnet.document.Document;
+import com.datasonnet.document.MediaType;
+import com.datasonnet.document.MediaTypes;
+import io.github.classgraph.ClassGraph;
+import io.github.classgraph.ScanResult;
+import org.apache.camel.Exchange;
+import org.apache.camel.Expression;
+import org.apache.camel.RuntimeExpressionException;
+import org.apache.camel.spi.ExpressionResultTypeAware;
+import org.apache.camel.support.ExchangeHelper;
+import org.apache.camel.support.ExpressionAdapter;
+import org.apache.camel.support.MessageHelper;
+import org.apache.commons.io.IOUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class DatasonnetExpression extends ExpressionAdapter implements ExpressionResultTypeAware {
+ private static final Logger LOGGER = LoggerFactory.getLogger(DatasonnetExpression.class);
+ private static final Map<String, String> CLASSPATH_IMPORTS = new HashMap<>();
+
+ static {
+ LOGGER.debug("One time classpath search...");
Review comment:
I would like to see this moved to the Language class, and do this in its doInit method.
##########
File path: components/camel-datasonnet/src/test/java/org/apache/camel/language/datasonnet/CamelDatasonnetTest.java
##########
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.language.datasonnet;
+
+import java.io.InputStream;
+import java.text.SimpleDateFormat;
+import java.util.Arrays;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.test.spring.junit5.CamelSpringTestSupport;
+import org.apache.commons.io.IOUtils;
+import org.junit.jupiter.api.Disabled;
+import org.junit.jupiter.api.Test;
+import org.skyscreamer.jsonassert.JSONAssert;
+import org.springframework.context.support.AbstractApplicationContext;
+import org.springframework.context.support.ClassPathXmlApplicationContext;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+public class CamelDatasonnetTest extends CamelSpringTestSupport {
+ private MockEndpoint mock;// = getMockEndpoint("mock:direct:end");
+
+ @Override
+ protected AbstractApplicationContext createApplicationContext() {
+ return new ClassPathXmlApplicationContext("org/apache/camel/language.datasonnet/camel-context.xml");
+ }
+
+ @Test
+ public void testTransform() throws Exception {
+ runCamelTest(loadResourceAsString("simpleMapping_payload.json"),
+ loadResourceAsString("simpleMapping_result.json"),
+ "direct:basicTransform");
+ }
+
+ @Test
+ public void testTransformXML() throws Exception {
+ runCamelTest(loadResourceAsString("payload.xml"),
+ loadResourceAsString("readXMLExtTest.json"),
+ "direct:transformXML");
+ }
+
+ @Test
+ public void testTransformCSV() throws Exception {
+ runCamelTest(loadResourceAsString("payload.csv"),
+ "{\"account\":\"123\"}",
+ "direct:transformCSV");
+ }
+
+ @Test
+ public void testDatasonnetScript() throws Exception {
+ runCamelTest(loadResourceAsString("simpleMapping_payload.json"),
+ loadResourceAsString("simpleMapping_result.json"),
+ "direct:datasonnetScript");
+ }
+
+ @Test
+ public void testNamedImports() throws Exception {
+ runCamelTest("{}",
+ loadResourceAsString("namedImports_result.json"),
+ "direct:namedImports");
+ }
+
+ // TODO: 9/8/20 need to pass result type param to language
Review comment:
Fix this TODO
##########
File path: components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
##########
@@ -0,0 +1,193 @@
+[[datasonnet-language]]
+= DataSonnet Language
+:docTitle: DataSonnet
+:artifactId: camel-datasonnet
+:description: To use DataSonnet scripts in Camel expressions or predicates.
+:since: 3.7
+:supportLevel: Preview
+include::{cq-version}@camel-quarkus:ROOT:partial$reference/languages/datasonnet.adoc[opts=optional]
+
+*Since Camel {since}*
+
+Camel supports https://datasonnet.com/[DataSonnet] transformations to allow an Expression or Predicate to be used in the Java DSL or xref:manual::xml-configuration.adoc[XML
+Configuration].
+
+To use a DataSonnet expression use the following Java code:
+[source,java]
+---------------------------------------
+... datasonnet("someDSExpression") ...
+---------------------------------------
+
+== Example
+
+Here is a simple example using a DataSonnet expression as a predicate in a Message Filter:
+
+[source,java]
+------------------------------------------------------------------------------------------------
+// lets route if a line item is over $100
+from("queue:foo")
+ .filter(datasonnet("ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null"))
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet>ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null</datasonnet>
+ <to uri="queue:bar"/>
+ </filter>
+</route>
+-----------------------------------------------------------------------------
+
+Here is an example of a simple DataSonnet expression as a transformation EIP. This example will transform an XML body with
+`lineItems` into JSON while filtering out lines that are under 100.
+
+[source,java]
+------------------------------------------------------------------------------------------------
+from("queue:foo")
+ .transform(datasonnet("ds.filter(body.lineItems, function(item) item > 100)", String.class)
+ .bodyMediaType("application/xml").outputMediaType("application/json")
+ )
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet bodyMediaType="application/xml" outputMediaType="application/json" resultTypeName="java.lang.String" >
+ ds.filter(body.lineItems, function(item) item > 100)
+ </datasonnet>
+ <to uri="queue:bar"/>
+ </filter>
+</route>
+-----------------------------------------------------------------------------
+
+== Setting result type
+
+The xref:datasonnet-language.adoc[DataSonnet] expression will return a `com.datasonnet.document.Document` by default. The
+document preserves the content type metadata along with the contents of the result of the transformation. In predicates,
+however, the Document will be automatically unwrapped and the boolean content will be returned. Similarly any times you
+want the content in a specific result type like a String. To do this you have to instruct the
+xref:datasonnet-language.adoc[DataSonnet] which result type to return.
+
+In Java DSL:
+
+[source,java]
+----
+datasonnet("body.foo", String.class)
+----
+
+In Spring DSL you use the *resultType* attribute to provide a fully
Review comment:
Spring DSL -> XML DSL (that is the term we favour)
##########
File path: components/camel-datasonnet/src/main/scala/org/apache/camel/language/datasonnet/CML.scala
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.language.datasonnet
+
+import com.datasonnet.document.{DefaultDocument, Document, MediaTypes}
+import com.datasonnet.spi.{DataFormatService, Library, PluginException}
+import org.apache.camel.Exchange
+import sjsonnet.Std.builtin
+import sjsonnet.{Materializer, Val}
+
+object CML extends Library {
Review comment:
Port this to plain Java code so everyone can maintain it
##########
File path: components/camel-datasonnet/src/main/docs/datasonnet-language.adoc
##########
@@ -0,0 +1,193 @@
+[[datasonnet-language]]
+= DataSonnet Language
+:docTitle: DataSonnet
+:artifactId: camel-datasonnet
+:description: To use DataSonnet scripts in Camel expressions or predicates.
+:since: 3.7
+:supportLevel: Preview
+include::{cq-version}@camel-quarkus:ROOT:partial$reference/languages/datasonnet.adoc[opts=optional]
+
+*Since Camel {since}*
+
+Camel supports https://datasonnet.com/[DataSonnet] transformations to allow an Expression or Predicate to be used in the Java DSL or xref:manual::xml-configuration.adoc[XML
+Configuration].
+
+To use a DataSonnet expression use the following Java code:
+[source,java]
+---------------------------------------
+... datasonnet("someDSExpression") ...
+---------------------------------------
+
+== Example
+
+Here is a simple example using a DataSonnet expression as a predicate in a Message Filter:
+
+[source,java]
+------------------------------------------------------------------------------------------------
+// lets route if a line item is over $100
+from("queue:foo")
+ .filter(datasonnet("ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null"))
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet>ds.arrays.firstWith(body.lineItems, function(item) item > 100) != null</datasonnet>
+ <to uri="queue:bar"/>
+ </filter>
+</route>
+-----------------------------------------------------------------------------
+
+Here is an example of a simple DataSonnet expression as a transformation EIP. This example will transform an XML body with
+`lineItems` into JSON while filtering out lines that are under 100.
+
+[source,java]
+------------------------------------------------------------------------------------------------
+from("queue:foo")
+ .transform(datasonnet("ds.filter(body.lineItems, function(item) item > 100)", String.class)
+ .bodyMediaType("application/xml").outputMediaType("application/json")
+ )
+ .to("queue:bar")
+------------------------------------------------------------------------------------------------
+
+And the Spring DSL:
+
+[source,xml]
+-----------------------------------------------------------------------------
+<route>
+ <from uri="queue:foo"/>
+ <filter>
+ <datasonnet bodyMediaType="application/xml" outputMediaType="application/json" resultTypeName="java.lang.String" >
+ ds.filter(body.lineItems, function(item) item > 100)
Review comment:
Same as before
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org