You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@freemarker.apache.org by sg...@apache.org on 2020/06/24 12:34:47 UTC
[freemarker-generator] branch FREEMARKER-146 updated (b2c4da1 ->
57b1474)
This is an automated email from the ASF dual-hosted git repository.
sgoeschl pushed a change to branch FREEMARKER-146
in repository https://gitbox.apache.org/repos/asf/freemarker-generator.git.
from b2c4da1 FREEMARKER-146 Cleanly separate example templates and data from user-supplied content
new 03b5d6d FREEMARKER-146 Cleanly separate example templates and data from user-supplied content
new 57b1474 FREEMARKER-146 Cleanly separate example templates and data from user-supplied content
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
freemarker-generator-cli/README.md | 2 +-
.../examples/templates/dataframe/example.ftl | 6 +--
.../examples/templates/excel/csv/custom.ftl | 2 +-
freemarker-generator-cli/run-examples.sh | 2 +-
.../src/main/scripts/run-examples.sh | 2 +-
.../freemarker/generator/cli/ExamplesTest.java | 2 +-
.../freemarker/generator/cli/ManualTest.java | 4 +-
.../templates/csv/csv/transform.ftl | 30 ++----------
.../templates/csv/html/transform.ftl | 40 +++++++---------
.../templates/csv/md/transform.ftl | 22 +++------
.../templates/excel/csv/transform.ftl | 9 ++--
.../templates/lib/commons-csv.ftl | 55 ++++++++++++++++++++++
.../generator/tools/dataframe/DataFrameTool.java | 8 ++++
13 files changed, 107 insertions(+), 77 deletions(-)
create mode 100644 freemarker-generator-cli/templates/lib/commons-csv.ftl
[freemarker-generator] 01/02: FREEMARKER-146 Cleanly separate
example templates and data from user-supplied content
Posted by sg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sgoeschl pushed a commit to branch FREEMARKER-146
in repository https://gitbox.apache.org/repos/asf/freemarker-generator.git
commit 03b5d6d0a25e8e20a2ab32f8702d3576de328172
Author: Siegfried Goeschl <si...@gmail.com>
AuthorDate: Wed Jun 24 13:43:53 2020 +0200
FREEMARKER-146 Cleanly separate example templates and data from user-supplied content
---
.../freemarker/generator/tools/dataframe/DataFrameTool.java | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/dataframe/DataFrameTool.java b/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/dataframe/DataFrameTool.java
index 2d95bac..bef1b7a 100644
--- a/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/dataframe/DataFrameTool.java
+++ b/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/dataframe/DataFrameTool.java
@@ -18,6 +18,7 @@ package org.apache.freemarker.generator.tools.dataframe;
import de.unknownreality.dataframe.DataFrame;
import de.unknownreality.dataframe.DataFrameWriter;
+import de.unknownreality.dataframe.DefaultDataFrame;
import de.unknownreality.dataframe.sort.SortColumn.Direction;
import de.unknownreality.dataframe.transform.ColumnDataFrameTransform;
import de.unknownreality.dataframe.transform.CountTransformer;
@@ -42,6 +43,13 @@ import static de.unknownreality.dataframe.DataFrameWriter.DEFAULT_PRINT_FORMAT;
public class DataFrameTool {
/**
+ * Create a default data frame.
+ */
+ public DataFrame create() {
+ return new DefaultDataFrame();
+ }
+
+ /**
* Create a data frame from Apache Commons CSVParser.
*
* @param csvParser CSV Parser
[freemarker-generator] 02/02: FREEMARKER-146 Cleanly separate
example templates and data from user-supplied content
Posted by sg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sgoeschl pushed a commit to branch FREEMARKER-146
in repository https://gitbox.apache.org/repos/asf/freemarker-generator.git
commit 57b1474ad2f57461cdd6d41c3b648e943e2e8515
Author: Siegfried Goeschl <si...@gmail.com>
AuthorDate: Wed Jun 24 14:34:32 2020 +0200
FREEMARKER-146 Cleanly separate example templates and data from user-supplied content
---
freemarker-generator-cli/README.md | 2 +-
.../examples/templates/dataframe/example.ftl | 6 +--
.../examples/templates/excel/csv/custom.ftl | 2 +-
freemarker-generator-cli/run-examples.sh | 2 +-
.../src/main/scripts/run-examples.sh | 2 +-
.../freemarker/generator/cli/ExamplesTest.java | 2 +-
.../freemarker/generator/cli/ManualTest.java | 4 +-
.../templates/csv/csv/transform.ftl | 30 ++----------
.../templates/csv/html/transform.ftl | 40 +++++++---------
.../templates/csv/md/transform.ftl | 22 +++------
.../templates/excel/csv/transform.ftl | 9 ++--
.../templates/lib/commons-csv.ftl | 55 ++++++++++++++++++++++
12 files changed, 99 insertions(+), 77 deletions(-)
diff --git a/freemarker-generator-cli/README.md b/freemarker-generator-cli/README.md
index 168b8fc..70179f2 100644
--- a/freemarker-generator-cli/README.md
+++ b/freemarker-generator-cli/README.md
@@ -894,7 +894,7 @@ While this looks small and tidy there are some nifty features
Sometimes you have a CSV file which is not quite right - you need to change the format. Lets have a look how `freemarker-cli` can help
-> bin/freemarker-cli -PCVS_IN_DELIMITER=COMMA -PCSV_OUT_DELIMITER=PIPE -t templates/csv/csv/transform.ftl ./examples/data/csv/contract.csv
+> bin/freemarker-cli -PCVS_IN_DELIMITER=COMMA -PCSV_TARGET_DELIMITER=PIPE -t templates/csv/csv/transform.ftl ./examples/data/csv/contract.csv
renders the following template
diff --git a/freemarker-generator-cli/examples/templates/dataframe/example.ftl b/freemarker-generator-cli/examples/templates/dataframe/example.ftl
index 16c6a51..d1eef7b 100644
--- a/freemarker-generator-cli/examples/templates/dataframe/example.ftl
+++ b/freemarker-generator-cli/examples/templates/dataframe/example.ftl
@@ -47,9 +47,9 @@ Group By Age & Country
${DataFrameTool.print(users.groupBy("country", "age").sort("country"))}
<#--------------------------------------------------------------------------->
<#function csvInFormat>
- <#assign format = CSVTool.formats[CSV_IN_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_IN_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_IN_WITH_HEADER!"false">
+ <#assign format = CSVTool.formats[CSV_SOURCE_FORMAT!"DEFAULT"]>
+ <#assign delimiter = CSVTool.toDelimiter(CSV_SOURCE_DELIMITER!format.getDelimiter())>
+ <#assign withHeader = CSV_SOURCE_WITH_HEADER!"false">
<#assign format = format.withDelimiter(delimiter)>
<#if withHeader?boolean>
<#assign format = format.withHeader()>
diff --git a/freemarker-generator-cli/examples/templates/excel/csv/custom.ftl b/freemarker-generator-cli/examples/templates/excel/csv/custom.ftl
index d73623b..ac4b4be 100644
--- a/freemarker-generator-cli/examples/templates/excel/csv/custom.ftl
+++ b/freemarker-generator-cli/examples/templates/excel/csv/custom.ftl
@@ -15,7 +15,7 @@
specific language governing permissions and limitations
under the License.
-->
-<#assign format = CSV_OUT_FORMAT!"DEFAULT">
+<#assign format = CSV_TARGET_FORMAT!"DEFAULT">
<#assign salt = SystemTool.parameters["salt"]!"salt">
<#-- Parse the first data source & sheet of the Excel document -->
<#assign workbook = ExcelTool.parse(DataSources.get(0))>
diff --git a/freemarker-generator-cli/run-examples.sh b/freemarker-generator-cli/run-examples.sh
index d8d7925..f1ad546 100755
--- a/freemarker-generator-cli/run-examples.sh
+++ b/freemarker-generator-cli/run-examples.sh
@@ -106,7 +106,7 @@ fi
#############################################################################
echo "examples/templates/dataframe/example.ftl"
-$FREEMARKER_CMD -PCSV_IN_DELIMITER=SEMICOLON -PCSV_IN_WITH_HEADER=true -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv > target/out/dataframe.txt || { echo >&2 "Test failed. Aborting."; exit 1; }
+$FREEMARKER_CMD -PCSV_SOURCE_DELIMITER=SEMICOLON -PCSV_SOURCE_WITH_HEADER=true -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv > target/out/dataframe.txt || { echo >&2 "Test failed. Aborting."; exit 1; }
#############################################################################
# Grok
diff --git a/freemarker-generator-cli/src/main/scripts/run-examples.sh b/freemarker-generator-cli/src/main/scripts/run-examples.sh
index b95e700..53f4dcd 100755
--- a/freemarker-generator-cli/src/main/scripts/run-examples.sh
+++ b/freemarker-generator-cli/src/main/scripts/run-examples.sh
@@ -106,7 +106,7 @@ fi
#############################################################################
echo "examples/templates/dataframe/example.ftl"
-$FREEMARKER_CMD -PCSV_IN_DELIMITER=SEMICOLON -PCSV_IN_WITH_HEADER=true -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv > target/out/dataframe.txt || { echo >&2 "Test failed. Aborting."; exit 1; }
+$FREEMARKER_CMD -PCSV_SOURCE_DELIMITER=SEMICOLON -PCSV_SOURCE_WITH_HEADER=true -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv > target/out/dataframe.txt || { echo >&2 "Test failed. Aborting."; exit 1; }
#############################################################################
# Grok
diff --git a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ExamplesTest.java b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ExamplesTest.java
index 63c792f..0c3820c 100644
--- a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ExamplesTest.java
+++ b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ExamplesTest.java
@@ -105,7 +105,7 @@ public class ExamplesTest extends AbstractMainTest {
@Test
public void shouldRunDataFrameExamples() throws IOException {
- assertValid(execute("-PCSV_IN_DELIMITER=SEMICOLON -PCSV_IN_WITH_HEADER=true -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv"));
+ assertValid(execute("-PCSV_SOURCE_DELIMITER=SEMICOLON -PCSV_SOURCE_WITH_HEADER=true -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv"));
}
@Test
diff --git a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
index b12488a..faa6080 100644
--- a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
+++ b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
@@ -24,8 +24,8 @@ import java.util.Arrays;
public class ManualTest {
private static final String SPACE = " ";
- private static final String CMD = "-PCSV_IN_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv";
- // private static final String CMD = "-PCSV_IN_WITH_HEADER=true -PCSV_IN_FORMAT=DEFAULT -PCSV_OUT_FORMAT=EXCEL -t templates/csv/csv/transform.ftl examples/data/csv/contract.csv";
+ // private static final String CMD = "-PCSV_SOURCE_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv";
+ private static final String CMD = "-PCSV_SOURCE_WITH_HEADER=true -PCSV_SOURCE_FORMAT=DEFAULT -PCSV_TARGET_FORMAT=EXCEL -t templates/csv/md/transform.ftl examples/data/csv/contract.csv";
public static void main(String[] args) {
Main.execute(toArgs(CMD));
diff --git a/freemarker-generator-cli/templates/csv/csv/transform.ftl b/freemarker-generator-cli/templates/csv/csv/transform.ftl
index 0976ef0..656131d 100644
--- a/freemarker-generator-cli/templates/csv/csv/transform.ftl
+++ b/freemarker-generator-cli/templates/csv/csv/transform.ftl
@@ -1,3 +1,4 @@
+<#ftl output_format="plainText" strip_whitespace=true>
<#--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -14,31 +15,10 @@
specific language governing permissions and limitations
under the License.
-->
+<#import "/templates/lib/commons-csv.ftl" as csv />
<#assign dataSource = DataSources.get(0)>
-<#assign csvParser = CSVTool.parse(dataSource, csvInFormat())>
-<#assign csvPrinter = CSVTool.printer(csvOutFormat())>
+<#assign csvParser = CSVTool.parse(dataSource, csv.sourceFormat())>
+<#assign csvPrinter = CSVTool.printer(csv.targetFormat())>
<#list csvParser.iterator() as record>
${csvPrinter.printRecord(record)}<#t>
-</#list>
-
-<#function csvInFormat>
- <#assign format = CSVTool.formats[CSV_IN_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_IN_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_IN_WITH_HEADER!"false">
- <#assign format = format.withDelimiter(delimiter)>
- <#if withHeader?boolean>
- <#assign format = format.withHeader()>
- </#if>
- <#return format>
-</#function>
-
-<#function csvOutFormat>
- <#assign format = CSVTool.formats[CSV_OUT_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_OUT_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_OUT_WITH_HEADER!"false">
- <#assign format = format.withDelimiter(delimiter)>
- <#if withHeader?boolean>
- <#assign format = format.withHeader>
- </#if>
- <#return format>
-</#function>
+</#list>
\ No newline at end of file
diff --git a/freemarker-generator-cli/templates/csv/html/transform.ftl b/freemarker-generator-cli/templates/csv/html/transform.ftl
index af62744..cda4cd6 100644
--- a/freemarker-generator-cli/templates/csv/html/transform.ftl
+++ b/freemarker-generator-cli/templates/csv/html/transform.ftl
@@ -1,4 +1,4 @@
-<#ftl output_format="HTML" >
+<#ftl output_format="HTML" strip_whitespace=true>
<#--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -15,8 +15,9 @@
specific language governing permissions and limitations
under the License.
-->
+<#import "/templates/lib/commons-csv.ftl" as csv />
<#assign dataSource = DataSources.get(0)>
-<#assign csvParser = CSVTool.parse(dataSource, csvInFormat())>
+<#assign csvParser = CSVTool.parse(dataSource, csv.sourceFormat())>
<#assign csvHeaders = csvParser.getHeaderNames()>
<#--------------------------------------------------------------------------->
<!DOCTYPE html>
@@ -38,28 +39,21 @@
</html>
<#--------------------------------------------------------------------------->
<#macro writeHeaders headers>
- <tr>
- <#list headers as header>
- <th>${header}</th>
- </#list>
- </tr>
+ <#if headers?has_content>
+ <tr>
+ <#list headers as header>
+ <th>${header}</th>
+ </#list>
+ </tr>
+ </#if>
</#macro>
<#--------------------------------------------------------------------------->
<#macro writeColumns record>
- <tr>
- <#list record.iterator() as field>
- <th>${field}</th>
- </#list>
- </tr>
-</#macro>
-<#--------------------------------------------------------------------------->
-<#function csvInFormat>
- <#assign format = CSVTool.formats[CSV_IN_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_IN_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_IN_WITH_HEADER!"false">
- <#assign format = format.withDelimiter(delimiter)>
- <#if withHeader?boolean>
- <#assign format = format.withHeader()>
+ <#if record?has_content>
+ <tr>
+ <#list record.iterator() as field>
+ <td>${field}</td>
+ </#list>
+ </tr>
</#if>
- <#return format>
-</#function>
+</#macro>
\ No newline at end of file
diff --git a/freemarker-generator-cli/templates/csv/md/transform.ftl b/freemarker-generator-cli/templates/csv/md/transform.ftl
index fbd2c9f..2766125 100644
--- a/freemarker-generator-cli/templates/csv/md/transform.ftl
+++ b/freemarker-generator-cli/templates/csv/md/transform.ftl
@@ -14,8 +14,9 @@
specific language governing permissions and limitations
under the License.
-->
+<#import "/templates/lib/commons-csv.ftl" as csv />
<#assign dataSource = DataSources.get(0)>
-<#assign csvParser = CSVTool.parse(dataSource, csvInFormat())>
+<#assign csvParser = CSVTool.parse(dataSource, csv.sourceFormat())>
<#assign headers = (csvParser.getHeaderMap()!{})?keys>
<#assign records = csvParser.records>
<#--------------------------------------------------------------------------->
@@ -32,18 +33,9 @@
</#macro>
<#--------------------------------------------------------------------------->
<#macro writeColums columns>
- <#list columns as column>
- | ${column.iterator()?join(" | ", "")} |
- </#list>
-</#macro>
-<#--------------------------------------------------------------------------->
-<#function csvInFormat>
- <#assign format = CSVTool.formats[CSV_IN_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_IN_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_IN_WITH_HEADER!"false">
- <#assign format = format.withDelimiter(delimiter)>
- <#if withHeader?boolean>
- <#assign format = format.withHeader()>
+ <#if columns?has_content>
+ <#list columns as column>
+ | ${column.iterator()?join(" | ", "")} |
+ </#list>
</#if>
- <#return format>
-</#function>
+</#macro>
diff --git a/freemarker-generator-cli/templates/excel/csv/transform.ftl b/freemarker-generator-cli/templates/excel/csv/transform.ftl
index ec4f35c..794e942 100644
--- a/freemarker-generator-cli/templates/excel/csv/transform.ftl
+++ b/freemarker-generator-cli/templates/excel/csv/transform.ftl
@@ -16,11 +16,12 @@
under the License.
-->
<#-- Parse the first data source & sheet of the Excel document -->
+<#import "/templates/lib/commons-csv.ftl" as csv />
<#assign workbook = ExcelTool.parse(DataSources.get(0))>
<#assign sheet = ExcelTool.getSheets(workbook)[0]>
<#assign records = ExcelTool.toTable(sheet)>
<#-- Setup CSVPrinter -->
-<#assign csvPrinter = CSVTool.printer(csvOutFormat())>
+<#assign csvPrinter = CSVTool.printer(csv.targetFormat())>
<#-- Print each line of the Excel as CSV record -->
<#compress>
<#list records as record>
@@ -29,9 +30,9 @@
</#compress>
<#--------------------------------------------------------------------------->
<#function csvOutFormat>
- <#assign format = CSVTool.formats[CSV_OUT_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_OUT_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_OUT_WITH_HEADER!"false">
+ <#assign format = CSVTool.formats[CSV_TARGET_FORMAT!"DEFAULT"]>
+ <#assign delimiter = CSVTool.toDelimiter(CSV_TARGET_DELIMITER!format.getDelimiter())>
+ <#assign withHeader = CSV_TARGET_WITH_HEADER!"false">
<#assign format = format.withDelimiter(delimiter)>
<#if withHeader?boolean>
<#assign format = format.withHeader>
diff --git a/freemarker-generator-cli/templates/lib/commons-csv.ftl b/freemarker-generator-cli/templates/lib/commons-csv.ftl
new file mode 100644
index 0000000..4082fa8
--- /dev/null
+++ b/freemarker-generator-cli/templates/lib/commons-csv.ftl
@@ -0,0 +1,55 @@
+<#ftl output_format="plainText" strip_text="true">
+<#--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+<#---
+ Detemine the CSV format for reading a CSV files using user-supplied
+ parameters from the data model.
+
+ * CSV_SOURCE_FORMAT - see https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/CSVFormat.html
+ * CSV_SOURCE_DELIMITER - symbolic name of delimiter, e.g. "COLON" or "SEMICOLON"
+ * CSV_SOURCE_WITH_HEADER - whether the first rows are headers
+-->
+<#function sourceFormat>
+ <#assign format = CSVTool.formats[CSV_SOURCE_FORMAT!"DEFAULT"]>
+ <#assign delimiter = CSVTool.toDelimiter(CSV_SOURCE_DELIMITER!format.getDelimiter())>
+ <#assign withHeader = CSV_SOURCE_WITH_HEADER!"false">
+ <#assign format = format.withDelimiter(delimiter)>
+ <#if withHeader?boolean>
+ <#assign format = format.withHeader()>
+ </#if>
+ <#return format>
+</#function>
+
+<#---
+ Detemine the CSV format for printing a CSV files using user-supplied
+ parameters from the data model.
+
+ * CSV_TARGET_FORMAT - see https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/CSVFormat.html
+ * CSV_TARGET_DELIMITER - symbolic name of delimiter, e.g. "COLON" or "SEMICOLON"
+ * CSV_TARGET_WITH_HEADER - whether the first rows are headers
+-->
+<#function targetFormat>
+ <#assign format = CSVTool.formats[CSV_TARGET_FORMAT!"DEFAULT"]>
+ <#assign delimiter = CSVTool.toDelimiter(CSV_TARGET_DELIMITER!format.getDelimiter())>
+ <#assign withHeader = CSV_TARGET_WITH_HEADER!"false">
+ <#assign format = format.withDelimiter(delimiter)>
+ <#if withHeader?boolean>
+ <#assign format = format.withHeader>
+ </#if>
+ <#return format>
+</#function>