You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@freemarker.apache.org by sg...@apache.org on 2020/06/24 20:03:29 UTC
[freemarker-generator] 01/01: FREEMARKER-147 Complete Maven site
documenation
This is an automated email from the ASF dual-hosted git repository.
sgoeschl pushed a commit to branch FREEMARKER-147
in repository https://gitbox.apache.org/repos/asf/freemarker-generator.git
commit ef2465c7bc925e2b8b68c4de3a62956d37c35094
Author: Siegfried Goeschl <si...@gmail.com>
AuthorDate: Wed Jun 24 22:03:12 2020 +0200
FREEMARKER-147 Complete Maven site documenation
---
.../generator/base/template/TemplateOutput.java | 2 +
.../generator/base/template/TemplateSource.java | 2 +-
.../base/template/TemplateTransformations.java | 3 +
.../src/site/markdown/index.md | 4 +-
freemarker-generator-cli/README.md | 2 +-
.../examples/templates/dataframe/example.ftl | 14 +---
.../templates/json/dataframe/github-users.ftl | 20 ++++++
freemarker-generator-cli/pom.xml | 2 +-
.../src/site/markdown/cli/concepts/data-models.md | 8 +--
.../src/site/markdown/cli/concepts/data-sources.md | 2 +-
.../src/site/markdown/cli/concepts/named-uris.md | 2 +-
.../markdown/cli/concepts/template-includes.md | 17 +++++
.../site/markdown/cli/concepts/template-loading.md | 47 +++++++++++++
.../site/markdown/cli/concepts/transformation.md | 4 +-
.../site/markdown/cli/concepts/user-parameters.md | 2 +-
.../src/site/markdown/cli/tools/dataframe.md | 43 +++++++-----
.../src/site/markdown/cli/tools/overview.md | 20 ++++++
.../site/markdown/cli/usage/cli-configuration.md | 77 ++++++++++++++++++++++
.../markdown/cli/usage/transforming-directories.md | 4 +-
.../src/site/markdown/index.md | 6 +-
.../freemarker/generator/cli/ManualTest.java | 4 +-
.../{examples => }/templates/cat.ftl | 1 -
freemarker-generator-tools/pom.xml | 4 +-
.../generator/tools/excel/ExcelTool.java | 2 +-
.../src/site/markdown/index.md | 67 ++++++++++++++++++-
pom.xml | 5 ++
26 files changed, 316 insertions(+), 48 deletions(-)
diff --git a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java
index 1e5cb50..5e8e8a5 100644
--- a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java
+++ b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java
@@ -29,6 +29,8 @@ import static java.util.Objects.requireNonNull;
* Information about where to write the ouput of a template. Initially we
* wanted to use a <code>FileWriter</code> but it requires actually an
* existing output file (otherwise a FileNotFound exception is thrown).
+ * An alternative could be a "LazyFileWriter" which creates the file on
+ * the first write access.
*/
public class TemplateOutput {
diff --git a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java
index 42c36b6..4bc86bf 100644
--- a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java
+++ b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java
@@ -23,7 +23,7 @@ import java.nio.charset.StandardCharsets;
/**
* Information how to load a template. The template is either
- * provided as source code or as template path resolved
+ * provided as literal text or as template path resolved
* by FreeMarker's template loader.
*/
public class TemplateSource {
diff --git a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java
index ed65111..6c09b60 100644
--- a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java
+++ b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java
@@ -22,6 +22,9 @@ import java.util.List;
import static java.util.Objects.requireNonNull;
+/**
+ * Keeps track of all transformations being executed.
+ */
public class TemplateTransformations {
private final List<TemplateTransformation> templateTransformations;
diff --git a/freemarker-generator-base/src/site/markdown/index.md b/freemarker-generator-base/src/site/markdown/index.md
index 2fd9f95..212345d 100644
--- a/freemarker-generator-base/src/site/markdown/index.md
+++ b/freemarker-generator-base/src/site/markdown/index.md
@@ -1 +1,3 @@
-TBD
\ No newline at end of file
+# Apache FreeMarker CLI Base
+
+This package contains infrastructure code not depending on `Apache FreeMarker` directly since `DataSources` and their creation might be useful for un-related command-line tools.
diff --git a/freemarker-generator-cli/README.md b/freemarker-generator-cli/README.md
index 70179f2..9573c1a 100644
--- a/freemarker-generator-cli/README.md
+++ b/freemarker-generator-cli/README.md
@@ -1523,7 +1523,7 @@ Within the script a FreeMarker data model is set up and passed to the template -
| Entry | Description |
|-----------------------|-----------------------------------------------------------------------------------------------------------|
| CSVTool | Process CSV files using [Apache Commons CSV](https://commons.apache.org/proper/commons-csv/) |
-| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
+| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
| ExecTool | Execute command line tools using [Apache Commons Exec](https://commons.apache.org/proper/commons-exec/) |
| ExcelTool | Process Excels files (XLS, XLSX) using [Apache POI](https://poi.apache.org) |
| DataSources | Helper class to find data sources, e.g. by name, extension or index |
diff --git a/freemarker-generator-cli/examples/templates/dataframe/example.ftl b/freemarker-generator-cli/examples/templates/dataframe/example.ftl
index d1eef7b..9929d21 100644
--- a/freemarker-generator-cli/examples/templates/dataframe/example.ftl
+++ b/freemarker-generator-cli/examples/templates/dataframe/example.ftl
@@ -1,4 +1,3 @@
-<#ftl output_format="plainText">
<#--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -16,7 +15,7 @@
under the License.
-->
<#assign dataSource = DataSources.get(0)>
-<#assign csvParser = CSVTool.parse(dataSource, csvInFormat())>
+<#assign csvParser = CSVTool.parse(dataSource, CSVTool.formats["DATAFRAME"])>
<#assign users = DataFrameTool.fromCSVParser(csvParser)>
Original Data
@@ -45,14 +44,3 @@ ${DataFrameTool.print(users.getColumn("country").transform(DataFrameTool.transfo
Group By Age & Country
=============================================================================
${DataFrameTool.print(users.groupBy("country", "age").sort("country"))}
-<#--------------------------------------------------------------------------->
-<#function csvInFormat>
- <#assign format = CSVTool.formats[CSV_SOURCE_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_SOURCE_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_SOURCE_WITH_HEADER!"false">
- <#assign format = format.withDelimiter(delimiter)>
- <#if withHeader?boolean>
- <#assign format = format.withHeader()>
- </#if>
- <#return format>
-</#function>
diff --git a/freemarker-generator-cli/examples/templates/json/dataframe/github-users.ftl b/freemarker-generator-cli/examples/templates/json/dataframe/github-users.ftl
new file mode 100644
index 0000000..a6547fc
--- /dev/null
+++ b/freemarker-generator-cli/examples/templates/json/dataframe/github-users.ftl
@@ -0,0 +1,20 @@
+<#ftl output_format="plainText" >
+<#--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<#assign json = GsonTool.parse(DataSources.get(0))>
+<#assign dataframe = DataFrameTool.fromMaps(json)>
+${DataFrameTool.print(dataframe)}
diff --git a/freemarker-generator-cli/pom.xml b/freemarker-generator-cli/pom.xml
index 47756a9..55482df 100644
--- a/freemarker-generator-cli/pom.xml
+++ b/freemarker-generator-cli/pom.xml
@@ -177,7 +177,7 @@
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-simple</artifactId>
+ <artifactId>slf4j-nop</artifactId>
</dependency>
<!-- Testing -->
<dependency>
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md
index c05323c..c8707c7 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md
@@ -9,14 +9,14 @@ A `DataModel` is an eagerly loaded `DataSource` available in Apache FreeMarker's
Expose the fields of the JSON data source in FreeMarker's model
```
-bin/freemarker-cli --data-model https://xkcd.com/info.0.json -i '<a href="${img}">${title}</a>'
+bin/freemarker-cli --data-model https://xkcd.com/info.0.json -i '<a href="${img}">${title}</a>'; echo
<a href="https://imgs.xkcd.com/comics/scenario_4.png">Scenario 4</a>
```
Exposed the JSON data source as variable `post` in FreeMarker's model
```
-bin/freemarker-cli --data-model post=https://jsonplaceholder.typicode.com/posts/2 -i 'post title is: ${post.title}'
+bin/freemarker-cli --data-model post=https://jsonplaceholder.typicode.com/posts/2 -i 'post title is: ${post.title}'; echo
post title is: qui est esse
```
@@ -31,14 +31,14 @@ USER=sgoeschl
Expose a single envionment variable in theFreeMarker model
```
-bin/freemarker-cli --data-model NAME=env:///USER -i 'Hello ${NAME}'
+bin/freemarker-cli --data-model NAME=env:///USER -i 'Hello ${NAME}'; echo
Hello sgoeschl
```
Alternatively use the short command line options, e.g.
```
-bin/freemarker-cli -m NAME=env:///USER -i 'Hello ${NAME}!'
+bin/freemarker-cli -m NAME=env:///USER -i 'Hello ${NAME}!'; echo
Hello sgoeschl!
```
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md
index 548a65b..0eafef8 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md
@@ -60,7 +60,7 @@ FreeMarker CLI DataSources
or load them from a directory
```
-bin/freemarker-cli -t templates/info.ftl -s examples/data/
+bin/freemarker-cli -t templates/info.ftl -s examples/data
FreeMarker CLI DataSources
------------------------------------------------------------------------------
[#1], name=combined-access.log, group=default, contentType=text/plain, charset=UTF-8, length=2,068 Bytes
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md
index 1c99b27..693efac 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md
@@ -1,4 +1,4 @@
-# Named URIs
+## Named URIs
Named URIs allow to identify `DataSources` and pass additional information
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/template-includes.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-includes.md
new file mode 100644
index 0000000..8aa0dc3
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-includes.md
@@ -0,0 +1,17 @@
+### Template Includes
+
+When writing more and more templates you find reusable snippets which should not be copy/pasted but organized as [templates includes](https://freemarker.apache.org/docs/ref_directive_include.html).
+
+E.g. `Apache Commons CSV` FTL code is included as shown below
+
+```
+<#import "/templates/lib/commons-csv.ftl" as csv />
+<#assign dataSource = DataSources.get(0)>
+<#assign csvParser = CSVTool.parse(dataSource, csv.sourceFormat())>
+```
+
+What is happening here?
+
+* The template loader picks up `/templates/lib/commons-csv.ftl` from the `Apache FreeMarker CLI` installation directory and imports it as `csv`
+* The imported template exports two methods `csv.sourceFormat()` and `csv.targetFormat()`
+* The exposed methods do the right thing based on the user-supplied parameters and can be re-used for all CSV related templates
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/template-loading.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-loading.md
new file mode 100644
index 0000000..15ce5a3
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-loading.md
@@ -0,0 +1,47 @@
+## Template Loading
+
+`Apache FreeMarker CLI` loads templates from multiple locations in the following order
+
+* Current working directory
+* Optional `~/.freemarker-cli` directory
+* `Apache FreeMarker CLI` installation directory
+
+You can check the available template locations easily on the command line
+
+```
+freemarker-cli -t templates/info.ftl
+
+FreeMarker CLI Template Directories
+------------------------------------------------------------------------------
+[#1] /Users/sgoeschl/work/github/apache/freemarker-generator
+[#2] /Users/sgoeschl/.freemarker-cli
+[#3] /Applications/Java/freemarker-cli-2.0.0
+```
+
+### Template Loaders In Action
+
+Let's assume you have `freemarker-cli` in your path and you execute
+
+```
+> cd /
+> which freemarker-cli
+/Applications/Java/freemarker-cli-2.0.0/bin/freemarker-cli
+> freemarker-cli -t templates/cat.ftl https://jsonplaceholder.typicode.com/posts/2
+{
+ "userId": 1,
+ "id": 2,
+ "title": "qui est esse",
+ "body": "est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla"
+}
+```
+
+In your root directory there is no `templates/cat.ftl` but it is picked up from `/Applications/Java/freemarker-cli-2.0.0`
+
+### Literal Templates
+
+You can also directly specify a template to be loaded without relying on template loaders
+
+```
+freemarker-cli -t some-directory/transform.ftl some-directory/contract.csv
+```
+
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md
index 966743b..be6da2f 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md
@@ -1,5 +1,7 @@
## Transformation
+The `freemarker-cli` generates text output based on FreeMarker templates and data
+
* A command line invocation requires 1..n `templates` and 0..n `data sources` / `data models`
* A command line invocation is mapped to a series of `transformations`
* The `transformation` consists of exactly one `template`, 0..n `data sources` / `data models` and an `output`
@@ -9,4 +11,4 @@
* an output directory
* When the output is written to a directory
* the structure of the input directory is preserved
- * a "ftl" file externsion is removed
+ * a `ftl file externsion is removed
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md
index 46f2d2e..b4957f5 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md
@@ -1,4 +1,4 @@
-# User-Supplied Parameters
+## User-Supplied Parameters
User-supplied parameters allow to pass additional information to an Apache FreeMarker template
diff --git a/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md b/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md
index 83809ed..c8daa5d 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md
@@ -1,8 +1,8 @@
-# DataFrameTool
+## DataFrameTool
The `DataFrameTool` uses [nRo/DataFrame](https://github.com/nRo/DataFrame) to convert tabular data into a `DataFrame`.
-A `DataFrame` allows declartive filtering and transformation of tabular data, i.e. little code to write.
+A `DataFrame` allows declarative filtering and transformation of tabular data, i.e. less code to write.
Currently the following sources are supported
@@ -10,7 +10,7 @@ Currently the following sources are supported
* JSON arrays represented as collection of maps
* Excel sheets represented as rows
-## CSV Examples
+### Working With CSV
[nRo/DataFrame]("https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv") provides the following CSV file
@@ -27,15 +27,27 @@ Schmitt;30;Germany
Meier;30;Germany
```
-and create a `DateFrame` using the following code
+and create a `DateFrame` using the following code snippet
```
-<#assign cvsFormat = CSVTool.formats["DEFAULT"].withHeader().withDelimiter(';')>
-<#assign csvParser = CSVTool.parse(DataSources.get(0), cvsFormat)>
-<#assign users = DataFrameTool.toDataFrame(csvParser)>
+<#assign dataSource = DataSources.get(0)>
+<#assign csvParser = CSVTool.parse(dataSource, CSVTool.formats["DATAFRAME"])>
+<#assign users = DataFrameTool.fromCSVParser(csvParser)>
+```
+
+The example can be executed by running
+
+```
+bin/freemarker-cli -PCSV_SOURCE_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv
+```
+
+or
+
+```
+bin/freemarker-cli -PCSV_SOURCE_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv
```
-### Select By Age
+#### Select By Age
```
${DataFrameTool.print(users.select("(age > 40)"))}
@@ -53,7 +65,7 @@ which shows
└────────────┴────────────┴────────────┘
```
-### Complex Select & Sort
+#### Complex Select & Sort
Now we want to create a new `DataFrame` by selecting `name` and `country`
@@ -82,7 +94,7 @@ which shows
└────────────┴────────────┴────────────┘
```
-### Count Column Values
+#### Count Column Values
Let's assume we want to count the records for each `country`
@@ -106,7 +118,7 @@ returns the following `DataFrame`
└────────────┴────────────┘
```
-### Group By Age And Country
+#### Group By Age And Country
Let's assume that we want to group the `DataFrame` by `age` and `country`
@@ -136,10 +148,11 @@ which results in
└────────────┴────────────┘
```
-## JSON Examples
+### Working With JSON
-Here we load a `examples/data/json/github-users.json` which represents a tabular data be
-being parsed as a list of maps and print the JSOB as dataframe
+Here we load a `examples/data/json/github-users.json` which represents a tabular
+data being parsed as a list of maps and print the JSON as dataframe. Technically
+it is a list of maps hence we invoke `DataFrameTool.fromMaps()
```
./bin/freemarker-cli \
@@ -165,7 +178,7 @@ being parsed as a list of maps and print the JSOB as dataframe
└────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┘
```
-## Excel Examples
+### Working With Excel
Let's transform an Excel Sheet to a `DataFrame` being printed using the following template
diff --git a/freemarker-generator-cli/src/site/markdown/cli/tools/overview.md b/freemarker-generator-cli/src/site/markdown/cli/tools/overview.md
new file mode 100644
index 0000000..e5a64c6
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/tools/overview.md
@@ -0,0 +1,20 @@
+## Overview
+
+The following `tools` are currently implemented
+
+| Tool | Description |
+|-----------------------|-----------------------------------------------------------------------------------------------------------|
+| CSVTool | Process CSV files using [Apache Commons CSV](https://commons.apache.org/proper/commons-csv/) |
+| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
+| ExecTool | Execute command line tools using [Apache Commons Exec](https://commons.apache.org/proper/commons-exec/) |
+| ExcelTool | Process Excels files (XLS, XLSX) using [Apache POI](https://poi.apache.org) |
+| FreeMarkerTool | Expose useful FreeMarker classes |
+| GrokTool | Process text files using [Grok](https://github.com/thekrakken/java-grok) instead of regular expressions |
+| GsonTool | Process JSON files using [GSON](https://github.com/google/gson) |
+| JsonPathTool | Process JSON file using [Java JSON Path](https://github.com/json-path/JsonPath) |
+| JsoupTool | Processing HTML files using [Jsoup](https://jsoup.org) |
+| PropertiesTool | Process JDK properties files |
+| SystemTool | System-related utility methods |
+| UUIDTool | Create UUIDs |
+| XmlTool | Process XML files using [Apache FreeMarker](https://freemarker.apache.org/docs/xgui.html) |
+| YamlTool | Process YAML files using [SnakeYAML](https://bitbucket.org/asomov/snakeyaml/wiki/Home) |
diff --git a/freemarker-generator-cli/src/site/markdown/cli/usage/cli-configuration.md b/freemarker-generator-cli/src/site/markdown/cli/usage/cli-configuration.md
new file mode 100644
index 0000000..499f3c2
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/usage/cli-configuration.md
@@ -0,0 +1,77 @@
+## CLI Configuration
+
+The `Apache FreeMarker CLI` configuration can be tweaked to
+
+* Change the underlying `Apache FreeMarker Configuration`
+* Instantiate custom tools
+* Storing user-specific templates
+
+### Apache FreeMarker CLI Properties File
+
+`Apache FreeMarker CLI` reads the `conf/freemarker-cli.properties`
+
+```
+#############################################################################
+# General FreeMarker Configuration
+# See https://freemarker.apache.org/docs/api/freemarker/template/Configuration.html#setSetting-java.lang.String-java.lang.String-
+#############################################################################
+# freemarker.configuration.setting.locale=JVM default
+
+#############################################################################
+# Configure FreeMarker Tools (name -> implementation class)
+#############################################################################
+freemarker.tools.CSVTool=org.apache.freemarker.generator.tools.commonscsv.CommonsCSVTool
+freemarker.tools.DataFrameTool=org.apache.freemarker.generator.tools.dataframe.DataFrameTool
+freemarker.tools.ExcelTool=org.apache.freemarker.generator.tools.excel.ExcelTool
+freemarker.tools.ExecTool=org.apache.freemarker.generator.tools.commonsexec.CommonsExecTool
+freemarker.tools.FreeMarkerTool=org.apache.freemarker.generator.tools.freemarker.FreeMarkerTool
+freemarker.tools.GrokTool=org.apache.freemarker.generator.tools.grok.GrokTool
+freemarker.tools.GsonTool=org.apache.freemarker.generator.tools.gson.GsonTool
+freemarker.tools.JsonPathTool=org.apache.freemarker.generator.tools.jsonpath.JsonPathTool
+freemarker.tools.JsoupTool=org.apache.freemarker.generator.tools.jsoup.JsoupTool
+freemarker.tools.PropertiesTool=org.apache.freemarker.generator.tools.properties.PropertiesTool
+freemarker.tools.SystemTool=org.apache.freemarker.generator.tools.system.SystemTool
+freemarker.tools.UUIDTool=org.apache.freemarker.generator.tools.uuid.UUIDTool
+freemarker.tools.XmlTool=org.apache.freemarker.generator.tools.xml.XmlTool
+freemarker.tools.YamlTool=org.apache.freemarker.generator.tools.snakeyaml.SnakeYamlTool
+```
+
+Changing this file allows to tweak the underlying `Apache FreeMarker Configuration` and add custom tools.
+
+### Storing User-Specific Templates
+
+Over the time you will accumalate more and more `Apache FreeMarker` templates - some of them are stored within a project but some of the more general might be free-floating and you don't want to store them in the installation directory.
+
+To give those free-floating templates a home `Apache FreeMarker CLI` tries to read templates from `~/freemarker-cli`, e.g.
+
+```
+tree ~/.freemarker-cli/
+/Users/sgoeschl/.freemarker-cli/
+`-- templates
+ `-- json
+ |-- confluence
+ | |-- aws
+ | | `-- describe.ftl
+ |-- csv
+ | `-- swagger-endpoints.ftl
+ |-- ftl
+ | `-- customer-user-products.ftl
+ |-- html
+ | `-- customer-user-products.ftl
+ `-- md
+ `-- customer-user-products.ftl
+```
+
+If a `~/freemarker-cli` is found it will be automatically added to the `FreeMarker Template Loader` (for more information see https://freemarker.apache.org/docs/pgui_config_templateloading.html)
+
+You can easily check this, e.g.
+
+```
+> freemarker-cli -t templates/info.ftl
+
+FreeMarker CLI Template Directories
+------------------------------------------------------------------------------
+[#1] /Users/sgoeschl
+[#2] /Users/sgoeschl/.freemarker-cli
+[#3] /Applications/Java/freemarker-cli-2.0.0
+```
diff --git a/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md b/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md
index 9f6a36f..de95e22 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md
@@ -1,6 +1,6 @@
## Transforming Directories
-FreeMarker CLI supports the transformation of directories
+`Apache FreeMarker CLI` supports the transformation of directories
* Transform an input directory recursively into an output directory
* If a template has a ".ftl" extension this extension will be removed after processing
@@ -13,7 +13,7 @@ The following sample files are used
* template/nginx/nginx.conf.ftl
```
-appassembler> tree examples/data/template/
+tree examples/data/template/
examples/data/template/
|-- application.properties
`-- nginx
diff --git a/freemarker-generator-cli/src/site/markdown/index.md b/freemarker-generator-cli/src/site/markdown/index.md
index 2eb0dfd..d4526e9 100644
--- a/freemarker-generator-cli/src/site/markdown/index.md
+++ b/freemarker-generator-cli/src/site/markdown/index.md
@@ -6,12 +6,16 @@
* [Data Sources](cli/concepts/data-sources.html)
* [Data Models](cli/concepts/data-models.html)
* [User-Supplied Parameters](cli/concepts/user-parameters.html)
+* [Template Loading](cli/concepts/template-loading.html)
+* [Template Includes](cli/concepts/template-includes.html)
* [Transformation](cli/concepts/transformation.html)
-### Tools
+### Working With Tools
+* [Overview](cli/tools/overview.html)
* [DataFrameTool](cli/tools/dataframe.html)
### Usage
+* [CLI Configuration](cli/usage/cli-configuration.html)
* [Transforming Directories](cli/usage/transforming-directories.html)
diff --git a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
index faa6080..98d1673 100644
--- a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
+++ b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
@@ -25,7 +25,9 @@ public class ManualTest {
private static final String SPACE = " ";
// private static final String CMD = "-PCSV_SOURCE_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv";
- private static final String CMD = "-PCSV_SOURCE_WITH_HEADER=true -PCSV_SOURCE_FORMAT=DEFAULT -PCSV_TARGET_FORMAT=EXCEL -t templates/csv/md/transform.ftl examples/data/csv/contract.csv";
+ // private static final String CMD = "-PCSV_SOURCE_WITH_HEADER=true -PCSV_SOURCE_FORMAT=DEFAULT -PCSV_TARGET_FORMAT=EXCEL -t templates/csv/md/transform.ftl examples/data/csv/contract.csv";
+ private static final String CMD = "-t examples/templates/json/dataframe/github-users.ftl examples/data/json/github-users.json";
+
public static void main(String[] args) {
Main.execute(toArgs(CMD));
diff --git a/freemarker-generator-cli/examples/templates/cat.ftl b/freemarker-generator-cli/templates/cat.ftl
similarity index 94%
rename from freemarker-generator-cli/examples/templates/cat.ftl
rename to freemarker-generator-cli/templates/cat.ftl
index a878261..c912c12 100644
--- a/freemarker-generator-cli/examples/templates/cat.ftl
+++ b/freemarker-generator-cli/templates/cat.ftl
@@ -1,4 +1,3 @@
-<#ftl output_format="plainText" strip_whitespace=true>
<#--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
diff --git a/freemarker-generator-tools/pom.xml b/freemarker-generator-tools/pom.xml
index 57c6b59..256987a 100644
--- a/freemarker-generator-tools/pom.xml
+++ b/freemarker-generator-tools/pom.xml
@@ -122,10 +122,12 @@
<reporting>
<plugins>
<plugin>
- <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
</plugin>
<plugin>
+ <artifactId>maven-jxr-plugin</artifactId>
+ </plugin>
+ <plugin>
<groupId>org.apache.rat</groupId>
<artifactId>apache-rat-plugin</artifactId>
<configuration>
diff --git a/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java b/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java
index b5ead1d..08133fc 100644
--- a/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java
+++ b/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java
@@ -53,7 +53,7 @@ public class ExcelTool {
// make sure that the workbook is closed together with the data source
return dataSource.addClosable(workbook);
} catch (IOException e) {
- throw new RuntimeException("Failed to parse Ecxel data source: " + dataSource, e);
+ throw new RuntimeException("Failed to parse Excel data source: " + dataSource, e);
}
}
diff --git a/freemarker-generator-tools/src/site/markdown/index.md b/freemarker-generator-tools/src/site/markdown/index.md
index 2fd9f95..19d0bb2 100644
--- a/freemarker-generator-tools/src/site/markdown/index.md
+++ b/freemarker-generator-tools/src/site/markdown/index.md
@@ -1 +1,66 @@
-TBD
\ No newline at end of file
+# Apache FreeMarker CLI Tools
+
+## Overview
+
+The implementation of the `Apache FreeMarker CLI Tools` was inspired by [Apache Velocity Tools](https://velocity.apache.org/tools/devel/) - a `tool` is just a POJO (plain old Java object) that is "useful" in a template and is not meant to be rendered in the output.
+
+Let's have a look at the anatomy and life-cycle of a `Apache FreeMarker CLI Tool`
+
+* The meta-data, e.g. class name, is read from `freemarker-cli.properties`
+* It provides a default constructor
+* Its `toString` methods prints a short description
+* It exposes public methods being used directly by the template
+* It is instantiated once and is multi-thread-safe
+
+## Available Tools
+
+The following `tools` are currently implemented
+
+| Tool | Description |
+|-----------------------|-----------------------------------------------------------------------------------------------------------|
+| CSVTool | Process CSV files using [Apache Commons CSV](https://commons.apache.org/proper/commons-csv/) |
+| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
+| ExecTool | Execute command line tools using [Apache Commons Exec](https://commons.apache.org/proper/commons-exec/) |
+| ExcelTool | Process Excels files (XLS, XLSX) using [Apache POI](https://poi.apache.org) |
+| FreeMarkerTool | Expose useful FreeMarker classes |
+| GrokTool | Process text files using [Grok](https://github.com/thekrakken/java-grok) instead of regular expressions |
+| GsonTool | Process JSON files using [GSON](https://github.com/google/gson) |
+| JsonPathTool | Process JSON file using [Java JSON Path](https://github.com/json-path/JsonPath) |
+| JsoupTool | Processing HTML files using [Jsoup](https://jsoup.org) |
+| PropertiesTool | Process JDK properties files |
+| SystemTool | System-related utility methods |
+| UUIDTool | Create UUIDs |
+| XmlTool | Process XML files using [Apache FreeMarker](https://freemarker.apache.org/docs/xgui.html) |
+| YamlTool | Process YAML files using [SnakeYAML](https://bitbucket.org/asomov/snakeyaml/wiki/Home) |
+
+## Advanced Topics
+
+### Auto-closing Resources
+
+The user can create objects which need to be closed later on to avoid excessive resource usage. This is less of a concern for a short-lived CLI application but if many data sources are processed or the code is used in a different context the problem becomes more severe.
+
+The `Excel Tool` provides the following code to keep track of `Closables`
+
+```java
+package org.apache.freemarker.generator.tools.excel;
+
+public class ExcelTool {
+
+ public Workbook parse(DataSource dataSource) {
+ try (InputStream is = dataSource.getUnsafeInputStream()) {
+ final Workbook workbook = WorkbookFactory.create(is);
+ // make sure that the workbook is closed together with the data source
+ return dataSource.addClosable(workbook);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to parse Excel data source: " + dataSource, e);
+ }
+ }
+}
+```
+
+So what is happening here
+
+* The `Workbook` is tracked by the originating `DataSource`
+* The `DataSource` implements the `Closable` interface
+* All `DataSources` are closed automatically when rendering is done.
+
diff --git a/pom.xml b/pom.xml
index 58b46f7..316229f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -123,6 +123,11 @@
<version>1.7.28</version>
</dependency>
<dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-nop</artifactId>
+ <version>1.7.28</version>
+ </dependency>
+ <dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>