You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@freemarker.apache.org by sg...@apache.org on 2020/06/24 20:03:28 UTC
[freemarker-generator] branch FREEMARKER-147 created (now ef2465c)
This is an automated email from the ASF dual-hosted git repository.
sgoeschl pushed a change to branch FREEMARKER-147
in repository https://gitbox.apache.org/repos/asf/freemarker-generator.git.
at ef2465c FREEMARKER-147 Complete Maven site documenation
This branch includes the following new commits:
new ef2465c FREEMARKER-147 Complete Maven site documenation
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[freemarker-generator] 01/01: FREEMARKER-147 Complete Maven site
documenation
Posted by sg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
sgoeschl pushed a commit to branch FREEMARKER-147
in repository https://gitbox.apache.org/repos/asf/freemarker-generator.git
commit ef2465c7bc925e2b8b68c4de3a62956d37c35094
Author: Siegfried Goeschl <si...@gmail.com>
AuthorDate: Wed Jun 24 22:03:12 2020 +0200
FREEMARKER-147 Complete Maven site documenation
---
.../generator/base/template/TemplateOutput.java | 2 +
.../generator/base/template/TemplateSource.java | 2 +-
.../base/template/TemplateTransformations.java | 3 +
.../src/site/markdown/index.md | 4 +-
freemarker-generator-cli/README.md | 2 +-
.../examples/templates/dataframe/example.ftl | 14 +---
.../templates/json/dataframe/github-users.ftl | 20 ++++++
freemarker-generator-cli/pom.xml | 2 +-
.../src/site/markdown/cli/concepts/data-models.md | 8 +--
.../src/site/markdown/cli/concepts/data-sources.md | 2 +-
.../src/site/markdown/cli/concepts/named-uris.md | 2 +-
.../markdown/cli/concepts/template-includes.md | 17 +++++
.../site/markdown/cli/concepts/template-loading.md | 47 +++++++++++++
.../site/markdown/cli/concepts/transformation.md | 4 +-
.../site/markdown/cli/concepts/user-parameters.md | 2 +-
.../src/site/markdown/cli/tools/dataframe.md | 43 +++++++-----
.../src/site/markdown/cli/tools/overview.md | 20 ++++++
.../site/markdown/cli/usage/cli-configuration.md | 77 ++++++++++++++++++++++
.../markdown/cli/usage/transforming-directories.md | 4 +-
.../src/site/markdown/index.md | 6 +-
.../freemarker/generator/cli/ManualTest.java | 4 +-
.../{examples => }/templates/cat.ftl | 1 -
freemarker-generator-tools/pom.xml | 4 +-
.../generator/tools/excel/ExcelTool.java | 2 +-
.../src/site/markdown/index.md | 67 ++++++++++++++++++-
pom.xml | 5 ++
26 files changed, 316 insertions(+), 48 deletions(-)
diff --git a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java
index 1e5cb50..5e8e8a5 100644
--- a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java
+++ b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateOutput.java
@@ -29,6 +29,8 @@ import static java.util.Objects.requireNonNull;
* Information about where to write the ouput of a template. Initially we
* wanted to use a <code>FileWriter</code> but it requires actually an
* existing output file (otherwise a FileNotFound exception is thrown).
+ * An alternative could be a "LazyFileWriter" which creates the file on
+ * the first write access.
*/
public class TemplateOutput {
diff --git a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java
index 42c36b6..4bc86bf 100644
--- a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java
+++ b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateSource.java
@@ -23,7 +23,7 @@ import java.nio.charset.StandardCharsets;
/**
* Information how to load a template. The template is either
- * provided as source code or as template path resolved
+ * provided as literal text or as template path resolved
* by FreeMarker's template loader.
*/
public class TemplateSource {
diff --git a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java
index ed65111..6c09b60 100644
--- a/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java
+++ b/freemarker-generator-base/src/main/java/org/apache/freemarker/generator/base/template/TemplateTransformations.java
@@ -22,6 +22,9 @@ import java.util.List;
import static java.util.Objects.requireNonNull;
+/**
+ * Keeps track of all transformations being executed.
+ */
public class TemplateTransformations {
private final List<TemplateTransformation> templateTransformations;
diff --git a/freemarker-generator-base/src/site/markdown/index.md b/freemarker-generator-base/src/site/markdown/index.md
index 2fd9f95..212345d 100644
--- a/freemarker-generator-base/src/site/markdown/index.md
+++ b/freemarker-generator-base/src/site/markdown/index.md
@@ -1 +1,3 @@
-TBD
\ No newline at end of file
+# Apache FreeMarker CLI Base
+
+This package contains infrastructure code not depending on `Apache FreeMarker` directly since `DataSources` and their creation might be useful for un-related command-line tools.
diff --git a/freemarker-generator-cli/README.md b/freemarker-generator-cli/README.md
index 70179f2..9573c1a 100644
--- a/freemarker-generator-cli/README.md
+++ b/freemarker-generator-cli/README.md
@@ -1523,7 +1523,7 @@ Within the script a FreeMarker data model is set up and passed to the template -
| Entry | Description |
|-----------------------|-----------------------------------------------------------------------------------------------------------|
| CSVTool | Process CSV files using [Apache Commons CSV](https://commons.apache.org/proper/commons-csv/) |
-| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
+| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
| ExecTool | Execute command line tools using [Apache Commons Exec](https://commons.apache.org/proper/commons-exec/) |
| ExcelTool | Process Excels files (XLS, XLSX) using [Apache POI](https://poi.apache.org) |
| DataSources | Helper class to find data sources, e.g. by name, extension or index |
diff --git a/freemarker-generator-cli/examples/templates/dataframe/example.ftl b/freemarker-generator-cli/examples/templates/dataframe/example.ftl
index d1eef7b..9929d21 100644
--- a/freemarker-generator-cli/examples/templates/dataframe/example.ftl
+++ b/freemarker-generator-cli/examples/templates/dataframe/example.ftl
@@ -1,4 +1,3 @@
-<#ftl output_format="plainText">
<#--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -16,7 +15,7 @@
under the License.
-->
<#assign dataSource = DataSources.get(0)>
-<#assign csvParser = CSVTool.parse(dataSource, csvInFormat())>
+<#assign csvParser = CSVTool.parse(dataSource, CSVTool.formats["DATAFRAME"])>
<#assign users = DataFrameTool.fromCSVParser(csvParser)>
Original Data
@@ -45,14 +44,3 @@ ${DataFrameTool.print(users.getColumn("country").transform(DataFrameTool.transfo
Group By Age & Country
=============================================================================
${DataFrameTool.print(users.groupBy("country", "age").sort("country"))}
-<#--------------------------------------------------------------------------->
-<#function csvInFormat>
- <#assign format = CSVTool.formats[CSV_SOURCE_FORMAT!"DEFAULT"]>
- <#assign delimiter = CSVTool.toDelimiter(CSV_SOURCE_DELIMITER!format.getDelimiter())>
- <#assign withHeader = CSV_SOURCE_WITH_HEADER!"false">
- <#assign format = format.withDelimiter(delimiter)>
- <#if withHeader?boolean>
- <#assign format = format.withHeader()>
- </#if>
- <#return format>
-</#function>
diff --git a/freemarker-generator-cli/examples/templates/json/dataframe/github-users.ftl b/freemarker-generator-cli/examples/templates/json/dataframe/github-users.ftl
new file mode 100644
index 0000000..a6547fc
--- /dev/null
+++ b/freemarker-generator-cli/examples/templates/json/dataframe/github-users.ftl
@@ -0,0 +1,20 @@
+<#ftl output_format="plainText" >
+<#--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<#assign json = GsonTool.parse(DataSources.get(0))>
+<#assign dataframe = DataFrameTool.fromMaps(json)>
+${DataFrameTool.print(dataframe)}
diff --git a/freemarker-generator-cli/pom.xml b/freemarker-generator-cli/pom.xml
index 47756a9..55482df 100644
--- a/freemarker-generator-cli/pom.xml
+++ b/freemarker-generator-cli/pom.xml
@@ -177,7 +177,7 @@
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
- <artifactId>slf4j-simple</artifactId>
+ <artifactId>slf4j-nop</artifactId>
</dependency>
<!-- Testing -->
<dependency>
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md
index c05323c..c8707c7 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-models.md
@@ -9,14 +9,14 @@ A `DataModel` is an eagerly loaded `DataSource` available in Apache FreeMarker's
Expose the fields of the JSON data source in FreeMarker's model
```
-bin/freemarker-cli --data-model https://xkcd.com/info.0.json -i '<a href="${img}">${title}</a>'
+bin/freemarker-cli --data-model https://xkcd.com/info.0.json -i '<a href="${img}">${title}</a>'; echo
<a href="https://imgs.xkcd.com/comics/scenario_4.png">Scenario 4</a>
```
Exposed the JSON data source as variable `post` in FreeMarker's model
```
-bin/freemarker-cli --data-model post=https://jsonplaceholder.typicode.com/posts/2 -i 'post title is: ${post.title}'
+bin/freemarker-cli --data-model post=https://jsonplaceholder.typicode.com/posts/2 -i 'post title is: ${post.title}'; echo
post title is: qui est esse
```
@@ -31,14 +31,14 @@ USER=sgoeschl
Expose a single envionment variable in theFreeMarker model
```
-bin/freemarker-cli --data-model NAME=env:///USER -i 'Hello ${NAME}'
+bin/freemarker-cli --data-model NAME=env:///USER -i 'Hello ${NAME}'; echo
Hello sgoeschl
```
Alternatively use the short command line options, e.g.
```
-bin/freemarker-cli -m NAME=env:///USER -i 'Hello ${NAME}!'
+bin/freemarker-cli -m NAME=env:///USER -i 'Hello ${NAME}!'; echo
Hello sgoeschl!
```
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md
index 548a65b..0eafef8 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/data-sources.md
@@ -60,7 +60,7 @@ FreeMarker CLI DataSources
or load them from a directory
```
-bin/freemarker-cli -t templates/info.ftl -s examples/data/
+bin/freemarker-cli -t templates/info.ftl -s examples/data
FreeMarker CLI DataSources
------------------------------------------------------------------------------
[#1], name=combined-access.log, group=default, contentType=text/plain, charset=UTF-8, length=2,068 Bytes
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md
index 1c99b27..693efac 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/named-uris.md
@@ -1,4 +1,4 @@
-# Named URIs
+## Named URIs
Named URIs allow to identify `DataSources` and pass additional information
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/template-includes.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-includes.md
new file mode 100644
index 0000000..8aa0dc3
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-includes.md
@@ -0,0 +1,17 @@
+### Template Includes
+
+When writing more and more templates you find reusable snippets which should not be copy/pasted but organized as [templates includes](https://freemarker.apache.org/docs/ref_directive_include.html).
+
+E.g. `Apache Commons CSV` FTL code is included as shown below
+
+```
+<#import "/templates/lib/commons-csv.ftl" as csv />
+<#assign dataSource = DataSources.get(0)>
+<#assign csvParser = CSVTool.parse(dataSource, csv.sourceFormat())>
+```
+
+What is happening here?
+
+* The template loader picks up `/templates/lib/commons-csv.ftl` from the `Apache FreeMarker CLI` installation directory and imports it as `csv`
+* The imported template exports two methods `csv.sourceFormat()` and `csv.targetFormat()`
+* The exposed methods do the right thing based on the user-supplied parameters and can be re-used for all CSV related templates
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/template-loading.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-loading.md
new file mode 100644
index 0000000..15ce5a3
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/template-loading.md
@@ -0,0 +1,47 @@
+## Template Loading
+
+`Apache FreeMarker CLI` loads templates from multiple locations in the following order
+
+* Current working directory
+* Optional `~/.freemarker-cli` directory
+* `Apache FreeMarker CLI` installation directory
+
+You can check the available template locations easily on the command line
+
+```
+freemarker-cli -t templates/info.ftl
+
+FreeMarker CLI Template Directories
+------------------------------------------------------------------------------
+[#1] /Users/sgoeschl/work/github/apache/freemarker-generator
+[#2] /Users/sgoeschl/.freemarker-cli
+[#3] /Applications/Java/freemarker-cli-2.0.0
+```
+
+### Template Loaders In Action
+
+Let's assume you have `freemarker-cli` in your path and you execute
+
+```
+> cd /
+> which freemarker-cli
+/Applications/Java/freemarker-cli-2.0.0/bin/freemarker-cli
+> freemarker-cli -t templates/cat.ftl https://jsonplaceholder.typicode.com/posts/2
+{
+ "userId": 1,
+ "id": 2,
+ "title": "qui est esse",
+ "body": "est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla"
+}
+```
+
+In your root directory there is no `templates/cat.ftl` but it is picked up from `/Applications/Java/freemarker-cli-2.0.0`
+
+### Literal Templates
+
+You can also directly specify a template to be loaded without relying on template loaders
+
+```
+freemarker-cli -t some-directory/transform.ftl some-directory/contract.csv
+```
+
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md
index 966743b..be6da2f 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/transformation.md
@@ -1,5 +1,7 @@
## Transformation
+The `freemarker-cli` generates text output based on FreeMarker templates and data
+
* A command line invocation requires 1..n `templates` and 0..n `data sources` / `data models`
* A command line invocation is mapped to a series of `transformations`
* The `transformation` consists of exactly one `template`, 0..n `data sources` / `data models` and an `output`
@@ -9,4 +11,4 @@
* an output directory
* When the output is written to a directory
* the structure of the input directory is preserved
- * a "ftl" file externsion is removed
+ * a `ftl file externsion is removed
diff --git a/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md b/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md
index 46f2d2e..b4957f5 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/concepts/user-parameters.md
@@ -1,4 +1,4 @@
-# User-Supplied Parameters
+## User-Supplied Parameters
User-supplied parameters allow to pass additional information to an Apache FreeMarker template
diff --git a/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md b/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md
index 83809ed..c8daa5d 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/tools/dataframe.md
@@ -1,8 +1,8 @@
-# DataFrameTool
+## DataFrameTool
The `DataFrameTool` uses [nRo/DataFrame](https://github.com/nRo/DataFrame) to convert tabular data into a `DataFrame`.
-A `DataFrame` allows declartive filtering and transformation of tabular data, i.e. little code to write.
+A `DataFrame` allows declarative filtering and transformation of tabular data, i.e. less code to write.
Currently the following sources are supported
@@ -10,7 +10,7 @@ Currently the following sources are supported
* JSON arrays represented as collection of maps
* Excel sheets represented as rows
-## CSV Examples
+### Working With CSV
[nRo/DataFrame]("https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv") provides the following CSV file
@@ -27,15 +27,27 @@ Schmitt;30;Germany
Meier;30;Germany
```
-and create a `DateFrame` using the following code
+and create a `DateFrame` using the following code snippet
```
-<#assign cvsFormat = CSVTool.formats["DEFAULT"].withHeader().withDelimiter(';')>
-<#assign csvParser = CSVTool.parse(DataSources.get(0), cvsFormat)>
-<#assign users = DataFrameTool.toDataFrame(csvParser)>
+<#assign dataSource = DataSources.get(0)>
+<#assign csvParser = CSVTool.parse(dataSource, CSVTool.formats["DATAFRAME"])>
+<#assign users = DataFrameTool.fromCSVParser(csvParser)>
+```
+
+The example can be executed by running
+
+```
+bin/freemarker-cli -PCSV_SOURCE_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl examples/data/csv/dataframe.csv
+```
+
+or
+
+```
+bin/freemarker-cli -PCSV_SOURCE_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv
```
-### Select By Age
+#### Select By Age
```
${DataFrameTool.print(users.select("(age > 40)"))}
@@ -53,7 +65,7 @@ which shows
└────────────┴────────────┴────────────┘
```
-### Complex Select & Sort
+#### Complex Select & Sort
Now we want to create a new `DataFrame` by selecting `name` and `country`
@@ -82,7 +94,7 @@ which shows
└────────────┴────────────┴────────────┘
```
-### Count Column Values
+#### Count Column Values
Let's assume we want to count the records for each `country`
@@ -106,7 +118,7 @@ returns the following `DataFrame`
└────────────┴────────────┘
```
-### Group By Age And Country
+#### Group By Age And Country
Let's assume that we want to group the `DataFrame` by `age` and `country`
@@ -136,10 +148,11 @@ which results in
└────────────┴────────────┘
```
-## JSON Examples
+### Working With JSON
-Here we load a `examples/data/json/github-users.json` which represents a tabular data be
-being parsed as a list of maps and print the JSOB as dataframe
+Here we load a `examples/data/json/github-users.json` which represents a tabular
+data being parsed as a list of maps and print the JSON as dataframe. Technically
+it is a list of maps hence we invoke `DataFrameTool.fromMaps()
```
./bin/freemarker-cli \
@@ -165,7 +178,7 @@ being parsed as a list of maps and print the JSOB as dataframe
└────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┴────────────┘
```
-## Excel Examples
+### Working With Excel
Let's transform an Excel Sheet to a `DataFrame` being printed using the following template
diff --git a/freemarker-generator-cli/src/site/markdown/cli/tools/overview.md b/freemarker-generator-cli/src/site/markdown/cli/tools/overview.md
new file mode 100644
index 0000000..e5a64c6
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/tools/overview.md
@@ -0,0 +1,20 @@
+## Overview
+
+The following `tools` are currently implemented
+
+| Tool | Description |
+|-----------------------|-----------------------------------------------------------------------------------------------------------|
+| CSVTool | Process CSV files using [Apache Commons CSV](https://commons.apache.org/proper/commons-csv/) |
+| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
+| ExecTool | Execute command line tools using [Apache Commons Exec](https://commons.apache.org/proper/commons-exec/) |
+| ExcelTool | Process Excels files (XLS, XLSX) using [Apache POI](https://poi.apache.org) |
+| FreeMarkerTool | Expose useful FreeMarker classes |
+| GrokTool | Process text files using [Grok](https://github.com/thekrakken/java-grok) instead of regular expressions |
+| GsonTool | Process JSON files using [GSON](https://github.com/google/gson) |
+| JsonPathTool | Process JSON file using [Java JSON Path](https://github.com/json-path/JsonPath) |
+| JsoupTool | Processing HTML files using [Jsoup](https://jsoup.org) |
+| PropertiesTool | Process JDK properties files |
+| SystemTool | System-related utility methods |
+| UUIDTool | Create UUIDs |
+| XmlTool | Process XML files using [Apache FreeMarker](https://freemarker.apache.org/docs/xgui.html) |
+| YamlTool | Process YAML files using [SnakeYAML](https://bitbucket.org/asomov/snakeyaml/wiki/Home) |
diff --git a/freemarker-generator-cli/src/site/markdown/cli/usage/cli-configuration.md b/freemarker-generator-cli/src/site/markdown/cli/usage/cli-configuration.md
new file mode 100644
index 0000000..499f3c2
--- /dev/null
+++ b/freemarker-generator-cli/src/site/markdown/cli/usage/cli-configuration.md
@@ -0,0 +1,77 @@
+## CLI Configuration
+
+The `Apache FreeMarker CLI` configuration can be tweaked to
+
+* Change the underlying `Apache FreeMarker Configuration`
+* Instantiate custom tools
+* Storing user-specific templates
+
+### Apache FreeMarker CLI Properties File
+
+`Apache FreeMarker CLI` reads the `conf/freemarker-cli.properties`
+
+```
+#############################################################################
+# General FreeMarker Configuration
+# See https://freemarker.apache.org/docs/api/freemarker/template/Configuration.html#setSetting-java.lang.String-java.lang.String-
+#############################################################################
+# freemarker.configuration.setting.locale=JVM default
+
+#############################################################################
+# Configure FreeMarker Tools (name -> implementation class)
+#############################################################################
+freemarker.tools.CSVTool=org.apache.freemarker.generator.tools.commonscsv.CommonsCSVTool
+freemarker.tools.DataFrameTool=org.apache.freemarker.generator.tools.dataframe.DataFrameTool
+freemarker.tools.ExcelTool=org.apache.freemarker.generator.tools.excel.ExcelTool
+freemarker.tools.ExecTool=org.apache.freemarker.generator.tools.commonsexec.CommonsExecTool
+freemarker.tools.FreeMarkerTool=org.apache.freemarker.generator.tools.freemarker.FreeMarkerTool
+freemarker.tools.GrokTool=org.apache.freemarker.generator.tools.grok.GrokTool
+freemarker.tools.GsonTool=org.apache.freemarker.generator.tools.gson.GsonTool
+freemarker.tools.JsonPathTool=org.apache.freemarker.generator.tools.jsonpath.JsonPathTool
+freemarker.tools.JsoupTool=org.apache.freemarker.generator.tools.jsoup.JsoupTool
+freemarker.tools.PropertiesTool=org.apache.freemarker.generator.tools.properties.PropertiesTool
+freemarker.tools.SystemTool=org.apache.freemarker.generator.tools.system.SystemTool
+freemarker.tools.UUIDTool=org.apache.freemarker.generator.tools.uuid.UUIDTool
+freemarker.tools.XmlTool=org.apache.freemarker.generator.tools.xml.XmlTool
+freemarker.tools.YamlTool=org.apache.freemarker.generator.tools.snakeyaml.SnakeYamlTool
+```
+
+Changing this file allows to tweak the underlying `Apache FreeMarker Configuration` and add custom tools.
+
+### Storing User-Specific Templates
+
+Over the time you will accumalate more and more `Apache FreeMarker` templates - some of them are stored within a project but some of the more general might be free-floating and you don't want to store them in the installation directory.
+
+To give those free-floating templates a home `Apache FreeMarker CLI` tries to read templates from `~/freemarker-cli`, e.g.
+
+```
+tree ~/.freemarker-cli/
+/Users/sgoeschl/.freemarker-cli/
+`-- templates
+ `-- json
+ |-- confluence
+ | |-- aws
+ | | `-- describe.ftl
+ |-- csv
+ | `-- swagger-endpoints.ftl
+ |-- ftl
+ | `-- customer-user-products.ftl
+ |-- html
+ | `-- customer-user-products.ftl
+ `-- md
+ `-- customer-user-products.ftl
+```
+
+If a `~/freemarker-cli` is found it will be automatically added to the `FreeMarker Template Loader` (for more information see https://freemarker.apache.org/docs/pgui_config_templateloading.html)
+
+You can easily check this, e.g.
+
+```
+> freemarker-cli -t templates/info.ftl
+
+FreeMarker CLI Template Directories
+------------------------------------------------------------------------------
+[#1] /Users/sgoeschl
+[#2] /Users/sgoeschl/.freemarker-cli
+[#3] /Applications/Java/freemarker-cli-2.0.0
+```
diff --git a/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md b/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md
index 9f6a36f..de95e22 100644
--- a/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md
+++ b/freemarker-generator-cli/src/site/markdown/cli/usage/transforming-directories.md
@@ -1,6 +1,6 @@
## Transforming Directories
-FreeMarker CLI supports the transformation of directories
+`Apache FreeMarker CLI` supports the transformation of directories
* Transform an input directory recursively into an output directory
* If a template has a ".ftl" extension this extension will be removed after processing
@@ -13,7 +13,7 @@ The following sample files are used
* template/nginx/nginx.conf.ftl
```
-appassembler> tree examples/data/template/
+tree examples/data/template/
examples/data/template/
|-- application.properties
`-- nginx
diff --git a/freemarker-generator-cli/src/site/markdown/index.md b/freemarker-generator-cli/src/site/markdown/index.md
index 2eb0dfd..d4526e9 100644
--- a/freemarker-generator-cli/src/site/markdown/index.md
+++ b/freemarker-generator-cli/src/site/markdown/index.md
@@ -6,12 +6,16 @@
* [Data Sources](cli/concepts/data-sources.html)
* [Data Models](cli/concepts/data-models.html)
* [User-Supplied Parameters](cli/concepts/user-parameters.html)
+* [Template Loading](cli/concepts/template-loading.html)
+* [Template Includes](cli/concepts/template-includes.html)
* [Transformation](cli/concepts/transformation.html)
-### Tools
+### Working With Tools
+* [Overview](cli/tools/overview.html)
* [DataFrameTool](cli/tools/dataframe.html)
### Usage
+* [CLI Configuration](cli/usage/cli-configuration.html)
* [Transforming Directories](cli/usage/transforming-directories.html)
diff --git a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
index faa6080..98d1673 100644
--- a/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
+++ b/freemarker-generator-cli/src/test/java/org/apache/freemarker/generator/cli/ManualTest.java
@@ -25,7 +25,9 @@ public class ManualTest {
private static final String SPACE = " ";
// private static final String CMD = "-PCSV_SOURCE_FORMAT=DATAFRAME -t examples/templates/dataframe/example.ftl https://raw.githubusercontent.com/nRo/DataFrame/master/src/test/resources/users.csv";
- private static final String CMD = "-PCSV_SOURCE_WITH_HEADER=true -PCSV_SOURCE_FORMAT=DEFAULT -PCSV_TARGET_FORMAT=EXCEL -t templates/csv/md/transform.ftl examples/data/csv/contract.csv";
+ // private static final String CMD = "-PCSV_SOURCE_WITH_HEADER=true -PCSV_SOURCE_FORMAT=DEFAULT -PCSV_TARGET_FORMAT=EXCEL -t templates/csv/md/transform.ftl examples/data/csv/contract.csv";
+ private static final String CMD = "-t examples/templates/json/dataframe/github-users.ftl examples/data/json/github-users.json";
+
public static void main(String[] args) {
Main.execute(toArgs(CMD));
diff --git a/freemarker-generator-cli/examples/templates/cat.ftl b/freemarker-generator-cli/templates/cat.ftl
similarity index 94%
rename from freemarker-generator-cli/examples/templates/cat.ftl
rename to freemarker-generator-cli/templates/cat.ftl
index a878261..c912c12 100644
--- a/freemarker-generator-cli/examples/templates/cat.ftl
+++ b/freemarker-generator-cli/templates/cat.ftl
@@ -1,4 +1,3 @@
-<#ftl output_format="plainText" strip_whitespace=true>
<#--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
diff --git a/freemarker-generator-tools/pom.xml b/freemarker-generator-tools/pom.xml
index 57c6b59..256987a 100644
--- a/freemarker-generator-tools/pom.xml
+++ b/freemarker-generator-tools/pom.xml
@@ -122,10 +122,12 @@
<reporting>
<plugins>
<plugin>
- <groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
</plugin>
<plugin>
+ <artifactId>maven-jxr-plugin</artifactId>
+ </plugin>
+ <plugin>
<groupId>org.apache.rat</groupId>
<artifactId>apache-rat-plugin</artifactId>
<configuration>
diff --git a/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java b/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java
index b5ead1d..08133fc 100644
--- a/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java
+++ b/freemarker-generator-tools/src/main/java/org/apache/freemarker/generator/tools/excel/ExcelTool.java
@@ -53,7 +53,7 @@ public class ExcelTool {
// make sure that the workbook is closed together with the data source
return dataSource.addClosable(workbook);
} catch (IOException e) {
- throw new RuntimeException("Failed to parse Ecxel data source: " + dataSource, e);
+ throw new RuntimeException("Failed to parse Excel data source: " + dataSource, e);
}
}
diff --git a/freemarker-generator-tools/src/site/markdown/index.md b/freemarker-generator-tools/src/site/markdown/index.md
index 2fd9f95..19d0bb2 100644
--- a/freemarker-generator-tools/src/site/markdown/index.md
+++ b/freemarker-generator-tools/src/site/markdown/index.md
@@ -1 +1,66 @@
-TBD
\ No newline at end of file
+# Apache FreeMarker CLI Tools
+
+## Overview
+
+The implementation of the `Apache FreeMarker CLI Tools` was inspired by [Apache Velocity Tools](https://velocity.apache.org/tools/devel/) - a `tool` is just a POJO (plain old Java object) that is "useful" in a template and is not meant to be rendered in the output.
+
+Let's have a look at the anatomy and life-cycle of a `Apache FreeMarker CLI Tool`
+
+* The meta-data, e.g. class name, is read from `freemarker-cli.properties`
+* It provides a default constructor
+* Its `toString` methods prints a short description
+* It exposes public methods being used directly by the template
+* It is instantiated once and is multi-thread-safe
+
+## Available Tools
+
+The following `tools` are currently implemented
+
+| Tool | Description |
+|-----------------------|-----------------------------------------------------------------------------------------------------------|
+| CSVTool | Process CSV files using [Apache Commons CSV](https://commons.apache.org/proper/commons-csv/) |
+| DataFrameTool | Bridge to [nRo/DataFrame](https://github.com/nRo/DataFrame) |
+| ExecTool | Execute command line tools using [Apache Commons Exec](https://commons.apache.org/proper/commons-exec/) |
+| ExcelTool | Process Excels files (XLS, XLSX) using [Apache POI](https://poi.apache.org) |
+| FreeMarkerTool | Expose useful FreeMarker classes |
+| GrokTool | Process text files using [Grok](https://github.com/thekrakken/java-grok) instead of regular expressions |
+| GsonTool | Process JSON files using [GSON](https://github.com/google/gson) |
+| JsonPathTool | Process JSON file using [Java JSON Path](https://github.com/json-path/JsonPath) |
+| JsoupTool | Processing HTML files using [Jsoup](https://jsoup.org) |
+| PropertiesTool | Process JDK properties files |
+| SystemTool | System-related utility methods |
+| UUIDTool | Create UUIDs |
+| XmlTool | Process XML files using [Apache FreeMarker](https://freemarker.apache.org/docs/xgui.html) |
+| YamlTool | Process YAML files using [SnakeYAML](https://bitbucket.org/asomov/snakeyaml/wiki/Home) |
+
+## Advanced Topics
+
+### Auto-closing Resources
+
+The user can create objects which need to be closed later on to avoid excessive resource usage. This is less of a concern for a short-lived CLI application but if many data sources are processed or the code is used in a different context the problem becomes more severe.
+
+The `Excel Tool` provides the following code to keep track of `Closables`
+
+```java
+package org.apache.freemarker.generator.tools.excel;
+
+public class ExcelTool {
+
+ public Workbook parse(DataSource dataSource) {
+ try (InputStream is = dataSource.getUnsafeInputStream()) {
+ final Workbook workbook = WorkbookFactory.create(is);
+ // make sure that the workbook is closed together with the data source
+ return dataSource.addClosable(workbook);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to parse Excel data source: " + dataSource, e);
+ }
+ }
+}
+```
+
+So what is happening here
+
+* The `Workbook` is tracked by the originating `DataSource`
+* The `DataSource` implements the `Closable` interface
+* All `DataSources` are closed automatically when rendering is done.
+
diff --git a/pom.xml b/pom.xml
index 58b46f7..316229f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -123,6 +123,11 @@
<version>1.7.28</version>
</dependency>
<dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-nop</artifactId>
+ <version>1.7.28</version>
+ </dependency>
+ <dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>