You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by bz...@apache.org on 2016/11/16 04:31:52 UTC
zeppelin git commit: [ZEPPELIN-1614] Remove markdown4j dep
Repository: zeppelin
Updated Branches:
refs/heads/master 0960dbc25 -> ab2cdfd38
[ZEPPELIN-1614] Remove markdown4j dep
### What is this PR for?
Remove unmanaged, old library markdown4j dep which exists just for Websequence and YUML plugins.
(related to https://github.com/apache/zeppelin/pull/1384)
By adding Websequence and YUML plugins to pegdown
- Removing markdown4j dependency which is unmanaged library currently.
- Addtionally, we can remove `markdown.parser.type` options in **markdown interpreter**
- Fixing some bugs in Websequence and YUML plugins
- Enable others to add more plugins using pegdown sytnax.
### What type of PR is it?
Improvement
### Todos
Nothing
### What is the Jira issue?
[JIRA - ZEPPELIN-1614](https://issues.apache.org/jira/browse/ZEPPELIN-1614)
### How should this be tested?
Some functional tests are included.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? - YES
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - YES
Author: 1ambda <1a...@gmail.com>
Closes #1594 from 1ambda/feat/remove-markdown4j-dep and squashes the following commits:
5af1607 [1ambda] fix: Failed tests in InterpreterRestApiTest
c57fdcb [1ambda] docs: Update markdown.md
5c62236 [1ambda] docs: Update upgrade.md for '%md'
a1e779d [1ambda] style: Use zeppelin checkstyle.xml
13e0dc4 [1ambda] Update: interpreter setting and docs
de3549d [1ambda] chore: Cleanup duplicated markdown4j license
7c5d41e [1ambda] fix: Parse style param optionally in webseq
8831ca1 [1ambda] fix: Wrap exceptions in catch stmt
9268695 [1ambda] Revert "fix: Cleanup unused Markdown4j Parser"
33fb800 [1ambda] Revert "docs: Remove markdown.parser.type option"
fddc459 [1ambda] Revert "chore: Remove markdown4j dep and update license"
a59ebbd [1ambda] Revert "fix: Set {} to avoid 503"
4e48933 [1ambda] Revert "fix: Parse style param optionally in webseq"
8cfb2c8 [1ambda] Revert "fix: style and misspell in docs"
73956e0 [1ambda] Revert "fix: Propagate exception in YUML plugin"
1b7787f [1ambda] fix: Propagate exception in YUML plugin
c656d08 [1ambda] fix: style and misspell in docs
dc4f110 [1ambda] fix: Parse style param optionally in webseq
b43e14e [1ambda] fix: Set {} to avoid 503
c48cc53 [1ambda] chore: Remove markdown4j dep and update license
81fdfcc [1ambda] docs: Remove markdown.parser.type option
cf19f0b [1ambda] fix: Cleanup unused Markdown4j Parser
98b2809 [1ambda] fix: Add missing docs
3e9716d [1ambda] feat: Yuml markdown plugin
3247c67 [1ambda] feat: Support webseq markdown plugin
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/ab2cdfd3
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/ab2cdfd3
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/ab2cdfd3
Branch: refs/heads/master
Commit: ab2cdfd384c0cc96a1ef10ff224d2a6b8a5fce06
Parents: 0960dbc
Author: 1ambda <1a...@gmail.com>
Authored: Thu Nov 10 18:03:27 2016 +0900
Committer: Alexander Bezzubov <bz...@apache.org>
Committed: Wed Nov 16 05:31:45 2016 +0100
----------------------------------------------------------------------
STYLE.md | 4 +-
.../markdown-example-markdown4j-parser.png | Bin 264309 -> 0 bytes
.../markdown-example-pegdown-parser-plugins.png | Bin 0 -> 264309 bytes
.../docs-img/markdown-interpreter-setting.png | Bin 67890 -> 79032 bytes
docs/install/upgrade.md | 1 +
docs/interpreter/markdown.md | 38 ++---
.../org/apache/zeppelin/markdown/Markdown.java | 14 +-
.../zeppelin/markdown/Markdown4jParser.java | 4 +-
.../zeppelin/markdown/MarkdownParser.java | 4 +-
.../org/apache/zeppelin/markdown/ParamVar.java | 41 +++++
.../apache/zeppelin/markdown/PegdownParser.java | 30 ++--
.../markdown/PegdownWebSequencelPlugin.java | 149 +++++++++++++++++++
.../zeppelin/markdown/PegdownYumlPlugin.java | 143 ++++++++++++++++++
.../src/main/resources/interpreter-setting.json | 4 +-
.../zeppelin/markdown/PegdownParserTest.java | 36 ++++-
zeppelin-distribution/src/bin_license/LICENSE | 1 -
.../zeppelin/rest/InterpreterRestApiTest.java | 11 +-
17 files changed, 436 insertions(+), 44 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/STYLE.md
----------------------------------------------------------------------
diff --git a/STYLE.md b/STYLE.md
index 9a0242e..db6a40c 100644
--- a/STYLE.md
+++ b/STYLE.md
@@ -7,7 +7,7 @@ app/styles/looknfeel
Overall look and theme of the Zeppelin notebook page can be customized here.
### Code Syntax Highlighting
-There are two parts to code highlighting. First, Zeppelin uses the Ace Editor for its note paragraphs. Color style for this can be changed by setting theme on the editor instance. Second, Zeppelin's Markdown interpreter calls markdown4j to emit HTML, and such content may contain <pre><code> tags that can be consumed by Highlight.js.
+There are two parts to code highlighting. First, Zeppelin uses the Ace Editor for its note paragraphs. Color style for this can be changed by setting theme on the editor instance. Second, Zeppelin's Markdown interpreter calls pegdown parser to emit HTML, and such content may contain <pre><code> tags that can be consumed by Highlight.js.
#### Theme on Ace Editor
app/scripts/controllers/paragraph.js
@@ -16,7 +16,7 @@ Call setTheme on the editor with the theme path/name.
[List of themes on GitHub](https://github.com/ajaxorg/ace/tree/master/lib/ace/theme)
#### Style for Markdown Code Blocks
-Highlight.js parses and converts <pre><code> blocks from markdown4j into keywords and language syntax with proper styles. It also attempts to infer the best fitting language if it is not provided. The visual style can be changed by simply including the desired [stylesheet](https://github.com/components/highlightjs/tree/master/styles) into app/index.html. See the next section on build.
+Highlight.js parses and converts <pre><code> blocks from pegdown parser into keywords and language syntax with proper styles. It also attempts to infer the best fitting language if it is not provided. The visual style can be changed by simply including the desired [stylesheet](https://github.com/components/highlightjs/tree/master/styles) into app/index.html. See the next section on build.
Note that code block background color is overriden in app/styles/notebook.css (look for .paragraph .tableDisplay .hljs).
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/docs/assets/themes/zeppelin/img/docs-img/markdown-example-markdown4j-parser.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/markdown-example-markdown4j-parser.png b/docs/assets/themes/zeppelin/img/docs-img/markdown-example-markdown4j-parser.png
deleted file mode 100644
index e3455e3..0000000
Binary files a/docs/assets/themes/zeppelin/img/docs-img/markdown-example-markdown4j-parser.png and /dev/null differ
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/docs/assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser-plugins.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser-plugins.png b/docs/assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser-plugins.png
new file mode 100644
index 0000000..e3455e3
Binary files /dev/null and b/docs/assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser-plugins.png differ
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/docs/assets/themes/zeppelin/img/docs-img/markdown-interpreter-setting.png
----------------------------------------------------------------------
diff --git a/docs/assets/themes/zeppelin/img/docs-img/markdown-interpreter-setting.png b/docs/assets/themes/zeppelin/img/docs-img/markdown-interpreter-setting.png
index 33c13ec..1d42779 100644
Binary files a/docs/assets/themes/zeppelin/img/docs-img/markdown-interpreter-setting.png and b/docs/assets/themes/zeppelin/img/docs-img/markdown-interpreter-setting.png differ
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/docs/install/upgrade.md
----------------------------------------------------------------------
diff --git a/docs/install/upgrade.md b/docs/install/upgrade.md
index a203f6f..a29ed8d 100644
--- a/docs/install/upgrade.md
+++ b/docs/install/upgrade.md
@@ -53,3 +53,4 @@ So, copying `notebook` and `conf` directory should be enough.
- Mapping from `%jdbc(prefix)` to `%prefix` is no longer available. Instead, you can use %[interpreter alias] with multiple interpreter setttings on GUI.
- Usage of `ZEPPELIN_PORT` is not supported in ssl mode. Instead use `ZEPPELIN_SSL_PORT` to configure the ssl port. Value from `ZEPPELIN_PORT` is used only when `ZEPPELIN_SSL` is set to `false`.
- The support on Spark 1.1.x to 1.3.x is deprecated.
+ - From 0.7, we uses `pegdown` as the `markdown.parser.type` option for the `%md` interpreter. Rendered markdown might be different from what you expected
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/docs/interpreter/markdown.md
----------------------------------------------------------------------
diff --git a/docs/interpreter/markdown.md b/docs/interpreter/markdown.md
index 257e146..46fd170 100644
--- a/docs/interpreter/markdown.md
+++ b/docs/interpreter/markdown.md
@@ -25,14 +25,24 @@ limitations under the License.
## Overview
[Markdown](http://daringfireball.net/projects/markdown/) is a plain text formatting syntax designed so that it can be converted to HTML.
-Apache Zeppelin uses [markdown4j](https://github.com/jdcasey/markdown4j) and [pegdown](https://github.com/sirthias/pegdown) as markdown parsers.
+Apache Zeppelin uses [pegdown](https://github.com/sirthias/pegdown) and [markdown4j](https://github.com/jdcasey/markdown4j) as markdown parsers.
In Zeppelin notebook, you can use ` %md ` in the beginning of a paragraph to invoke the Markdown interpreter and generate static html from Markdown plain text.
-In Zeppelin, Markdown interpreter is enabled by default and uses the [markdown4j](https://github.com/jdcasey/markdown4j) parser.
+In Zeppelin, Markdown interpreter is enabled by default and uses the [pegdown](https://github.com/sirthias/pegdown) parser.
<img src="../assets/themes/zeppelin/img/docs-img/markdown-interpreter-setting.png" width="60%" />
+## Example
+
+The following example demonstrates the basic usage of Markdown in a Zeppelin notebook.
+
+<img src="../assets/themes/zeppelin/img/docs-img/markdown-example.png" width="70%" />
+
+## Mathematical expression
+
+Markdown interpreter leverages %html display system internally. That means you can mix mathematical expressions with markdown syntax. For more information, please see [Mathematical Expression](../displaysystem/basicdisplaysystem.html#mathematical-expressions) section.
+
## Configuration
<table class="table-configuration">
<tr>
@@ -42,31 +52,25 @@ In Zeppelin, Markdown interpreter is enabled by default and uses the [markdown4j
</tr>
<tr>
<td>markdown.parser.type</td>
- <td>markdown4j</td>
- <td>Markdown Parser Type. <br/> Available values: markdown4j, pegdown.</td>
+ <td>pegdown</td>
+ <td>Markdown Parser Type. <br/> Available values: pegdown, markdown4j.</td>
</tr>
</table>
-## Example
-
-The following example demonstrates the basic usage of Markdown in a Zeppelin notebook.
-<img src="../assets/themes/zeppelin/img/docs-img/markdown-example.png" width="70%" />
+### Pegdown Parser
-## Mathematical expression
+`pegdown` parser provides github flavored markdown.
-Markdown interpreter leverages %html display system internally. That means you can mix mathematical expressions with markdown syntax. For more information, please see [Mathematical Expression](../displaysystem/basicdisplaysystem.html#mathematical-expressions) section.
+<img src="../assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser.png" width="70%" />
+`pegdown` parser provides [YUML](http://yuml.me/) and [Websequence](https://www.websequencediagrams.com/) plugins also.
+<img src="../assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser-plugins.png" width="70%" />
### Markdown4j Parser
-`markdown4j` parser provides [YUML](http://yuml.me/) and [Websequence](https://www.websequencediagrams.com/) extensions
-
-<img src="../assets/themes/zeppelin/img/docs-img/markdown-example-markdown4j-parser.png" width="70%" />
-
-### Pegdown Parser
+Since pegdown parser is more accurate and provides much more markdown syntax
+`markdown4j` option might be removed later. But keep this parser for the backward compatibility.
-`pegdown` parser provides github flavored markdown.
-<img src="../assets/themes/zeppelin/img/docs-img/markdown-example-pegdown-parser.png" width="70%" />
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
----------------------------------------------------------------------
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java b/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
index a811eab..c908a54 100644
--- a/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
@@ -33,13 +33,17 @@ import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-/** MarkdownInterpreter interpreter for Zeppelin. */
+/**
+ * MarkdownInterpreter interpreter for Zeppelin.
+ */
public class Markdown extends Interpreter {
private static final Logger LOGGER = LoggerFactory.getLogger(Markdown.class);
private MarkdownParser parser;
- /** Markdown Parser Type. */
+ /**
+ * Markdown Parser Type.
+ */
public enum MarkdownParserType {
PEGDOWN {
@Override
@@ -82,7 +86,8 @@ public class Markdown extends Interpreter {
}
@Override
- public void close() {}
+ public void close() {
+ }
@Override
public InterpreterResult interpret(String markdownText, InterpreterContext interpreterContext) {
@@ -99,7 +104,8 @@ public class Markdown extends Interpreter {
}
@Override
- public void cancel(InterpreterContext context) {}
+ public void cancel(InterpreterContext context) {
+ }
@Override
public FormType getFormType() {
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown4jParser.java
----------------------------------------------------------------------
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown4jParser.java b/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown4jParser.java
index 68ca41b..78f8137 100644
--- a/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown4jParser.java
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/Markdown4jParser.java
@@ -23,7 +23,9 @@ import org.slf4j.LoggerFactory;
import java.io.IOException;
-/** Markdown Parser using markdown4j processor . */
+/**
+ * Markdown Parser using markdown4j processor.
+ */
public class Markdown4jParser implements MarkdownParser {
private Markdown4jProcessor processor;
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/java/org/apache/zeppelin/markdown/MarkdownParser.java
----------------------------------------------------------------------
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/MarkdownParser.java b/markdown/src/main/java/org/apache/zeppelin/markdown/MarkdownParser.java
index 056ca26..2f8717e 100644
--- a/markdown/src/main/java/org/apache/zeppelin/markdown/MarkdownParser.java
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/MarkdownParser.java
@@ -17,7 +17,9 @@
package org.apache.zeppelin.markdown;
-/** Abstract Markdown Parser. */
+/**
+ * Abstract Markdown Parser.
+ */
public interface MarkdownParser {
String render(String markdownText);
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/java/org/apache/zeppelin/markdown/ParamVar.java
----------------------------------------------------------------------
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/ParamVar.java b/markdown/src/main/java/org/apache/zeppelin/markdown/ParamVar.java
new file mode 100644
index 0000000..14828e0
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/ParamVar.java
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import org.parboiled.support.Var;
+
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Implementation of Var to support parameter parsing.
+ *
+ * @param <K> Key
+ * @param <V> Value
+ */
+public class ParamVar<K, V> extends Var<Map<K, V>> {
+
+ public ParamVar() {
+ super(new HashMap<K, V>());
+ }
+
+ public boolean put(K key, V value) {
+ get().put(key, value);
+ return true;
+ }
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownParser.java
----------------------------------------------------------------------
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownParser.java b/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownParser.java
index bbadd97..61237e4 100644
--- a/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownParser.java
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownParser.java
@@ -19,17 +19,23 @@ package org.apache.zeppelin.markdown;
import org.pegdown.Extensions;
import org.pegdown.PegDownProcessor;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
+import org.pegdown.plugins.PegDownPlugins;
-/** Markdown Parser using pegdown processor. */
+/**
+ * Markdown Parser using pegdown processor.
+ */
public class PegdownParser implements MarkdownParser {
private PegDownProcessor processor;
+ public static final long PARSING_TIMEOUT_AS_MILLIS = 5000;
+ public static final int OPTIONS = Extensions.ALL_WITH_OPTIONALS - Extensions.ANCHORLINKS;
+
public PegdownParser() {
- int pegdownOptions = Extensions.ALL_WITH_OPTIONALS - Extensions.ANCHORLINKS;
- int parsingTimeoutAsMillis = 5000;
- processor = new PegDownProcessor(pegdownOptions, parsingTimeoutAsMillis);
+ PegDownPlugins plugins = new PegDownPlugins.Builder()
+ .withPlugin(PegdownYumlPlugin.class)
+ .withPlugin(PegdownWebSequencelPlugin.class)
+ .build();
+ processor = new PegDownProcessor(OPTIONS, PARSING_TIMEOUT_AS_MILLIS, plugins);
}
@Override
@@ -45,12 +51,14 @@ public class PegdownParser implements MarkdownParser {
return html;
}
- /** wrap with markdown class div to styling DOM using css. */
+ /**
+ * wrap with markdown class div to styling DOM using css.
+ */
public static String wrapWithMarkdownClassDiv(String html) {
return new StringBuilder()
- .append("<div class=\"markdown-body\">\n")
- .append(html)
- .append("\n</div>")
- .toString();
+ .append("<div class=\"markdown-body\">\n")
+ .append(html)
+ .append("\n</div>")
+ .toString();
}
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownWebSequencelPlugin.java
----------------------------------------------------------------------
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownWebSequencelPlugin.java b/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownWebSequencelPlugin.java
new file mode 100644
index 0000000..8a5d6ea
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownWebSequencelPlugin.java
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.parboiled.BaseParser;
+import org.parboiled.Rule;
+import org.parboiled.support.StringBuilderVar;
+import org.pegdown.Parser;
+import org.pegdown.ast.ExpImageNode;
+import org.pegdown.ast.TextNode;
+import org.pegdown.plugins.BlockPluginParser;
+import org.pegdown.plugins.PegDownPlugins;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.io.OutputStreamWriter;
+import java.net.URL;
+import java.net.URLConnection;
+import java.net.URLEncoder;
+import java.nio.charset.StandardCharsets;
+
+/**
+ * Pegdown plugin for Websequence diagram
+ */
+public class PegdownWebSequencelPlugin extends Parser implements BlockPluginParser {
+
+ private static final String WEBSEQ_URL = "http://www.websequencediagrams.com";
+
+ public PegdownWebSequencelPlugin() {
+ super(PegdownParser.OPTIONS,
+ PegdownParser.PARSING_TIMEOUT_AS_MILLIS,
+ DefaultParseRunnerProvider);
+ }
+
+ public PegdownWebSequencelPlugin(Integer opts, Long millis, ParseRunnerProvider provider,
+ PegDownPlugins plugins) {
+ super(opts, millis, provider, plugins);
+ }
+
+ public static final String TAG = "%%%";
+
+ Rule StartMarker() {
+ return Sequence(Spn1(), TAG, Sp(), "sequence", Sp());
+ }
+
+ String EndMarker() {
+ return TAG;
+ }
+
+ Rule Body() {
+ return OneOrMore(TestNot(TAG), BaseParser.ANY);
+ }
+
+ Rule BlockRule() {
+ StringBuilderVar style = new StringBuilderVar();
+ StringBuilderVar body = new StringBuilderVar();
+
+ return NodeSequence(
+ StartMarker(),
+ Optional(
+ String("style="),
+ Sequence(OneOrMore(Letter()), style.append(match()), Spn1())),
+ Sequence(Body(), body.append(match())),
+ EndMarker(),
+ push(
+ new ExpImageNode("title",
+ createWebsequenceUrl(style.getString(), body.getString()),
+ new TextNode("")))
+ );
+ }
+
+ public static String createWebsequenceUrl(String style,
+ String content) {
+
+ style = StringUtils.defaultString(style, "default");
+
+ OutputStreamWriter writer = null;
+ BufferedReader reader = null;
+
+ String webSeqUrl = "";
+
+ try {
+ String query = new StringBuilder()
+ .append("style=")
+ .append(style)
+ .append("&message=")
+ .append(URLEncoder.encode(content, "UTF-8"))
+ .append("&apiVersion=1")
+ .toString();
+
+ URL url = new URL(WEBSEQ_URL);
+ URLConnection conn = url.openConnection();
+ conn.setDoOutput(true);
+ writer = new OutputStreamWriter(conn.getOutputStream(), StandardCharsets.UTF_8);
+ writer.write(query);
+ writer.flush();
+
+ StringBuilder response = new StringBuilder();
+ reader = new BufferedReader(
+ new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8));
+ String line;
+ while ((line = reader.readLine()) != null) {
+ response.append(line);
+ }
+
+ writer.close();
+ reader.close();
+
+ String json = response.toString();
+
+ int start = json.indexOf("?png=");
+ int end = json.indexOf("\"", start);
+
+ if (start != -1 && end != -1) {
+ webSeqUrl = WEBSEQ_URL + "/" + json.substring(start, end);
+ }
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to get proper response from websequencediagrams.com", e);
+ } finally {
+ IOUtils.closeQuietly(writer);
+ IOUtils.closeQuietly(reader);
+ }
+
+ return webSeqUrl;
+ }
+
+ @Override
+ public Rule[] blockPluginRules() {
+ return new Rule[]{BlockRule()};
+ }
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownYumlPlugin.java
----------------------------------------------------------------------
diff --git a/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownYumlPlugin.java b/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownYumlPlugin.java
new file mode 100644
index 0000000..083bc5b
--- /dev/null
+++ b/markdown/src/main/java/org/apache/zeppelin/markdown/PegdownYumlPlugin.java
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zeppelin.markdown;
+
+import static org.apache.commons.lang3.StringUtils.defaultString;
+
+import org.parboiled.BaseParser;
+import org.parboiled.Rule;
+import org.parboiled.support.StringBuilderVar;
+import org.pegdown.Parser;
+import org.pegdown.ast.ExpImageNode;
+import org.pegdown.ast.TextNode;
+import org.pegdown.plugins.BlockPluginParser;
+import org.pegdown.plugins.PegDownPlugins;
+
+import java.io.UnsupportedEncodingException;
+import java.net.URLEncoder;
+import java.util.Map;
+
+/**
+ * Pegdown plugin for YUML
+ */
+public class PegdownYumlPlugin extends Parser implements BlockPluginParser {
+
+ public PegdownYumlPlugin() {
+ super(PegdownParser.OPTIONS,
+ PegdownParser.PARSING_TIMEOUT_AS_MILLIS,
+ DefaultParseRunnerProvider);
+ }
+
+ public PegdownYumlPlugin(Integer options,
+ Long maxParsingTimeInMillis,
+ ParseRunnerProvider parseRunnerProvider,
+ PegDownPlugins plugins) {
+ super(options, maxParsingTimeInMillis, parseRunnerProvider, plugins);
+ }
+
+ public static final String TAG = "%%%";
+
+ Rule StartMarker() {
+ return Sequence(Spn1(), TAG, Sp(), "yuml", Sp());
+ }
+
+ String EndMarker() {
+ return TAG;
+ }
+
+ Rule ParameterName() {
+ return FirstOf("type", "style", "scale", "format", "dir");
+ }
+
+ Rule Body() {
+ return OneOrMore(TestNot(TAG), BaseParser.ANY);
+ }
+
+ Rule BlockRule() {
+ ParamVar<String, String> params = new ParamVar<String, String>();
+ StringBuilderVar name = new StringBuilderVar();
+ StringBuilderVar value = new StringBuilderVar();
+ StringBuilderVar body = new StringBuilderVar();
+
+ return NodeSequence(
+ StartMarker(),
+ ZeroOrMore(
+ Sequence(
+ ParameterName(), name.append(match()),
+ String("="),
+ OneOrMore(Alphanumeric()), value.append(match())),
+ Sp(),
+ params.put(name.getString(), value.getString()),
+ name.clear(), value.clear()),
+ Body(),
+ body.append(match()),
+ EndMarker(),
+ push(
+ new ExpImageNode(
+ "title", createYumlUrl(params.get(), body.getString()), new TextNode("")))
+ );
+ }
+
+ public static String createYumlUrl(Map<String, String> params, String body) {
+ StringBuilder inlined = new StringBuilder();
+ for (String line : body.split("\\r?\\n")) {
+ line = line.trim();
+ if (line.length() > 0) {
+ if (inlined.length() > 0) {
+ inlined.append(", ");
+ }
+ inlined.append(line);
+ }
+ }
+
+ String encodedBody = null;
+ try {
+ encodedBody = URLEncoder.encode(inlined.toString(), "UTF-8");
+ } catch (UnsupportedEncodingException e) {
+ new RuntimeException("Failed to encode YUML markdown body", e);
+ }
+
+ StringBuilder mergedStyle = new StringBuilder();
+ String style = defaultString(params.get("style"), "scruffy");
+ String type = defaultString(params.get("type"), "class");
+ String format = defaultString(params.get("format"), "svg");
+
+ mergedStyle.append(style);
+
+ if (null != params.get("dir")) {
+ mergedStyle.append(";dir:" + params.get("dir"));
+ }
+
+ if (null != params.get("scale")) {
+ mergedStyle.append(";scale:" + params.get("scale"));
+ }
+
+ return new StringBuilder()
+ .append("http://yuml.me/diagram/")
+ .append(mergedStyle.toString() + "/")
+ .append(type + "/")
+ .append(encodedBody)
+ .append("." + format)
+ .toString();
+ }
+
+ @Override
+ public Rule[] blockPluginRules() {
+ return new Rule[]{BlockRule()};
+ }
+}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/main/resources/interpreter-setting.json
----------------------------------------------------------------------
diff --git a/markdown/src/main/resources/interpreter-setting.json b/markdown/src/main/resources/interpreter-setting.json
index fed93a2..9e670da 100644
--- a/markdown/src/main/resources/interpreter-setting.json
+++ b/markdown/src/main/resources/interpreter-setting.json
@@ -7,8 +7,8 @@
"markdown.parser.type": {
"envName": "MARKDOWN_PARSER_TYPE",
"propertyName": "markdown.parser.type",
- "defaultValue": "markdown4j",
- "description": "Markdown Parser Type. Available values: markdown4j, pegdown. Default = markdown4j"
+ "defaultValue": "pegdown",
+ "description": "Markdown Parser Type. Available values: pegdown, markdown4j. Default = pegdown"
}
},
"editor": {
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/markdown/src/test/java/org/apache/zeppelin/markdown/PegdownParserTest.java
----------------------------------------------------------------------
diff --git a/markdown/src/test/java/org/apache/zeppelin/markdown/PegdownParserTest.java b/markdown/src/test/java/org/apache/zeppelin/markdown/PegdownParserTest.java
index 3b60155..46603b0 100644
--- a/markdown/src/test/java/org/apache/zeppelin/markdown/PegdownParserTest.java
+++ b/markdown/src/test/java/org/apache/zeppelin/markdown/PegdownParserTest.java
@@ -22,8 +22,13 @@ import static org.junit.Assert.assertEquals;
import java.util.Properties;
import org.apache.zeppelin.interpreter.InterpreterResult;
+
import static org.apache.zeppelin.markdown.PegdownParser.wrapWithMarkdownClassDiv;
+import static org.junit.Assert.assertThat;
+
+import org.hamcrest.CoreMatchers;
import org.junit.After;
+import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
@@ -262,7 +267,6 @@ public class PegdownParserTest {
@Test
public void testAlignedTable() {
-
String input =
new StringBuilder()
.append("| First Header | Second Header | Third Header |\n")
@@ -299,4 +303,34 @@ public class PegdownParserTest {
InterpreterResult result = md.interpret(input, null);
assertEquals(wrapWithMarkdownClassDiv(expected), result.message());
}
+
+ @Test
+ public void testWebsequencePlugin() {
+ String input =
+ new StringBuilder()
+ .append("\n \n %%% sequence style=modern-blue\n")
+ .append("title Authentication Sequence\n")
+ .append("Alice->Bob: Authentication Request\n")
+ .append("note right of Bob: Bob thinks about it\n")
+ .append("Bob->Alice: Authentication Response\n")
+ .append(" %%% ")
+ .toString();
+
+ InterpreterResult result = md.interpret(input, null);
+ assertThat(result.message(), CoreMatchers.containsString("<img src=\"http://www.websequencediagrams.com/?png="));
+ }
+
+ @Test
+ public void testYumlPlugin() {
+ String input = new StringBuilder()
+ .append("\n \n %%% yuml style=nofunky scale=120 format=svg\n")
+ .append("[Customer]<>-orders>[Order]\n")
+ .append("[Order]++-0..>[LineItem]\n")
+ .append("[Order]-[note:Aggregate root.]\n")
+ .append(" %%% ")
+ .toString();
+
+ InterpreterResult result = md.interpret(input, null);
+ assertThat(result.message(), CoreMatchers.containsString("<img src=\"http://yuml.me/diagram/"));
+ }
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/zeppelin-distribution/src/bin_license/LICENSE
----------------------------------------------------------------------
diff --git a/zeppelin-distribution/src/bin_license/LICENSE b/zeppelin-distribution/src/bin_license/LICENSE
index bab53fa..aa63d6b 100644
--- a/zeppelin-distribution/src/bin_license/LICENSE
+++ b/zeppelin-distribution/src/bin_license/LICENSE
@@ -316,7 +316,6 @@ The following components are provided under the BSD-style License.
(BSD-like) ASM asm-tree (org.ow2.asm:asm-tree:5.0.3 - http://asm.ow2.org/) - Copyright (c) 2000-2011 INRIA, France Telecom
(BSD-like) ASM asm-analysis (org.ow2.asm:asm-analysis:5.0.3 - http://asm.ow2.org/) - Copyright (c) 2000-2011 INRIA, France Telecom
(BSD-like) ASM asm-utils (org.ow2.asm:asm-utils:5.0.3 - http://asm.ow2.org/) - Copyright (c) 2000-2011 INRIA, France Telecom
- (New BSD License) Markdown4j (org.commonjava.googlecode.markdown4j:markdown4j:jar:2.2-cj-1.0 - https://code.google.com/p/markdown4j/)
(New BSD License) Py4J (net.sf.py4j:py4j:0.9 - http://py4j.sourceforge.net/)
(New BSD License) Py4J (net.sf.py4j:py4j:0.10.3 - http://py4j.sourceforge.net/) - https://github.com/bartdag/py4j/blob/0.10.3/LICENSE.txt
(New BSD License) Markdown4j (org.commonjava.googlecode.markdown4j:markdown4j:jar:2.2-cj-1.0 - https://code.google.com/p/markdown4j/)
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/ab2cdfd3/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
----------------------------------------------------------------------
diff --git a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
index 6ea9107..ada6c07 100644
--- a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
+++ b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/InterpreterRestApiTest.java
@@ -173,8 +173,7 @@ public class InterpreterRestApiTest extends AbstractTestRestApi {
while (p.getStatus() != Status.FINISHED) {
Thread.sleep(100);
}
- assertEquals("<p>markdown</p>\n", p.getResult().message());
-
+ assertEquals(p.getResult().message(), getSimulatedMarkdownResult("markdown"));
// restart interpreter
for (InterpreterSetting setting : ZeppelinServer.notebook.getInterpreterFactory().getInterpreterSettings(note.getId())) {
@@ -196,7 +195,7 @@ public class InterpreterRestApiTest extends AbstractTestRestApi {
while (p.getStatus() != Status.FINISHED) {
Thread.sleep(100);
}
- assertEquals("<p>markdown restarted</p>\n", p.getResult().message());
+ assertEquals(p.getResult().message(), getSimulatedMarkdownResult("markdown restarted"));
//cleanup
ZeppelinServer.notebook.removeNote(note.getId(), anonymous);
}
@@ -218,7 +217,7 @@ public class InterpreterRestApiTest extends AbstractTestRestApi {
while (p.getStatus() != Status.FINISHED) {
Thread.sleep(100);
}
- assertEquals("<p>markdown</p>\n", p.getResult().message());
+ assertEquals(p.getResult().message(), getSimulatedMarkdownResult("markdown"));
// get md interpreter
InterpreterSetting mdIntpSetting = null;
@@ -275,4 +274,8 @@ public class InterpreterRestApiTest extends AbstractTestRestApi {
assertThat("Test delete method:", delete, isAllowed());
delete.releaseConnection();
}
+
+ public static String getSimulatedMarkdownResult(String markdown) {
+ return String.format("<div class=\"markdown-body\">\n<p>%s</p>\n</div>", markdown);
+ }
}