Markdown support

[Juan Pablo Santos Rodríguez <ju...@gmail.com>]

Hi!

(last e-mail today, promise)

At work we've built a POC regarding a tool to generate some static sites.
We're automating JBake site's generation from some assets, templates and
markdown content (b/c JBake understands markdown). For content authors we
were looking for a good web-based contents-editor, with versioning
capabilities, visually appealing, integrated with our ldap... Something
like JSPWiki O:-) but supporting markdown.

Given that current master allows switching parsers/renderers, I've been
going at work with some customizations we needed to use JSPWiki for this
POC, whenever I've could look at the POC. Most of them are specific to our
infrastructure (workflows, page storage), but the Markdown support is
pretty agnostic to us so I asked if I could move it to JSPWiki, and I got
my ok :-)

In order to simplify things regarding code transfer I'll be putting it on a
personal github repo (my company is already ok with that), AL licensed. I
have to extract it from our current code-base, so it maybe some days until
I put it on a github repo for review prior to bringing in to master, but
this is a heads up.

Regarding the maturity of the development
* it is POC level, meaning is working and stable, but not feature complete.
It's enough to demonstrate that JSPWiki can parse/render Markdown, but
there are some rough edges
** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash with a
link for all headings) that I might overlooked so there might be differences
** No toolbar support on editors (especially on plain editor). Current
toolbars aren't thought with multiple parsers on mind, and as the POC
end-users are more than capable of writing markdown with their bare hands
:-) I've preferred to focus on working the parser and the renderer than on
this. In order to be feature complete, this should be done, and I haven't
thought on a way of doing it yet, so any idea would be more than welcome
O:-) (probably extending the parsers with a new method to expose the markup
associated which each toolbar button, and present it through a new JSP or
who knows how)
* The parsing / rendering is handled with Flexmark (markdown parser, AL
licensed) and some extensions. It requires Java 7 (we are on Java 6). Time
to think on 2.11 O:-)?


br,
juan pablo

Re: Markdown support

[Juan Pablo Santos Rodríguez <ju...@gmail.com>]

Hi,

just pushed https://github.com/juanpablo-santos/jspwiki-markdown

Briefly:

There are two custom renderers (normal and wysiwyg editing) and one custom
parser. All of them delegate to Flexmark's
specific parser/renderers, which are a bit different to JSPWiki's ones. In
any case, these Flexmark's parser/renderers are
built using MarkdownDocument.options( context ), to set up Flexmark
parsing/rendering with an custom extension which
works (only) with Markdown Links.

The parser is able of building the (markup) node structure which will get
translated to html/dom nodes; that is,
is only able of adding and deleting (markup) nodes, but it can't set any
html attribute, because there isn't any html yet
(markup nodes don't hold html attributes).

The renderers are more or less the opposite: able of modifying the
generated html, but can't modify its structure: you
may be able here to change and href, f.ex., but you won't be able to add or
remove additional markup.

This way of working is by Flexmark's design, broadly speaking.

On both cases, the behaviour is similar, the parser/renderer ends
delegating to a class inside org.apache.wiki.markdown,
which decides the type of action to perform on the Link's type, and a bunch
of classes following the State pattern,
performing the required, concrete, actions (i.e., parse and render a plugin
or a wiki variable, etc.).

There are also a couple of utility classes. Of them, LinkParsingOperations
contains several methods scattered throughout
JSPWiki source code. IMO, it should replace those functions and, before
2.11, those functions should proxy to this class,
as all those operations seem to belong to a common class.

Last but not least, noticed another issue with caching turned on: on
WYSIWYG editors, whole page is rendered, including
plugins, variable names, etc. I suspect this has to do more with JSPWiki
itself, caching too early and how the markdown
is parsed/rendered, but still have to get a good look at it. Thorough
end-to-end testing is needed O:-)

My idea is to let that repo a week or two so it can be studied (maybe
commit small fixes or something along that lines)
and, if no concerns are raised, move it to jspwiki master, most probably as
an standalone module (it will be preceded
by another mail to discuss those specifics).


br,
juan pablo


On Sun, Nov 12, 2017 at 9:41 PM, Juan Pablo Santos Rodríguez <
juanpablo.santos@gmail.com> wrote:

> Hi Dirk,
>
> lot of stuff to cover (meaning: long mail following), will try to address
> all the points:
>
> Initial design
> ------------------
>
> yes, the solution, as it is made consists on a MarkdownParser /
> MarkdownRendered,
> which acts as an alternative to current Parser/Renderer. There's no
> support for mixed
> markup, but I suppose that could be added too, via another
> Parser/Renderer, which
> could act as a Facade, selecting an underlying markup processing depending
> on user
> preferences (or whatever). But that raises more questions, though: once
> the page is
> stored on a given markup, what happens when a user with a different markup
> stored in
> his/her preferences? The type of markup should be also stored as a
> WikiPage attribute...
>
> This was done as POC for evaluating using JSPWiki as a Markdown
> content-publising
> tool. For that flexmark is used, which does 99% of the markup parsing, so
> moving this
> to the client side would imply a totally different take on this. I'm still
> extracting markdown
> support from other custom code, so I expect that by Tuesday/Wednesday I'll
> have it
> uploaded to a github repo.
>
> As it is now, once you commit to a markup style, you're set with that. At
> least as a
> starting point, it's possible to add mixed markup support later on.
>
> The Parser/Renderer is based on Flexmark + a custom JSPWiki extension.
> Currently
> it supports:
> * Normal Markdown
> * Wiki links (internal, external, interwiki, edit, etc.)
> * Variables
> * Almost all plugins (more on this below)
> * Inline images
> * Footnotes
> * ACLs
> * Metadata
> * WYSIWYG edition
>
> The generated HTML markup specific to JSPWiki is almost, if not equal, to
> the current
> JSPWiki markup Parser/Renderer.
>
> The editors
> ----------------
>
> I've been skimming through the editors. So far, it seems to me that the js
> files operate
> on generated html (no jspwiki markup handling), so that shouldn't be a
> problem, and the
> JSPs doesn't seem to perform any transformation.
>
> As this was done as a POC, I've been focusing mostly on the
> parsing/rendering that on
> anything else, so it's pretty possible that I've missed something on the
> editors side, but
> without digging on them too much, they seemed to play along well.
>
> Oh, and of course, adaptations to the plain editor *need* to be done. I'm
> thinking that the
> different parsers should make available their configuration up to the
> editors, so the plain
> editor is able to generate the appropiate markup for each parser.
>
> Migration
> -------------
>
> there's nothing yet, as the POC I was working on assumed no previous
> content, so I haven't
> thought a lot about this... As always, something can/could/should be done:
> there is a
> flexmark extension (haven't investigated that it, though) which performs
> an html to markdown
> conversion. Given that there are unit tests which transform current
> markdown to html, the
> process seems feasible, at least for simple pages. This process would not
> be straightforward
> JSPWikiMarkup -> HTML -> Markdown, though, some intermediate "polishing"
> would be
> likely to happen (I'm thinking on %%css styles, or identifying links which
> are wiki links,
> instead of full links).
>
> This process could also be made available as a maven plugin...
>
> As for the other way round (Markdown -> JSPWikiMarkup): haven't even
> thought of it yet.
> In any case, is migrating from one markup to another something likely to
> happen? Once
> you're using one markup and have a bunch of pages, what would trigger the
> need of migrating
> to another markup? Maybe if you're on JSPWikiMarkup you are thinking on
> moving to
> Markdown b/c that way you can also use the pages as Github pages, or
> something like
> that (although some markup would be lost). As said before, haven't thought
> on this, so any
> opinion is more than welcome...
>
> Yet to be done
> ---------------------
>
> * Plain editor toolbar support
>
> * Initial set of WikiPages for markdown
>
> * %%css constructs. A new extension for this should be made, there is no
> support for this.
>
> * markup migration tool?
>
> * plugins implementing HeadingListener (that is, TableOfContents) are not
> supported: TableOfContents
> plugin implements an ad-hoc interface, HeadingListener, which is fired by
> JSPWikiMarkupParser every
> time it finds a header (more precisely, for every heading,
> JSPWikiMarkupParser generates a "#" link
> with the section reference and then registers a HeadingListener).
>
> When this plugin is executed then, it knows about the different sections,
> so it can generate the TOC.
> The way flexmarks parses and renders markdown, doesn't allow to generate
> the TOC this way. Initially,
> the Markdown Parser/Renderer generated those #-links, but as soon I
> realized TableOfContents wasn't
> usable under markdown, I removed the generation of #-links and instead of
> executing the plugin,
> switched it to flexmark's own TOC extension, surrounded with some divs. So
> as for now, whereas
> TableOfContents generates something like:
>
> <div class="toc">
> <div class="collapsebox">
> <h4 id="section-TOC">title</h4>
> <ul>/<ol> <!-- either ul or ol -->
> <li class="toclevel-1">
> <li class="toclevel-2">
> <li class="toclevel-3">
> </div>
> </div>
>
> the markdown parser/render currently generates something like:
> <div class="toc">
> <div class="collapsebox">
> <h4 id="section-TOC">title</h4>
> <!-- Generated by Flexmark's TOC extension -->
> <ul>/<ol> <!-- either ul or ol -->
> <li> <!--level 1 -->
>   <ul>/<ol> <!-- either ul or ol -->
>   <li> <!--level 2 -->
>     <ul>/<ol> <!-- either ul or ol -->
>       <li></li> <!-- level 3 -->
>     </ul>/</ol>
>   </li>
>   </ul>/</ol>
> </li>
> </ul>/</ol>
> <!-- End generated by Flexmark's TOC extension -->
> </div>
> </div>
>
> So there are two ways of adding support for this:
> a) add enough css so that both html render more or less the same.
> Parameters parsing should be implemented anyways.
> b) add a new Flexmark extension for TOC, probably forking+adapting the
> existing TOC extension.
>
> Hope this clarifies a bit.
>
> br,
> juan pablo
>
> On Sat, Nov 11, 2017 at 9:36 AM, Dirk Frederickx <
> dirk.frederickx@gmail.com> wrote:
>
>> Hi Juan,
>>
>> Adding markup support would indeed be an excellent extension for jspwiki.
>>
>>
>> When changing the parser/render as you proposed, this would mean that
>> for one jspwiki instance the backend storage would be changed from
>> jspwiki-markup to markdown.
>>
>> I think this has some important limitations:
>> - you can not use both markup styles on the same wiki instance
>>   so users familiar with jspwiki-markup would be forced to switch
>> - we need a solution for migrating a wiki repository between both markups
>>   to allow smooth transitions
>> - we'll need a rewrite the wysiwyg editors,  as they are converting
>>   between html and jspwiki-markup
>> - as you indicated we need some adaptions to the plain editor
>>   but I believe this would mainly be configuration   (I can help with
>> that)
>>   (configuration to adapt the auto-completions, ...)
>>
>> ----
>>
>> Alternatively, we could look for a solution whereby a user can either use
>> markdown
>> or jspwiki-markup, whenever he/she choses to.
>> In this case, the backend would continue to use jspwiki-markup;
>> during editing the user can chose another type of input.
>>
>> The user would then select a "markdown editor" store in its
>> UserPreferences.
>> (similar as chosing a wysiwyg editor)
>> Markdown would be used during editing, and converted to jspwiki-markup
>> during saving.
>> With live-preview the user will see immediately the rendered page.
>>
>> This approach is similar how the wysiwyg editors work,  translating
>> between
>> HTML
>> and jspwiki-markup when saving the page.
>>
>>
>> A topic to be addressed is the fact that markdown has a few more features
>> than jspwiki-markup.
>> For example, with markdown you can go to 6 levels with headers,  jspwiki
>> has only 3 levels.
>> I guess most of these cases would be solvable through the use of specific
>> %%css-class constructs.
>> To be validated.
>>
>> ----
>>
>> Finally, we could also opt for a solution to mix different markup styles
>> in
>> one page.
>> You can write a page in jspwiki-markup,  but if you want to copy/paste
>> some
>> markdown, it can be put inside a PLUGIN or a %%dynamic-style.
>>
>> The syntax of a PLUGIN (server side markup conversion) may be a bit too
>> complex
>> for most users.
>>
>> [{MarkdownPlugin
>>
>> ####This Is a Header####
>> }]
>>
>> Taking the markdown conversion to the browser-level (with javascript)
>> would
>> be feasible as well.
>> (eg showdown http://www.showdownjs.com/  )
>> %%markdown
>>
>> ####This Is a Header####
>> /%
>>
>>
>> ----
>>
>> I believe the 2nd option would be preferred.
>> However,  a faster path to support markdown would be with a plugin or a
>> dynamic style.
>>
>>
>> br,
>> dirk
>>
>>
>>
>>
>> On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
>> juanpablo.santos@gmail.com> wrote:
>>
>> > Hi!
>> >
>> > (last e-mail today, promise)
>> >
>> > At work we've built a POC regarding a tool to generate some static
>> sites.
>> > We're automating JBake site's generation from some assets, templates and
>> > markdown content (b/c JBake understands markdown). For content authors
>> we
>> > were looking for a good web-based contents-editor, with versioning
>> > capabilities, visually appealing, integrated with our ldap... Something
>> > like JSPWiki O:-) but supporting markdown.
>> >
>> > Given that current master allows switching parsers/renderers, I've been
>> > going at work with some customizations we needed to use JSPWiki for this
>> > POC, whenever I've could look at the POC. Most of them are specific to
>> our
>> > infrastructure (workflows, page storage), but the Markdown support is
>> > pretty agnostic to us so I asked if I could move it to JSPWiki, and I
>> got
>> > my ok :-)
>> >
>> > In order to simplify things regarding code transfer I'll be putting it
>> on a
>> > personal github repo (my company is already ok with that), AL licensed.
>> I
>> > have to extract it from our current code-base, so it maybe some days
>> until
>> > I put it on a github repo for review prior to bringing in to master, but
>> > this is a heads up.
>> >
>> > Regarding the maturity of the development
>> > * it is POC level, meaning is working and stable, but not feature
>> complete.
>> > It's enough to demonstrate that JSPWiki can parse/render Markdown, but
>> > there are some rough edges
>> > ** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash
>> with a
>> > link for all headings) that I might overlooked so there might be
>> > differences
>> > ** No toolbar support on editors (especially on plain editor). Current
>> > toolbars aren't thought with multiple parsers on mind, and as the POC
>> > end-users are more than capable of writing markdown with their bare
>> hands
>> > :-) I've preferred to focus on working the parser and the renderer than
>> on
>> > this. In order to be feature complete, this should be done, and I
>> haven't
>> > thought on a way of doing it yet, so any idea would be more than welcome
>> > O:-) (probably extending the parsers with a new method to expose the
>> markup
>> > associated which each toolbar button, and present it through a new JSP
>> or
>> > who knows how)
>> > * The parsing / rendering is handled with Flexmark (markdown parser, AL
>> > licensed) and some extensions. It requires Java 7 (we are on Java 6).
>> Time
>> > to think on 2.11 O:-)?
>> >
>> >
>> > br,
>> > juan pablo
>> >
>>
>
>

Re: Markdown support

[Juan Pablo Santos Rodríguez <ju...@gmail.com>]

Hi,

if nobody disagrees, I'll begin the move of the markdown parser / renderer
most probably this weekend. As it requires Java 7 to build, my intention is
to move it as a separate maven module (except maybe a couple of utility
classes), providing a page at jspwiki.a.o with installation instructions,
stating clearly it is not yet feature complete. When it is, the next
release should state that we fully support Markdown. Also that way, 2.10
still remains running on Java 6, although the Jenkins job will have to use
Java 7..


br,
juan pablo

On Sun, Nov 19, 2017 at 11:20 PM, Dirk Frederickx <dirk.frederickx@gmail.com
> wrote:

> Hi Juan,
>
> - would it be possible to set markdown as the only markup? so you don't
> have to wrap with
> %%(%)markdown (I'm thinking on a do-nothing-parser-and-renderer, so it all
> gets done in the browser)
>
> >> I've been thinking about that also.
> >> One option could be to define a new UserPreference or jspwiki-property
> to add
> specific class or behaviour  (such as "markdown"  or "prettify")
> to the top of the wiki-page  so that it become default to all pages.
> >> (This is similar to the [{SET page-styles="..."}] variable,  but you
> still need to add this to each page.)
>
> >> But then you do not have any way to turn it of on a per page basis. ...
>
> >> Anyway,  the wrapper %%%markdown ... /% is very limited, and gives you
> the option to still use JSPWiki
> markup (like plugins, %%styles, ...)  mixed with the markdown.
>
>
> - plugins, variables, acls et all would be still rendered? (the
> do-nothing-parser-and-renderer would
> better be a do-almost-nothing-parser-and-renderer..)
>
> >>  Yep, as long as you keep them outside the %%markdown ... /% wrapper.
> >> ACL typically would appear at the start of a page, just before the
> markdown.
>
>
>
> br,
>     dirk
>
>
> On Sun, Nov 19, 2017 at 11:12 PM, Juan Pablo Santos Rodríguez <
> juanpablo.santos@gmail.com> wrote:
>
> > Hi Dirk,
> >
> > I got the same reaction when began to work on the markdown extension, I
> was
> > expecting it would
> > take a lot less code. I'll take a look to the markdown behaviour this
> week
> > to see how it looks like.
> > Couple of questions in the meantime O:-)
> > - would it be possible to set markdown as the only markup? so you don't
> > have to wrap with
> > %%(%)markdown (I'm thinking on a do-nothing-parser-and-renderer, so it
> all
> > gets done in the browser)
> > - plugins, variables, acls et all would be still rendered? (the
> > do-nothing-parser-and-renderer would
> > better be a do-almost-nothing-parser-and-renderer..)
> >
> > Regarding the jspwiki-markdown, I've pushed a small fix, so it uses ASF's
> > snapshot repo, in order
> > to download latest JSPWiki snaphot, it needs at least 2.10.3-git-42 to be
> > able to compile. Doing an
> > mvn clean install on latest master should also make the compilation
> > problems disappear.
> >
> > Finally, about " add the jspwiki-markdown as a dependency to jspwiki-war
> ",
> > that would mean
> > something like
> > https://github.com/juanpablo-santos/jspwiki-markdown/blob/
> > master/jspwiki-markdown-war/pom.xml#L36-L49
> > But for that you have to have access to latest snapshot, either by
> > compiling latest master,
> > or by pulling latest commit and trying again O:-) Either way, thanks for
> > looking into it!
> >
> >
> > br,
> > juan pablo
> >
> >
> >
> > On Sun, Nov 19, 2017 at 11:15 AM, Dirk Frederickx <
> > dirk.frederickx@gmail.com
> > > wrote:
> >
> > > Hi Juan,
> > >
> > > Wow !  It seems there are much more aspects to it then simple
> > > parsing/rendering :-)
> > >
> > > Concerning the TOC plugin,  the current generated HTML always looked
> > funny
> > > to me.  (i.e. a flat list of <li> items,  versus a regular nested UL/LI
> > > tree)
> > > We'd probably should improve the current TOC plugin to render  a
> regular
> > > UL/LI tree.
> > > The rest is indeed pure css formatting.
> > >
> > > ====
> > >
> > >
> > > I tried to quickly build a %%markdown behaviour based on an open-source
> > > javascript MARKDOWN rendering engine (marked.js),  which fully runs in
> > the
> > > browser.
> > > ( see https://jspwiki-wiki.apache.org/Wiki.jsp?page=Markdown%
> 20Behavior
> > )
> > >
> > > Usage:
> > > %%markdown
> > > {{{
> > >     ##here comes your markdown!##
> > > }}}
> > > /%
> > >
> > > Note: see https://issues.apache.org/jira/browse/JSPWIKI-1040  to
> > simplify
> > > this to
> > > %%%markdown
> > >     ##here comes your markdown!##
> > > /%
> > >
> > >
> > > This approach allows for mixing JSPWiki markup and MARKDOWN on the same
> > > page, while keeping the MARKDOWN syntax fully conform to the
> > specification.
> > >
> > > So there is no need to extend  the MARKDOWN syntax with additional
> > JSPWiki
> > > specific markup, like [{plugins}]  or %%css-style.
> > >
> > > It does support GFM, but apparently only partially.  ( more testing
> > > required -- this obviously depends on the chosen JS library  )
> > > I added support for prettified code blocks,  and section #hash-links.
> > > The plain editor  (with preview)  works OK.
> > >
> > > Current limitations:
> > > - Links require full URL format :  support for simple wiki-pages names
> > > would be cool
> > > - Table Of Contents support : re-rendering of a TOC should be done on
> the
> > > browser  (could be useful for JSPWiki anyway,  eg looking at the lack
> of
> > > TOC support for InsertPages etc...)
> > > - Plain-Editor:  Auto-suggestion and TAB-completion could be extended
> > with
> > > MARKDOWN specific configuration.
> > > - ...
> > >
> > >
> > > ----
> > >
> > >
> > > BTW:
> > > I tried to install the jspwiki-markdown from github , but got errors
> with
> > > the mvn clean install.
> > > Mainly due to missing symbols...
> > > => " add the jspwiki-markdown as a dependency to jspwiki-war " : I need
> > > your help here...
> > >
> > >
> > > Best regards,
> > >
> > > dirk
> > >
> > >
> > >
> > >
> > >
> > > On Sun, Nov 12, 2017 at 9:41 PM, Juan Pablo Santos Rodríguez <
> > > juanpablo.santos@gmail.com> wrote:
> > >
> > > > Hi Dirk,
> > > >
> > > > lot of stuff to cover (meaning: long mail following), will try to
> > address
> > > > all the points:
> > > >
> > > > Initial design
> > > > ------------------
> > > >
> > > > yes, the solution, as it is made consists on a MarkdownParser /
> > > > MarkdownRendered,
> > > > which acts as an alternative to current Parser/Renderer. There's no
> > > support
> > > > for mixed
> > > > markup, but I suppose that could be added too, via another
> > > Parser/Renderer,
> > > > which
> > > > could act as a Facade, selecting an underlying markup processing
> > > depending
> > > > on user
> > > > preferences (or whatever). But that raises more questions, though:
> once
> > > the
> > > > page is
> > > > stored on a given markup, what happens when a user with a different
> > > markup
> > > > stored in
> > > > his/her preferences? The type of markup should be also stored as a
> > > WikiPage
> > > > attribute...
> > > >
> > > > This was done as POC for evaluating using JSPWiki as a Markdown
> > > > content-publising
> > > > tool. For that flexmark is used, which does 99% of the markup
> parsing,
> > so
> > > > moving this
> > > > to the client side would imply a totally different take on this. I'm
> > > still
> > > > extracting markdown
> > > > support from other custom code, so I expect that by Tuesday/Wednesday
> > > I'll
> > > > have it
> > > > uploaded to a github repo.
> > > >
> > > > As it is now, once you commit to a markup style, you're set with
> that.
> > At
> > > > least as a
> > > > starting point, it's possible to add mixed markup support later on.
> > > >
> > > > The Parser/Renderer is based on Flexmark + a custom JSPWiki
> extension.
> > > > Currently
> > > > it supports:
> > > > * Normal Markdown
> > > > * Wiki links (internal, external, interwiki, edit, etc.)
> > > > * Variables
> > > > * Almost all plugins (more on this below)
> > > > * Inline images
> > > > * Footnotes
> > > > * ACLs
> > > > * Metadata
> > > > * WYSIWYG edition
> > > >
> > > > The generated HTML markup specific to JSPWiki is almost, if not
> equal,
> > to
> > > > the current
> > > > JSPWiki markup Parser/Renderer.
> > > >
> > > > The editors
> > > > ----------------
> > > >
> > > > I've been skimming through the editors. So far, it seems to me that
> the
> > > js
> > > > files operate
> > > > on generated html (no jspwiki markup handling), so that shouldn't be
> a
> > > > problem, and the
> > > > JSPs doesn't seem to perform any transformation.
> > > >
> > > > As this was done as a POC, I've been focusing mostly on the
> > > > parsing/rendering that on
> > > > anything else, so it's pretty possible that I've missed something on
> > the
> > > > editors side, but
> > > > without digging on them too much, they seemed to play along well.
> > > >
> > > > Oh, and of course, adaptations to the plain editor *need* to be done.
> > I'm
> > > > thinking that the
> > > > different parsers should make available their configuration up to the
> > > > editors, so the plain
> > > > editor is able to generate the appropiate markup for each parser.
> > > >
> > > > Migration
> > > > -------------
> > > >
> > > > there's nothing yet, as the POC I was working on assumed no previous
> > > > content, so I haven't
> > > > thought a lot about this... As always, something can/could/should be
> > > done:
> > > > there is a
> > > > flexmark extension (haven't investigated that it, though) which
> > performs
> > > an
> > > > html to markdown
> > > > conversion. Given that there are unit tests which transform current
> > > > markdown to html, the
> > > > process seems feasible, at least for simple pages. This process would
> > not
> > > > be straightforward
> > > > JSPWikiMarkup -> HTML -> Markdown, though, some intermediate
> > "polishing"
> > > > would be
> > > > likely to happen (I'm thinking on %%css styles, or identifying links
> > > which
> > > > are wiki links,
> > > > instead of full links).
> > > >
> > > > This process could also be made available as a maven plugin...
> > > >
> > > > As for the other way round (Markdown -> JSPWikiMarkup): haven't even
> > > > thought of it yet.
> > > > In any case, is migrating from one markup to another something likely
> > to
> > > > happen? Once
> > > > you're using one markup and have a bunch of pages, what would trigger
> > the
> > > > need of migrating
> > > > to another markup? Maybe if you're on JSPWikiMarkup you are thinking
> on
> > > > moving to
> > > > Markdown b/c that way you can also use the pages as Github pages, or
> > > > something like
> > > > that (although some markup would be lost). As said before, haven't
> > > thought
> > > > on this, so any
> > > > opinion is more than welcome...
> > > >
> > > > Yet to be done
> > > > ---------------------
> > > >
> > > > * Plain editor toolbar support
> > > >
> > > > * Initial set of WikiPages for markdown
> > > >
> > > > * %%css constructs. A new extension for this should be made, there is
> > no
> > > > support for this.
> > > >
> > > > * markup migration tool?
> > > >
> > > > * plugins implementing HeadingListener (that is, TableOfContents) are
> > not
> > > > supported: TableOfContents
> > > > plugin implements an ad-hoc interface, HeadingListener, which is
> fired
> > by
> > > > JSPWikiMarkupParser every
> > > > time it finds a header (more precisely, for every heading,
> > > > JSPWikiMarkupParser generates a "#" link
> > > > with the section reference and then registers a HeadingListener).
> > > >
> > > > When this plugin is executed then, it knows about the different
> > sections,
> > > > so it can generate the TOC.
> > > > The way flexmarks parses and renders markdown, doesn't allow to
> > generate
> > > > the TOC this way. Initially,
> > > > the Markdown Parser/Renderer generated those #-links, but as soon I
> > > > realized TableOfContents wasn't
> > > > usable under markdown, I removed the generation of #-links and
> instead
> > of
> > > > executing the plugin,
> > > > switched it to flexmark's own TOC extension, surrounded with some
> divs.
> > > So
> > > > as for now, whereas
> > > > TableOfContents generates something like:
> > > >
> > > > <div class="toc">
> > > > <div class="collapsebox">
> > > > <h4 id="section-TOC">title</h4>
> > > > <ul>/<ol> <!-- either ul or ol -->
> > > > <li class="toclevel-1">
> > > > <li class="toclevel-2">
> > > > <li class="toclevel-3">
> > > > </div>
> > > > </div>
> > > >
> > > > the markdown parser/render currently generates something like:
> > > > <div class="toc">
> > > > <div class="collapsebox">
> > > > <h4 id="section-TOC">title</h4>
> > > > <!-- Generated by Flexmark's TOC extension -->
> > > > <ul>/<ol> <!-- either ul or ol -->
> > > > <li> <!--level 1 -->
> > > >   <ul>/<ol> <!-- either ul or ol -->
> > > >   <li> <!--level 2 -->
> > > >     <ul>/<ol> <!-- either ul or ol -->
> > > >       <li></li> <!-- level 3 -->
> > > >     </ul>/</ol>
> > > >   </li>
> > > >   </ul>/</ol>
> > > > </li>
> > > > </ul>/</ol>
> > > > <!-- End generated by Flexmark's TOC extension -->
> > > > </div>
> > > > </div>
> > > >
> > > > So there are two ways of adding support for this:
> > > > a) add enough css so that both html render more or less the same.
> > > > Parameters parsing should be implemented anyways.
> > > > b) add a new Flexmark extension for TOC, probably forking+adapting
> the
> > > > existing TOC extension.
> > > >
> > > > Hope this clarifies a bit.
> > > >
> > > > br,
> > > > juan pablo
> > > >
> > > > On Sat, Nov 11, 2017 at 9:36 AM, Dirk Frederickx <
> > > > dirk.frederickx@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Juan,
> > > > >
> > > > > Adding markup support would indeed be an excellent extension for
> > > jspwiki.
> > > > >
> > > > >
> > > > > When changing the parser/render as you proposed, this would mean
> that
> > > > > for one jspwiki instance the backend storage would be changed from
> > > > > jspwiki-markup to markdown.
> > > > >
> > > > > I think this has some important limitations:
> > > > > - you can not use both markup styles on the same wiki instance
> > > > >   so users familiar with jspwiki-markup would be forced to switch
> > > > > - we need a solution for migrating a wiki repository between both
> > > markups
> > > > >   to allow smooth transitions
> > > > > - we'll need a rewrite the wysiwyg editors,  as they are converting
> > > > >   between html and jspwiki-markup
> > > > > - as you indicated we need some adaptions to the plain editor
> > > > >   but I believe this would mainly be configuration   (I can help
> with
> > > > that)
> > > > >   (configuration to adapt the auto-completions, ...)
> > > > >
> > > > > ----
> > > > >
> > > > > Alternatively, we could look for a solution whereby a user can
> either
> > > use
> > > > > markdown
> > > > > or jspwiki-markup, whenever he/she choses to.
> > > > > In this case, the backend would continue to use jspwiki-markup;
> > > > > during editing the user can chose another type of input.
> > > > >
> > > > > The user would then select a "markdown editor" store in its
> > > > > UserPreferences.
> > > > > (similar as chosing a wysiwyg editor)
> > > > > Markdown would be used during editing, and converted to
> > jspwiki-markup
> > > > > during saving.
> > > > > With live-preview the user will see immediately the rendered page.
> > > > >
> > > > > This approach is similar how the wysiwyg editors work,  translating
> > > > between
> > > > > HTML
> > > > > and jspwiki-markup when saving the page.
> > > > >
> > > > >
> > > > > A topic to be addressed is the fact that markdown has a few more
> > > features
> > > > > than jspwiki-markup.
> > > > > For example, with markdown you can go to 6 levels with headers,
> > > jspwiki
> > > > > has only 3 levels.
> > > > > I guess most of these cases would be solvable through the use of
> > > specific
> > > > > %%css-class constructs.
> > > > > To be validated.
> > > > >
> > > > > ----
> > > > >
> > > > > Finally, we could also opt for a solution to mix different markup
> > > styles
> > > > in
> > > > > one page.
> > > > > You can write a page in jspwiki-markup,  but if you want to
> > copy/paste
> > > > some
> > > > > markdown, it can be put inside a PLUGIN or a %%dynamic-style.
> > > > >
> > > > > The syntax of a PLUGIN (server side markup conversion) may be a bit
> > too
> > > > > complex
> > > > > for most users.
> > > > >
> > > > > [{MarkdownPlugin
> > > > >
> > > > > ####This Is a Header####
> > > > > }]
> > > > >
> > > > > Taking the markdown conversion to the browser-level (with
> javascript)
> > > > would
> > > > > be feasible as well.
> > > > > (eg showdown http://www.showdownjs.com/  )
> > > > > %%markdown
> > > > >
> > > > > ####This Is a Header####
> > > > > /%
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > I believe the 2nd option would be preferred.
> > > > > However,  a faster path to support markdown would be with a plugin
> > or a
> > > > > dynamic style.
> > > > >
> > > > >
> > > > > br,
> > > > > dirk
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
> > > > > juanpablo.santos@gmail.com> wrote:
> > > > >
> > > > > > Hi!
> > > > > >
> > > > > > (last e-mail today, promise)
> > > > > >
> > > > > > At work we've built a POC regarding a tool to generate some
> static
> > > > sites.
> > > > > > We're automating JBake site's generation from some assets,
> > templates
> > > > and
> > > > > > markdown content (b/c JBake understands markdown). For content
> > > authors
> > > > we
> > > > > > were looking for a good web-based contents-editor, with
> versioning
> > > > > > capabilities, visually appealing, integrated with our ldap...
> > > Something
> > > > > > like JSPWiki O:-) but supporting markdown.
> > > > > >
> > > > > > Given that current master allows switching parsers/renderers,
> I've
> > > been
> > > > > > going at work with some customizations we needed to use JSPWiki
> for
> > > > this
> > > > > > POC, whenever I've could look at the POC. Most of them are
> specific
> > > to
> > > > > our
> > > > > > infrastructure (workflows, page storage), but the Markdown
> support
> > is
> > > > > > pretty agnostic to us so I asked if I could move it to JSPWiki,
> > and I
> > > > got
> > > > > > my ok :-)
> > > > > >
> > > > > > In order to simplify things regarding code transfer I'll be
> putting
> > > it
> > > > > on a
> > > > > > personal github repo (my company is already ok with that), AL
> > > > licensed. I
> > > > > > have to extract it from our current code-base, so it maybe some
> > days
> > > > > until
> > > > > > I put it on a github repo for review prior to bringing in to
> > master,
> > > > but
> > > > > > this is a heads up.
> > > > > >
> > > > > > Regarding the maturity of the development
> > > > > > * it is POC level, meaning is working and stable, but not feature
> > > > > complete.
> > > > > > It's enough to demonstrate that JSPWiki can parse/render
> Markdown,
> > > but
> > > > > > there are some rough edges
> > > > > > ** JSPWikiMarkupParser does a lot of things (f.ex. generates a
> hash
> > > > with
> > > > > a
> > > > > > link for all headings) that I might overlooked so there might be
> > > > > > differences
> > > > > > ** No toolbar support on editors (especially on plain editor).
> > > Current
> > > > > > toolbars aren't thought with multiple parsers on mind, and as the
> > POC
> > > > > > end-users are more than capable of writing markdown with their
> bare
> > > > hands
> > > > > > :-) I've preferred to focus on working the parser and the
> renderer
> > > than
> > > > > on
> > > > > > this. In order to be feature complete, this should be done, and I
> > > > haven't
> > > > > > thought on a way of doing it yet, so any idea would be more than
> > > > welcome
> > > > > > O:-) (probably extending the parsers with a new method to expose
> > the
> > > > > markup
> > > > > > associated which each toolbar button, and present it through a
> new
> > > JSP
> > > > or
> > > > > > who knows how)
> > > > > > * The parsing / rendering is handled with Flexmark (markdown
> > parser,
> > > AL
> > > > > > licensed) and some extensions. It requires Java 7 (we are on Java
> > 6).
> > > > > Time
> > > > > > to think on 2.11 O:-)?
> > > > > >
> > > > > >
> > > > > > br,
> > > > > > juan pablo
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Markdown support

[Dirk Frederickx <di...@gmail.com>]

Hi Juan,

- would it be possible to set markdown as the only markup? so you don't
have to wrap with
%%(%)markdown (I'm thinking on a do-nothing-parser-and-renderer, so it all
gets done in the browser)

>> I've been thinking about that also.
>> One option could be to define a new UserPreference or jspwiki-property
to add
specific class or behaviour  (such as "markdown"  or "prettify")
to the top of the wiki-page  so that it become default to all pages.
>> (This is similar to the [{SET page-styles="..."}] variable,  but you
still need to add this to each page.)

>> But then you do not have any way to turn it of on a per page basis. ...

>> Anyway,  the wrapper %%%markdown ... /% is very limited, and gives you
the option to still use JSPWiki
markup (like plugins, %%styles, ...)  mixed with the markdown.


- plugins, variables, acls et all would be still rendered? (the
do-nothing-parser-and-renderer would
better be a do-almost-nothing-parser-and-renderer..)

>>  Yep, as long as you keep them outside the %%markdown ... /% wrapper.
>> ACL typically would appear at the start of a page, just before the
markdown.



br,
    dirk


On Sun, Nov 19, 2017 at 11:12 PM, Juan Pablo Santos Rodríguez <
juanpablo.santos@gmail.com> wrote:

> Hi Dirk,
>
> I got the same reaction when began to work on the markdown extension, I was
> expecting it would
> take a lot less code. I'll take a look to the markdown behaviour this week
> to see how it looks like.
> Couple of questions in the meantime O:-)
> - would it be possible to set markdown as the only markup? so you don't
> have to wrap with
> %%(%)markdown (I'm thinking on a do-nothing-parser-and-renderer, so it all
> gets done in the browser)
> - plugins, variables, acls et all would be still rendered? (the
> do-nothing-parser-and-renderer would
> better be a do-almost-nothing-parser-and-renderer..)
>
> Regarding the jspwiki-markdown, I've pushed a small fix, so it uses ASF's
> snapshot repo, in order
> to download latest JSPWiki snaphot, it needs at least 2.10.3-git-42 to be
> able to compile. Doing an
> mvn clean install on latest master should also make the compilation
> problems disappear.
>
> Finally, about " add the jspwiki-markdown as a dependency to jspwiki-war ",
> that would mean
> something like
> https://github.com/juanpablo-santos/jspwiki-markdown/blob/
> master/jspwiki-markdown-war/pom.xml#L36-L49
> But for that you have to have access to latest snapshot, either by
> compiling latest master,
> or by pulling latest commit and trying again O:-) Either way, thanks for
> looking into it!
>
>
> br,
> juan pablo
>
>
>
> On Sun, Nov 19, 2017 at 11:15 AM, Dirk Frederickx <
> dirk.frederickx@gmail.com
> > wrote:
>
> > Hi Juan,
> >
> > Wow !  It seems there are much more aspects to it then simple
> > parsing/rendering :-)
> >
> > Concerning the TOC plugin,  the current generated HTML always looked
> funny
> > to me.  (i.e. a flat list of <li> items,  versus a regular nested UL/LI
> > tree)
> > We'd probably should improve the current TOC plugin to render  a regular
> > UL/LI tree.
> > The rest is indeed pure css formatting.
> >
> > ====
> >
> >
> > I tried to quickly build a %%markdown behaviour based on an open-source
> > javascript MARKDOWN rendering engine (marked.js),  which fully runs in
> the
> > browser.
> > ( see https://jspwiki-wiki.apache.org/Wiki.jsp?page=Markdown%20Behavior
> )
> >
> > Usage:
> > %%markdown
> > {{{
> >     ##here comes your markdown!##
> > }}}
> > /%
> >
> > Note: see https://issues.apache.org/jira/browse/JSPWIKI-1040  to
> simplify
> > this to
> > %%%markdown
> >     ##here comes your markdown!##
> > /%
> >
> >
> > This approach allows for mixing JSPWiki markup and MARKDOWN on the same
> > page, while keeping the MARKDOWN syntax fully conform to the
> specification.
> >
> > So there is no need to extend  the MARKDOWN syntax with additional
> JSPWiki
> > specific markup, like [{plugins}]  or %%css-style.
> >
> > It does support GFM, but apparently only partially.  ( more testing
> > required -- this obviously depends on the chosen JS library  )
> > I added support for prettified code blocks,  and section #hash-links.
> > The plain editor  (with preview)  works OK.
> >
> > Current limitations:
> > - Links require full URL format :  support for simple wiki-pages names
> > would be cool
> > - Table Of Contents support : re-rendering of a TOC should be done on the
> > browser  (could be useful for JSPWiki anyway,  eg looking at the lack of
> > TOC support for InsertPages etc...)
> > - Plain-Editor:  Auto-suggestion and TAB-completion could be extended
> with
> > MARKDOWN specific configuration.
> > - ...
> >
> >
> > ----
> >
> >
> > BTW:
> > I tried to install the jspwiki-markdown from github , but got errors with
> > the mvn clean install.
> > Mainly due to missing symbols...
> > => " add the jspwiki-markdown as a dependency to jspwiki-war " : I need
> > your help here...
> >
> >
> > Best regards,
> >
> > dirk
> >
> >
> >
> >
> >
> > On Sun, Nov 12, 2017 at 9:41 PM, Juan Pablo Santos Rodríguez <
> > juanpablo.santos@gmail.com> wrote:
> >
> > > Hi Dirk,
> > >
> > > lot of stuff to cover (meaning: long mail following), will try to
> address
> > > all the points:
> > >
> > > Initial design
> > > ------------------
> > >
> > > yes, the solution, as it is made consists on a MarkdownParser /
> > > MarkdownRendered,
> > > which acts as an alternative to current Parser/Renderer. There's no
> > support
> > > for mixed
> > > markup, but I suppose that could be added too, via another
> > Parser/Renderer,
> > > which
> > > could act as a Facade, selecting an underlying markup processing
> > depending
> > > on user
> > > preferences (or whatever). But that raises more questions, though: once
> > the
> > > page is
> > > stored on a given markup, what happens when a user with a different
> > markup
> > > stored in
> > > his/her preferences? The type of markup should be also stored as a
> > WikiPage
> > > attribute...
> > >
> > > This was done as POC for evaluating using JSPWiki as a Markdown
> > > content-publising
> > > tool. For that flexmark is used, which does 99% of the markup parsing,
> so
> > > moving this
> > > to the client side would imply a totally different take on this. I'm
> > still
> > > extracting markdown
> > > support from other custom code, so I expect that by Tuesday/Wednesday
> > I'll
> > > have it
> > > uploaded to a github repo.
> > >
> > > As it is now, once you commit to a markup style, you're set with that.
> At
> > > least as a
> > > starting point, it's possible to add mixed markup support later on.
> > >
> > > The Parser/Renderer is based on Flexmark + a custom JSPWiki extension.
> > > Currently
> > > it supports:
> > > * Normal Markdown
> > > * Wiki links (internal, external, interwiki, edit, etc.)
> > > * Variables
> > > * Almost all plugins (more on this below)
> > > * Inline images
> > > * Footnotes
> > > * ACLs
> > > * Metadata
> > > * WYSIWYG edition
> > >
> > > The generated HTML markup specific to JSPWiki is almost, if not equal,
> to
> > > the current
> > > JSPWiki markup Parser/Renderer.
> > >
> > > The editors
> > > ----------------
> > >
> > > I've been skimming through the editors. So far, it seems to me that the
> > js
> > > files operate
> > > on generated html (no jspwiki markup handling), so that shouldn't be a
> > > problem, and the
> > > JSPs doesn't seem to perform any transformation.
> > >
> > > As this was done as a POC, I've been focusing mostly on the
> > > parsing/rendering that on
> > > anything else, so it's pretty possible that I've missed something on
> the
> > > editors side, but
> > > without digging on them too much, they seemed to play along well.
> > >
> > > Oh, and of course, adaptations to the plain editor *need* to be done.
> I'm
> > > thinking that the
> > > different parsers should make available their configuration up to the
> > > editors, so the plain
> > > editor is able to generate the appropiate markup for each parser.
> > >
> > > Migration
> > > -------------
> > >
> > > there's nothing yet, as the POC I was working on assumed no previous
> > > content, so I haven't
> > > thought a lot about this... As always, something can/could/should be
> > done:
> > > there is a
> > > flexmark extension (haven't investigated that it, though) which
> performs
> > an
> > > html to markdown
> > > conversion. Given that there are unit tests which transform current
> > > markdown to html, the
> > > process seems feasible, at least for simple pages. This process would
> not
> > > be straightforward
> > > JSPWikiMarkup -> HTML -> Markdown, though, some intermediate
> "polishing"
> > > would be
> > > likely to happen (I'm thinking on %%css styles, or identifying links
> > which
> > > are wiki links,
> > > instead of full links).
> > >
> > > This process could also be made available as a maven plugin...
> > >
> > > As for the other way round (Markdown -> JSPWikiMarkup): haven't even
> > > thought of it yet.
> > > In any case, is migrating from one markup to another something likely
> to
> > > happen? Once
> > > you're using one markup and have a bunch of pages, what would trigger
> the
> > > need of migrating
> > > to another markup? Maybe if you're on JSPWikiMarkup you are thinking on
> > > moving to
> > > Markdown b/c that way you can also use the pages as Github pages, or
> > > something like
> > > that (although some markup would be lost). As said before, haven't
> > thought
> > > on this, so any
> > > opinion is more than welcome...
> > >
> > > Yet to be done
> > > ---------------------
> > >
> > > * Plain editor toolbar support
> > >
> > > * Initial set of WikiPages for markdown
> > >
> > > * %%css constructs. A new extension for this should be made, there is
> no
> > > support for this.
> > >
> > > * markup migration tool?
> > >
> > > * plugins implementing HeadingListener (that is, TableOfContents) are
> not
> > > supported: TableOfContents
> > > plugin implements an ad-hoc interface, HeadingListener, which is fired
> by
> > > JSPWikiMarkupParser every
> > > time it finds a header (more precisely, for every heading,
> > > JSPWikiMarkupParser generates a "#" link
> > > with the section reference and then registers a HeadingListener).
> > >
> > > When this plugin is executed then, it knows about the different
> sections,
> > > so it can generate the TOC.
> > > The way flexmarks parses and renders markdown, doesn't allow to
> generate
> > > the TOC this way. Initially,
> > > the Markdown Parser/Renderer generated those #-links, but as soon I
> > > realized TableOfContents wasn't
> > > usable under markdown, I removed the generation of #-links and instead
> of
> > > executing the plugin,
> > > switched it to flexmark's own TOC extension, surrounded with some divs.
> > So
> > > as for now, whereas
> > > TableOfContents generates something like:
> > >
> > > <div class="toc">
> > > <div class="collapsebox">
> > > <h4 id="section-TOC">title</h4>
> > > <ul>/<ol> <!-- either ul or ol -->
> > > <li class="toclevel-1">
> > > <li class="toclevel-2">
> > > <li class="toclevel-3">
> > > </div>
> > > </div>
> > >
> > > the markdown parser/render currently generates something like:
> > > <div class="toc">
> > > <div class="collapsebox">
> > > <h4 id="section-TOC">title</h4>
> > > <!-- Generated by Flexmark's TOC extension -->
> > > <ul>/<ol> <!-- either ul or ol -->
> > > <li> <!--level 1 -->
> > >   <ul>/<ol> <!-- either ul or ol -->
> > >   <li> <!--level 2 -->
> > >     <ul>/<ol> <!-- either ul or ol -->
> > >       <li></li> <!-- level 3 -->
> > >     </ul>/</ol>
> > >   </li>
> > >   </ul>/</ol>
> > > </li>
> > > </ul>/</ol>
> > > <!-- End generated by Flexmark's TOC extension -->
> > > </div>
> > > </div>
> > >
> > > So there are two ways of adding support for this:
> > > a) add enough css so that both html render more or less the same.
> > > Parameters parsing should be implemented anyways.
> > > b) add a new Flexmark extension for TOC, probably forking+adapting the
> > > existing TOC extension.
> > >
> > > Hope this clarifies a bit.
> > >
> > > br,
> > > juan pablo
> > >
> > > On Sat, Nov 11, 2017 at 9:36 AM, Dirk Frederickx <
> > > dirk.frederickx@gmail.com>
> > > wrote:
> > >
> > > > Hi Juan,
> > > >
> > > > Adding markup support would indeed be an excellent extension for
> > jspwiki.
> > > >
> > > >
> > > > When changing the parser/render as you proposed, this would mean that
> > > > for one jspwiki instance the backend storage would be changed from
> > > > jspwiki-markup to markdown.
> > > >
> > > > I think this has some important limitations:
> > > > - you can not use both markup styles on the same wiki instance
> > > >   so users familiar with jspwiki-markup would be forced to switch
> > > > - we need a solution for migrating a wiki repository between both
> > markups
> > > >   to allow smooth transitions
> > > > - we'll need a rewrite the wysiwyg editors,  as they are converting
> > > >   between html and jspwiki-markup
> > > > - as you indicated we need some adaptions to the plain editor
> > > >   but I believe this would mainly be configuration   (I can help with
> > > that)
> > > >   (configuration to adapt the auto-completions, ...)
> > > >
> > > > ----
> > > >
> > > > Alternatively, we could look for a solution whereby a user can either
> > use
> > > > markdown
> > > > or jspwiki-markup, whenever he/she choses to.
> > > > In this case, the backend would continue to use jspwiki-markup;
> > > > during editing the user can chose another type of input.
> > > >
> > > > The user would then select a "markdown editor" store in its
> > > > UserPreferences.
> > > > (similar as chosing a wysiwyg editor)
> > > > Markdown would be used during editing, and converted to
> jspwiki-markup
> > > > during saving.
> > > > With live-preview the user will see immediately the rendered page.
> > > >
> > > > This approach is similar how the wysiwyg editors work,  translating
> > > between
> > > > HTML
> > > > and jspwiki-markup when saving the page.
> > > >
> > > >
> > > > A topic to be addressed is the fact that markdown has a few more
> > features
> > > > than jspwiki-markup.
> > > > For example, with markdown you can go to 6 levels with headers,
> > jspwiki
> > > > has only 3 levels.
> > > > I guess most of these cases would be solvable through the use of
> > specific
> > > > %%css-class constructs.
> > > > To be validated.
> > > >
> > > > ----
> > > >
> > > > Finally, we could also opt for a solution to mix different markup
> > styles
> > > in
> > > > one page.
> > > > You can write a page in jspwiki-markup,  but if you want to
> copy/paste
> > > some
> > > > markdown, it can be put inside a PLUGIN or a %%dynamic-style.
> > > >
> > > > The syntax of a PLUGIN (server side markup conversion) may be a bit
> too
> > > > complex
> > > > for most users.
> > > >
> > > > [{MarkdownPlugin
> > > >
> > > > ####This Is a Header####
> > > > }]
> > > >
> > > > Taking the markdown conversion to the browser-level (with javascript)
> > > would
> > > > be feasible as well.
> > > > (eg showdown http://www.showdownjs.com/  )
> > > > %%markdown
> > > >
> > > > ####This Is a Header####
> > > > /%
> > > >
> > > >
> > > > ----
> > > >
> > > > I believe the 2nd option would be preferred.
> > > > However,  a faster path to support markdown would be with a plugin
> or a
> > > > dynamic style.
> > > >
> > > >
> > > > br,
> > > > dirk
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
> > > > juanpablo.santos@gmail.com> wrote:
> > > >
> > > > > Hi!
> > > > >
> > > > > (last e-mail today, promise)
> > > > >
> > > > > At work we've built a POC regarding a tool to generate some static
> > > sites.
> > > > > We're automating JBake site's generation from some assets,
> templates
> > > and
> > > > > markdown content (b/c JBake understands markdown). For content
> > authors
> > > we
> > > > > were looking for a good web-based contents-editor, with versioning
> > > > > capabilities, visually appealing, integrated with our ldap...
> > Something
> > > > > like JSPWiki O:-) but supporting markdown.
> > > > >
> > > > > Given that current master allows switching parsers/renderers, I've
> > been
> > > > > going at work with some customizations we needed to use JSPWiki for
> > > this
> > > > > POC, whenever I've could look at the POC. Most of them are specific
> > to
> > > > our
> > > > > infrastructure (workflows, page storage), but the Markdown support
> is
> > > > > pretty agnostic to us so I asked if I could move it to JSPWiki,
> and I
> > > got
> > > > > my ok :-)
> > > > >
> > > > > In order to simplify things regarding code transfer I'll be putting
> > it
> > > > on a
> > > > > personal github repo (my company is already ok with that), AL
> > > licensed. I
> > > > > have to extract it from our current code-base, so it maybe some
> days
> > > > until
> > > > > I put it on a github repo for review prior to bringing in to
> master,
> > > but
> > > > > this is a heads up.
> > > > >
> > > > > Regarding the maturity of the development
> > > > > * it is POC level, meaning is working and stable, but not feature
> > > > complete.
> > > > > It's enough to demonstrate that JSPWiki can parse/render Markdown,
> > but
> > > > > there are some rough edges
> > > > > ** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash
> > > with
> > > > a
> > > > > link for all headings) that I might overlooked so there might be
> > > > > differences
> > > > > ** No toolbar support on editors (especially on plain editor).
> > Current
> > > > > toolbars aren't thought with multiple parsers on mind, and as the
> POC
> > > > > end-users are more than capable of writing markdown with their bare
> > > hands
> > > > > :-) I've preferred to focus on working the parser and the renderer
> > than
> > > > on
> > > > > this. In order to be feature complete, this should be done, and I
> > > haven't
> > > > > thought on a way of doing it yet, so any idea would be more than
> > > welcome
> > > > > O:-) (probably extending the parsers with a new method to expose
> the
> > > > markup
> > > > > associated which each toolbar button, and present it through a new
> > JSP
> > > or
> > > > > who knows how)
> > > > > * The parsing / rendering is handled with Flexmark (markdown
> parser,
> > AL
> > > > > licensed) and some extensions. It requires Java 7 (we are on Java
> 6).
> > > > Time
> > > > > to think on 2.11 O:-)?
> > > > >
> > > > >
> > > > > br,
> > > > > juan pablo
> > > > >
> > > >
> > >
> >
>

Re: Markdown support

[Juan Pablo Santos Rodríguez <ju...@gmail.com>]

Hi Dirk,

I got the same reaction when began to work on the markdown extension, I was
expecting it would
take a lot less code. I'll take a look to the markdown behaviour this week
to see how it looks like.
Couple of questions in the meantime O:-)
- would it be possible to set markdown as the only markup? so you don't
have to wrap with
%%(%)markdown (I'm thinking on a do-nothing-parser-and-renderer, so it all
gets done in the browser)
- plugins, variables, acls et all would be still rendered? (the
do-nothing-parser-and-renderer would
better be a do-almost-nothing-parser-and-renderer..)

Regarding the jspwiki-markdown, I've pushed a small fix, so it uses ASF's
snapshot repo, in order
to download latest JSPWiki snaphot, it needs at least 2.10.3-git-42 to be
able to compile. Doing an
mvn clean install on latest master should also make the compilation
problems disappear.

Finally, about " add the jspwiki-markdown as a dependency to jspwiki-war ",
that would mean
something like
https://github.com/juanpablo-santos/jspwiki-markdown/blob/master/jspwiki-markdown-war/pom.xml#L36-L49
But for that you have to have access to latest snapshot, either by
compiling latest master,
or by pulling latest commit and trying again O:-) Either way, thanks for
looking into it!


br,
juan pablo



On Sun, Nov 19, 2017 at 11:15 AM, Dirk Frederickx <dirk.frederickx@gmail.com
> wrote:

> Hi Juan,
>
> Wow !  It seems there are much more aspects to it then simple
> parsing/rendering :-)
>
> Concerning the TOC plugin,  the current generated HTML always looked funny
> to me.  (i.e. a flat list of <li> items,  versus a regular nested UL/LI
> tree)
> We'd probably should improve the current TOC plugin to render  a regular
> UL/LI tree.
> The rest is indeed pure css formatting.
>
> ====
>
>
> I tried to quickly build a %%markdown behaviour based on an open-source
> javascript MARKDOWN rendering engine (marked.js),  which fully runs in the
> browser.
> ( see https://jspwiki-wiki.apache.org/Wiki.jsp?page=Markdown%20Behavior  )
>
> Usage:
> %%markdown
> {{{
>     ##here comes your markdown!##
> }}}
> /%
>
> Note: see https://issues.apache.org/jira/browse/JSPWIKI-1040  to simplify
> this to
> %%%markdown
>     ##here comes your markdown!##
> /%
>
>
> This approach allows for mixing JSPWiki markup and MARKDOWN on the same
> page, while keeping the MARKDOWN syntax fully conform to the specification.
>
> So there is no need to extend  the MARKDOWN syntax with additional JSPWiki
> specific markup, like [{plugins}]  or %%css-style.
>
> It does support GFM, but apparently only partially.  ( more testing
> required -- this obviously depends on the chosen JS library  )
> I added support for prettified code blocks,  and section #hash-links.
> The plain editor  (with preview)  works OK.
>
> Current limitations:
> - Links require full URL format :  support for simple wiki-pages names
> would be cool
> - Table Of Contents support : re-rendering of a TOC should be done on the
> browser  (could be useful for JSPWiki anyway,  eg looking at the lack of
> TOC support for InsertPages etc...)
> - Plain-Editor:  Auto-suggestion and TAB-completion could be extended with
> MARKDOWN specific configuration.
> - ...
>
>
> ----
>
>
> BTW:
> I tried to install the jspwiki-markdown from github , but got errors with
> the mvn clean install.
> Mainly due to missing symbols...
> => " add the jspwiki-markdown as a dependency to jspwiki-war " : I need
> your help here...
>
>
> Best regards,
>
> dirk
>
>
>
>
>
> On Sun, Nov 12, 2017 at 9:41 PM, Juan Pablo Santos Rodríguez <
> juanpablo.santos@gmail.com> wrote:
>
> > Hi Dirk,
> >
> > lot of stuff to cover (meaning: long mail following), will try to address
> > all the points:
> >
> > Initial design
> > ------------------
> >
> > yes, the solution, as it is made consists on a MarkdownParser /
> > MarkdownRendered,
> > which acts as an alternative to current Parser/Renderer. There's no
> support
> > for mixed
> > markup, but I suppose that could be added too, via another
> Parser/Renderer,
> > which
> > could act as a Facade, selecting an underlying markup processing
> depending
> > on user
> > preferences (or whatever). But that raises more questions, though: once
> the
> > page is
> > stored on a given markup, what happens when a user with a different
> markup
> > stored in
> > his/her preferences? The type of markup should be also stored as a
> WikiPage
> > attribute...
> >
> > This was done as POC for evaluating using JSPWiki as a Markdown
> > content-publising
> > tool. For that flexmark is used, which does 99% of the markup parsing, so
> > moving this
> > to the client side would imply a totally different take on this. I'm
> still
> > extracting markdown
> > support from other custom code, so I expect that by Tuesday/Wednesday
> I'll
> > have it
> > uploaded to a github repo.
> >
> > As it is now, once you commit to a markup style, you're set with that. At
> > least as a
> > starting point, it's possible to add mixed markup support later on.
> >
> > The Parser/Renderer is based on Flexmark + a custom JSPWiki extension.
> > Currently
> > it supports:
> > * Normal Markdown
> > * Wiki links (internal, external, interwiki, edit, etc.)
> > * Variables
> > * Almost all plugins (more on this below)
> > * Inline images
> > * Footnotes
> > * ACLs
> > * Metadata
> > * WYSIWYG edition
> >
> > The generated HTML markup specific to JSPWiki is almost, if not equal, to
> > the current
> > JSPWiki markup Parser/Renderer.
> >
> > The editors
> > ----------------
> >
> > I've been skimming through the editors. So far, it seems to me that the
> js
> > files operate
> > on generated html (no jspwiki markup handling), so that shouldn't be a
> > problem, and the
> > JSPs doesn't seem to perform any transformation.
> >
> > As this was done as a POC, I've been focusing mostly on the
> > parsing/rendering that on
> > anything else, so it's pretty possible that I've missed something on the
> > editors side, but
> > without digging on them too much, they seemed to play along well.
> >
> > Oh, and of course, adaptations to the plain editor *need* to be done. I'm
> > thinking that the
> > different parsers should make available their configuration up to the
> > editors, so the plain
> > editor is able to generate the appropiate markup for each parser.
> >
> > Migration
> > -------------
> >
> > there's nothing yet, as the POC I was working on assumed no previous
> > content, so I haven't
> > thought a lot about this... As always, something can/could/should be
> done:
> > there is a
> > flexmark extension (haven't investigated that it, though) which performs
> an
> > html to markdown
> > conversion. Given that there are unit tests which transform current
> > markdown to html, the
> > process seems feasible, at least for simple pages. This process would not
> > be straightforward
> > JSPWikiMarkup -> HTML -> Markdown, though, some intermediate "polishing"
> > would be
> > likely to happen (I'm thinking on %%css styles, or identifying links
> which
> > are wiki links,
> > instead of full links).
> >
> > This process could also be made available as a maven plugin...
> >
> > As for the other way round (Markdown -> JSPWikiMarkup): haven't even
> > thought of it yet.
> > In any case, is migrating from one markup to another something likely to
> > happen? Once
> > you're using one markup and have a bunch of pages, what would trigger the
> > need of migrating
> > to another markup? Maybe if you're on JSPWikiMarkup you are thinking on
> > moving to
> > Markdown b/c that way you can also use the pages as Github pages, or
> > something like
> > that (although some markup would be lost). As said before, haven't
> thought
> > on this, so any
> > opinion is more than welcome...
> >
> > Yet to be done
> > ---------------------
> >
> > * Plain editor toolbar support
> >
> > * Initial set of WikiPages for markdown
> >
> > * %%css constructs. A new extension for this should be made, there is no
> > support for this.
> >
> > * markup migration tool?
> >
> > * plugins implementing HeadingListener (that is, TableOfContents) are not
> > supported: TableOfContents
> > plugin implements an ad-hoc interface, HeadingListener, which is fired by
> > JSPWikiMarkupParser every
> > time it finds a header (more precisely, for every heading,
> > JSPWikiMarkupParser generates a "#" link
> > with the section reference and then registers a HeadingListener).
> >
> > When this plugin is executed then, it knows about the different sections,
> > so it can generate the TOC.
> > The way flexmarks parses and renders markdown, doesn't allow to generate
> > the TOC this way. Initially,
> > the Markdown Parser/Renderer generated those #-links, but as soon I
> > realized TableOfContents wasn't
> > usable under markdown, I removed the generation of #-links and instead of
> > executing the plugin,
> > switched it to flexmark's own TOC extension, surrounded with some divs.
> So
> > as for now, whereas
> > TableOfContents generates something like:
> >
> > <div class="toc">
> > <div class="collapsebox">
> > <h4 id="section-TOC">title</h4>
> > <ul>/<ol> <!-- either ul or ol -->
> > <li class="toclevel-1">
> > <li class="toclevel-2">
> > <li class="toclevel-3">
> > </div>
> > </div>
> >
> > the markdown parser/render currently generates something like:
> > <div class="toc">
> > <div class="collapsebox">
> > <h4 id="section-TOC">title</h4>
> > <!-- Generated by Flexmark's TOC extension -->
> > <ul>/<ol> <!-- either ul or ol -->
> > <li> <!--level 1 -->
> >   <ul>/<ol> <!-- either ul or ol -->
> >   <li> <!--level 2 -->
> >     <ul>/<ol> <!-- either ul or ol -->
> >       <li></li> <!-- level 3 -->
> >     </ul>/</ol>
> >   </li>
> >   </ul>/</ol>
> > </li>
> > </ul>/</ol>
> > <!-- End generated by Flexmark's TOC extension -->
> > </div>
> > </div>
> >
> > So there are two ways of adding support for this:
> > a) add enough css so that both html render more or less the same.
> > Parameters parsing should be implemented anyways.
> > b) add a new Flexmark extension for TOC, probably forking+adapting the
> > existing TOC extension.
> >
> > Hope this clarifies a bit.
> >
> > br,
> > juan pablo
> >
> > On Sat, Nov 11, 2017 at 9:36 AM, Dirk Frederickx <
> > dirk.frederickx@gmail.com>
> > wrote:
> >
> > > Hi Juan,
> > >
> > > Adding markup support would indeed be an excellent extension for
> jspwiki.
> > >
> > >
> > > When changing the parser/render as you proposed, this would mean that
> > > for one jspwiki instance the backend storage would be changed from
> > > jspwiki-markup to markdown.
> > >
> > > I think this has some important limitations:
> > > - you can not use both markup styles on the same wiki instance
> > >   so users familiar with jspwiki-markup would be forced to switch
> > > - we need a solution for migrating a wiki repository between both
> markups
> > >   to allow smooth transitions
> > > - we'll need a rewrite the wysiwyg editors,  as they are converting
> > >   between html and jspwiki-markup
> > > - as you indicated we need some adaptions to the plain editor
> > >   but I believe this would mainly be configuration   (I can help with
> > that)
> > >   (configuration to adapt the auto-completions, ...)
> > >
> > > ----
> > >
> > > Alternatively, we could look for a solution whereby a user can either
> use
> > > markdown
> > > or jspwiki-markup, whenever he/she choses to.
> > > In this case, the backend would continue to use jspwiki-markup;
> > > during editing the user can chose another type of input.
> > >
> > > The user would then select a "markdown editor" store in its
> > > UserPreferences.
> > > (similar as chosing a wysiwyg editor)
> > > Markdown would be used during editing, and converted to jspwiki-markup
> > > during saving.
> > > With live-preview the user will see immediately the rendered page.
> > >
> > > This approach is similar how the wysiwyg editors work,  translating
> > between
> > > HTML
> > > and jspwiki-markup when saving the page.
> > >
> > >
> > > A topic to be addressed is the fact that markdown has a few more
> features
> > > than jspwiki-markup.
> > > For example, with markdown you can go to 6 levels with headers,
> jspwiki
> > > has only 3 levels.
> > > I guess most of these cases would be solvable through the use of
> specific
> > > %%css-class constructs.
> > > To be validated.
> > >
> > > ----
> > >
> > > Finally, we could also opt for a solution to mix different markup
> styles
> > in
> > > one page.
> > > You can write a page in jspwiki-markup,  but if you want to copy/paste
> > some
> > > markdown, it can be put inside a PLUGIN or a %%dynamic-style.
> > >
> > > The syntax of a PLUGIN (server side markup conversion) may be a bit too
> > > complex
> > > for most users.
> > >
> > > [{MarkdownPlugin
> > >
> > > ####This Is a Header####
> > > }]
> > >
> > > Taking the markdown conversion to the browser-level (with javascript)
> > would
> > > be feasible as well.
> > > (eg showdown http://www.showdownjs.com/  )
> > > %%markdown
> > >
> > > ####This Is a Header####
> > > /%
> > >
> > >
> > > ----
> > >
> > > I believe the 2nd option would be preferred.
> > > However,  a faster path to support markdown would be with a plugin or a
> > > dynamic style.
> > >
> > >
> > > br,
> > > dirk
> > >
> > >
> > >
> > >
> > > On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
> > > juanpablo.santos@gmail.com> wrote:
> > >
> > > > Hi!
> > > >
> > > > (last e-mail today, promise)
> > > >
> > > > At work we've built a POC regarding a tool to generate some static
> > sites.
> > > > We're automating JBake site's generation from some assets, templates
> > and
> > > > markdown content (b/c JBake understands markdown). For content
> authors
> > we
> > > > were looking for a good web-based contents-editor, with versioning
> > > > capabilities, visually appealing, integrated with our ldap...
> Something
> > > > like JSPWiki O:-) but supporting markdown.
> > > >
> > > > Given that current master allows switching parsers/renderers, I've
> been
> > > > going at work with some customizations we needed to use JSPWiki for
> > this
> > > > POC, whenever I've could look at the POC. Most of them are specific
> to
> > > our
> > > > infrastructure (workflows, page storage), but the Markdown support is
> > > > pretty agnostic to us so I asked if I could move it to JSPWiki, and I
> > got
> > > > my ok :-)
> > > >
> > > > In order to simplify things regarding code transfer I'll be putting
> it
> > > on a
> > > > personal github repo (my company is already ok with that), AL
> > licensed. I
> > > > have to extract it from our current code-base, so it maybe some days
> > > until
> > > > I put it on a github repo for review prior to bringing in to master,
> > but
> > > > this is a heads up.
> > > >
> > > > Regarding the maturity of the development
> > > > * it is POC level, meaning is working and stable, but not feature
> > > complete.
> > > > It's enough to demonstrate that JSPWiki can parse/render Markdown,
> but
> > > > there are some rough edges
> > > > ** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash
> > with
> > > a
> > > > link for all headings) that I might overlooked so there might be
> > > > differences
> > > > ** No toolbar support on editors (especially on plain editor).
> Current
> > > > toolbars aren't thought with multiple parsers on mind, and as the POC
> > > > end-users are more than capable of writing markdown with their bare
> > hands
> > > > :-) I've preferred to focus on working the parser and the renderer
> than
> > > on
> > > > this. In order to be feature complete, this should be done, and I
> > haven't
> > > > thought on a way of doing it yet, so any idea would be more than
> > welcome
> > > > O:-) (probably extending the parsers with a new method to expose the
> > > markup
> > > > associated which each toolbar button, and present it through a new
> JSP
> > or
> > > > who knows how)
> > > > * The parsing / rendering is handled with Flexmark (markdown parser,
> AL
> > > > licensed) and some extensions. It requires Java 7 (we are on Java 6).
> > > Time
> > > > to think on 2.11 O:-)?
> > > >
> > > >
> > > > br,
> > > > juan pablo
> > > >
> > >
> >
>

Re: Markdown support

[Dirk Frederickx <di...@gmail.com>]

Hi Juan,

Wow !  It seems there are much more aspects to it then simple
parsing/rendering :-)

Concerning the TOC plugin,  the current generated HTML always looked funny
to me.  (i.e. a flat list of <li> items,  versus a regular nested UL/LI
tree)
We'd probably should improve the current TOC plugin to render  a regular
UL/LI tree.
The rest is indeed pure css formatting.

====


I tried to quickly build a %%markdown behaviour based on an open-source
javascript MARKDOWN rendering engine (marked.js),  which fully runs in the
browser.
( see https://jspwiki-wiki.apache.org/Wiki.jsp?page=Markdown%20Behavior  )

Usage:
%%markdown
{{{
    ##here comes your markdown!##
}}}
/%

Note: see https://issues.apache.org/jira/browse/JSPWIKI-1040  to simplify
this to
%%%markdown
    ##here comes your markdown!##
/%


This approach allows for mixing JSPWiki markup and MARKDOWN on the same
page, while keeping the MARKDOWN syntax fully conform to the specification.

So there is no need to extend  the MARKDOWN syntax with additional JSPWiki
specific markup, like [{plugins}]  or %%css-style.

It does support GFM, but apparently only partially.  ( more testing
required -- this obviously depends on the chosen JS library  )
I added support for prettified code blocks,  and section #hash-links.
The plain editor  (with preview)  works OK.

Current limitations:
- Links require full URL format :  support for simple wiki-pages names
would be cool
- Table Of Contents support : re-rendering of a TOC should be done on the
browser  (could be useful for JSPWiki anyway,  eg looking at the lack of
TOC support for InsertPages etc...)
- Plain-Editor:  Auto-suggestion and TAB-completion could be extended with
MARKDOWN specific configuration.
- ...


----


BTW:
I tried to install the jspwiki-markdown from github , but got errors with
the mvn clean install.
Mainly due to missing symbols...
=> " add the jspwiki-markdown as a dependency to jspwiki-war " : I need
your help here...


Best regards,

dirk





On Sun, Nov 12, 2017 at 9:41 PM, Juan Pablo Santos Rodríguez <
juanpablo.santos@gmail.com> wrote:

> Hi Dirk,
>
> lot of stuff to cover (meaning: long mail following), will try to address
> all the points:
>
> Initial design
> ------------------
>
> yes, the solution, as it is made consists on a MarkdownParser /
> MarkdownRendered,
> which acts as an alternative to current Parser/Renderer. There's no support
> for mixed
> markup, but I suppose that could be added too, via another Parser/Renderer,
> which
> could act as a Facade, selecting an underlying markup processing depending
> on user
> preferences (or whatever). But that raises more questions, though: once the
> page is
> stored on a given markup, what happens when a user with a different markup
> stored in
> his/her preferences? The type of markup should be also stored as a WikiPage
> attribute...
>
> This was done as POC for evaluating using JSPWiki as a Markdown
> content-publising
> tool. For that flexmark is used, which does 99% of the markup parsing, so
> moving this
> to the client side would imply a totally different take on this. I'm still
> extracting markdown
> support from other custom code, so I expect that by Tuesday/Wednesday I'll
> have it
> uploaded to a github repo.
>
> As it is now, once you commit to a markup style, you're set with that. At
> least as a
> starting point, it's possible to add mixed markup support later on.
>
> The Parser/Renderer is based on Flexmark + a custom JSPWiki extension.
> Currently
> it supports:
> * Normal Markdown
> * Wiki links (internal, external, interwiki, edit, etc.)
> * Variables
> * Almost all plugins (more on this below)
> * Inline images
> * Footnotes
> * ACLs
> * Metadata
> * WYSIWYG edition
>
> The generated HTML markup specific to JSPWiki is almost, if not equal, to
> the current
> JSPWiki markup Parser/Renderer.
>
> The editors
> ----------------
>
> I've been skimming through the editors. So far, it seems to me that the js
> files operate
> on generated html (no jspwiki markup handling), so that shouldn't be a
> problem, and the
> JSPs doesn't seem to perform any transformation.
>
> As this was done as a POC, I've been focusing mostly on the
> parsing/rendering that on
> anything else, so it's pretty possible that I've missed something on the
> editors side, but
> without digging on them too much, they seemed to play along well.
>
> Oh, and of course, adaptations to the plain editor *need* to be done. I'm
> thinking that the
> different parsers should make available their configuration up to the
> editors, so the plain
> editor is able to generate the appropiate markup for each parser.
>
> Migration
> -------------
>
> there's nothing yet, as the POC I was working on assumed no previous
> content, so I haven't
> thought a lot about this... As always, something can/could/should be done:
> there is a
> flexmark extension (haven't investigated that it, though) which performs an
> html to markdown
> conversion. Given that there are unit tests which transform current
> markdown to html, the
> process seems feasible, at least for simple pages. This process would not
> be straightforward
> JSPWikiMarkup -> HTML -> Markdown, though, some intermediate "polishing"
> would be
> likely to happen (I'm thinking on %%css styles, or identifying links which
> are wiki links,
> instead of full links).
>
> This process could also be made available as a maven plugin...
>
> As for the other way round (Markdown -> JSPWikiMarkup): haven't even
> thought of it yet.
> In any case, is migrating from one markup to another something likely to
> happen? Once
> you're using one markup and have a bunch of pages, what would trigger the
> need of migrating
> to another markup? Maybe if you're on JSPWikiMarkup you are thinking on
> moving to
> Markdown b/c that way you can also use the pages as Github pages, or
> something like
> that (although some markup would be lost). As said before, haven't thought
> on this, so any
> opinion is more than welcome...
>
> Yet to be done
> ---------------------
>
> * Plain editor toolbar support
>
> * Initial set of WikiPages for markdown
>
> * %%css constructs. A new extension for this should be made, there is no
> support for this.
>
> * markup migration tool?
>
> * plugins implementing HeadingListener (that is, TableOfContents) are not
> supported: TableOfContents
> plugin implements an ad-hoc interface, HeadingListener, which is fired by
> JSPWikiMarkupParser every
> time it finds a header (more precisely, for every heading,
> JSPWikiMarkupParser generates a "#" link
> with the section reference and then registers a HeadingListener).
>
> When this plugin is executed then, it knows about the different sections,
> so it can generate the TOC.
> The way flexmarks parses and renders markdown, doesn't allow to generate
> the TOC this way. Initially,
> the Markdown Parser/Renderer generated those #-links, but as soon I
> realized TableOfContents wasn't
> usable under markdown, I removed the generation of #-links and instead of
> executing the plugin,
> switched it to flexmark's own TOC extension, surrounded with some divs. So
> as for now, whereas
> TableOfContents generates something like:
>
> <div class="toc">
> <div class="collapsebox">
> <h4 id="section-TOC">title</h4>
> <ul>/<ol> <!-- either ul or ol -->
> <li class="toclevel-1">
> <li class="toclevel-2">
> <li class="toclevel-3">
> </div>
> </div>
>
> the markdown parser/render currently generates something like:
> <div class="toc">
> <div class="collapsebox">
> <h4 id="section-TOC">title</h4>
> <!-- Generated by Flexmark's TOC extension -->
> <ul>/<ol> <!-- either ul or ol -->
> <li> <!--level 1 -->
>   <ul>/<ol> <!-- either ul or ol -->
>   <li> <!--level 2 -->
>     <ul>/<ol> <!-- either ul or ol -->
>       <li></li> <!-- level 3 -->
>     </ul>/</ol>
>   </li>
>   </ul>/</ol>
> </li>
> </ul>/</ol>
> <!-- End generated by Flexmark's TOC extension -->
> </div>
> </div>
>
> So there are two ways of adding support for this:
> a) add enough css so that both html render more or less the same.
> Parameters parsing should be implemented anyways.
> b) add a new Flexmark extension for TOC, probably forking+adapting the
> existing TOC extension.
>
> Hope this clarifies a bit.
>
> br,
> juan pablo
>
> On Sat, Nov 11, 2017 at 9:36 AM, Dirk Frederickx <
> dirk.frederickx@gmail.com>
> wrote:
>
> > Hi Juan,
> >
> > Adding markup support would indeed be an excellent extension for jspwiki.
> >
> >
> > When changing the parser/render as you proposed, this would mean that
> > for one jspwiki instance the backend storage would be changed from
> > jspwiki-markup to markdown.
> >
> > I think this has some important limitations:
> > - you can not use both markup styles on the same wiki instance
> >   so users familiar with jspwiki-markup would be forced to switch
> > - we need a solution for migrating a wiki repository between both markups
> >   to allow smooth transitions
> > - we'll need a rewrite the wysiwyg editors,  as they are converting
> >   between html and jspwiki-markup
> > - as you indicated we need some adaptions to the plain editor
> >   but I believe this would mainly be configuration   (I can help with
> that)
> >   (configuration to adapt the auto-completions, ...)
> >
> > ----
> >
> > Alternatively, we could look for a solution whereby a user can either use
> > markdown
> > or jspwiki-markup, whenever he/she choses to.
> > In this case, the backend would continue to use jspwiki-markup;
> > during editing the user can chose another type of input.
> >
> > The user would then select a "markdown editor" store in its
> > UserPreferences.
> > (similar as chosing a wysiwyg editor)
> > Markdown would be used during editing, and converted to jspwiki-markup
> > during saving.
> > With live-preview the user will see immediately the rendered page.
> >
> > This approach is similar how the wysiwyg editors work,  translating
> between
> > HTML
> > and jspwiki-markup when saving the page.
> >
> >
> > A topic to be addressed is the fact that markdown has a few more features
> > than jspwiki-markup.
> > For example, with markdown you can go to 6 levels with headers,  jspwiki
> > has only 3 levels.
> > I guess most of these cases would be solvable through the use of specific
> > %%css-class constructs.
> > To be validated.
> >
> > ----
> >
> > Finally, we could also opt for a solution to mix different markup styles
> in
> > one page.
> > You can write a page in jspwiki-markup,  but if you want to copy/paste
> some
> > markdown, it can be put inside a PLUGIN or a %%dynamic-style.
> >
> > The syntax of a PLUGIN (server side markup conversion) may be a bit too
> > complex
> > for most users.
> >
> > [{MarkdownPlugin
> >
> > ####This Is a Header####
> > }]
> >
> > Taking the markdown conversion to the browser-level (with javascript)
> would
> > be feasible as well.
> > (eg showdown http://www.showdownjs.com/  )
> > %%markdown
> >
> > ####This Is a Header####
> > /%
> >
> >
> > ----
> >
> > I believe the 2nd option would be preferred.
> > However,  a faster path to support markdown would be with a plugin or a
> > dynamic style.
> >
> >
> > br,
> > dirk
> >
> >
> >
> >
> > On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
> > juanpablo.santos@gmail.com> wrote:
> >
> > > Hi!
> > >
> > > (last e-mail today, promise)
> > >
> > > At work we've built a POC regarding a tool to generate some static
> sites.
> > > We're automating JBake site's generation from some assets, templates
> and
> > > markdown content (b/c JBake understands markdown). For content authors
> we
> > > were looking for a good web-based contents-editor, with versioning
> > > capabilities, visually appealing, integrated with our ldap... Something
> > > like JSPWiki O:-) but supporting markdown.
> > >
> > > Given that current master allows switching parsers/renderers, I've been
> > > going at work with some customizations we needed to use JSPWiki for
> this
> > > POC, whenever I've could look at the POC. Most of them are specific to
> > our
> > > infrastructure (workflows, page storage), but the Markdown support is
> > > pretty agnostic to us so I asked if I could move it to JSPWiki, and I
> got
> > > my ok :-)
> > >
> > > In order to simplify things regarding code transfer I'll be putting it
> > on a
> > > personal github repo (my company is already ok with that), AL
> licensed. I
> > > have to extract it from our current code-base, so it maybe some days
> > until
> > > I put it on a github repo for review prior to bringing in to master,
> but
> > > this is a heads up.
> > >
> > > Regarding the maturity of the development
> > > * it is POC level, meaning is working and stable, but not feature
> > complete.
> > > It's enough to demonstrate that JSPWiki can parse/render Markdown, but
> > > there are some rough edges
> > > ** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash
> with
> > a
> > > link for all headings) that I might overlooked so there might be
> > > differences
> > > ** No toolbar support on editors (especially on plain editor). Current
> > > toolbars aren't thought with multiple parsers on mind, and as the POC
> > > end-users are more than capable of writing markdown with their bare
> hands
> > > :-) I've preferred to focus on working the parser and the renderer than
> > on
> > > this. In order to be feature complete, this should be done, and I
> haven't
> > > thought on a way of doing it yet, so any idea would be more than
> welcome
> > > O:-) (probably extending the parsers with a new method to expose the
> > markup
> > > associated which each toolbar button, and present it through a new JSP
> or
> > > who knows how)
> > > * The parsing / rendering is handled with Flexmark (markdown parser, AL
> > > licensed) and some extensions. It requires Java 7 (we are on Java 6).
> > Time
> > > to think on 2.11 O:-)?
> > >
> > >
> > > br,
> > > juan pablo
> > >
> >
>

Re: Markdown support

[Juan Pablo Santos Rodríguez <ju...@gmail.com>]

Hi Dirk,

lot of stuff to cover (meaning: long mail following), will try to address
all the points:

Initial design
------------------

yes, the solution, as it is made consists on a MarkdownParser /
MarkdownRendered,
which acts as an alternative to current Parser/Renderer. There's no support
for mixed
markup, but I suppose that could be added too, via another Parser/Renderer,
which
could act as a Facade, selecting an underlying markup processing depending
on user
preferences (or whatever). But that raises more questions, though: once the
page is
stored on a given markup, what happens when a user with a different markup
stored in
his/her preferences? The type of markup should be also stored as a WikiPage
attribute...

This was done as POC for evaluating using JSPWiki as a Markdown
content-publising
tool. For that flexmark is used, which does 99% of the markup parsing, so
moving this
to the client side would imply a totally different take on this. I'm still
extracting markdown
support from other custom code, so I expect that by Tuesday/Wednesday I'll
have it
uploaded to a github repo.

As it is now, once you commit to a markup style, you're set with that. At
least as a
starting point, it's possible to add mixed markup support later on.

The Parser/Renderer is based on Flexmark + a custom JSPWiki extension.
Currently
it supports:
* Normal Markdown
* Wiki links (internal, external, interwiki, edit, etc.)
* Variables
* Almost all plugins (more on this below)
* Inline images
* Footnotes
* ACLs
* Metadata
* WYSIWYG edition

The generated HTML markup specific to JSPWiki is almost, if not equal, to
the current
JSPWiki markup Parser/Renderer.

The editors
----------------

I've been skimming through the editors. So far, it seems to me that the js
files operate
on generated html (no jspwiki markup handling), so that shouldn't be a
problem, and the
JSPs doesn't seem to perform any transformation.

As this was done as a POC, I've been focusing mostly on the
parsing/rendering that on
anything else, so it's pretty possible that I've missed something on the
editors side, but
without digging on them too much, they seemed to play along well.

Oh, and of course, adaptations to the plain editor *need* to be done. I'm
thinking that the
different parsers should make available their configuration up to the
editors, so the plain
editor is able to generate the appropiate markup for each parser.

Migration
-------------

there's nothing yet, as the POC I was working on assumed no previous
content, so I haven't
thought a lot about this... As always, something can/could/should be done:
there is a
flexmark extension (haven't investigated that it, though) which performs an
html to markdown
conversion. Given that there are unit tests which transform current
markdown to html, the
process seems feasible, at least for simple pages. This process would not
be straightforward
JSPWikiMarkup -> HTML -> Markdown, though, some intermediate "polishing"
would be
likely to happen (I'm thinking on %%css styles, or identifying links which
are wiki links,
instead of full links).

This process could also be made available as a maven plugin...

As for the other way round (Markdown -> JSPWikiMarkup): haven't even
thought of it yet.
In any case, is migrating from one markup to another something likely to
happen? Once
you're using one markup and have a bunch of pages, what would trigger the
need of migrating
to another markup? Maybe if you're on JSPWikiMarkup you are thinking on
moving to
Markdown b/c that way you can also use the pages as Github pages, or
something like
that (although some markup would be lost). As said before, haven't thought
on this, so any
opinion is more than welcome...

Yet to be done
---------------------

* Plain editor toolbar support

* Initial set of WikiPages for markdown

* %%css constructs. A new extension for this should be made, there is no
support for this.

* markup migration tool?

* plugins implementing HeadingListener (that is, TableOfContents) are not
supported: TableOfContents
plugin implements an ad-hoc interface, HeadingListener, which is fired by
JSPWikiMarkupParser every
time it finds a header (more precisely, for every heading,
JSPWikiMarkupParser generates a "#" link
with the section reference and then registers a HeadingListener).

When this plugin is executed then, it knows about the different sections,
so it can generate the TOC.
The way flexmarks parses and renders markdown, doesn't allow to generate
the TOC this way. Initially,
the Markdown Parser/Renderer generated those #-links, but as soon I
realized TableOfContents wasn't
usable under markdown, I removed the generation of #-links and instead of
executing the plugin,
switched it to flexmark's own TOC extension, surrounded with some divs. So
as for now, whereas
TableOfContents generates something like:

<div class="toc">
<div class="collapsebox">
<h4 id="section-TOC">title</h4>
<ul>/<ol> <!-- either ul or ol -->
<li class="toclevel-1">
<li class="toclevel-2">
<li class="toclevel-3">
</div>
</div>

the markdown parser/render currently generates something like:
<div class="toc">
<div class="collapsebox">
<h4 id="section-TOC">title</h4>
<!-- Generated by Flexmark's TOC extension -->
<ul>/<ol> <!-- either ul or ol -->
<li> <!--level 1 -->
  <ul>/<ol> <!-- either ul or ol -->
  <li> <!--level 2 -->
    <ul>/<ol> <!-- either ul or ol -->
      <li></li> <!-- level 3 -->
    </ul>/</ol>
  </li>
  </ul>/</ol>
</li>
</ul>/</ol>
<!-- End generated by Flexmark's TOC extension -->
</div>
</div>

So there are two ways of adding support for this:
a) add enough css so that both html render more or less the same.
Parameters parsing should be implemented anyways.
b) add a new Flexmark extension for TOC, probably forking+adapting the
existing TOC extension.

Hope this clarifies a bit.

br,
juan pablo

On Sat, Nov 11, 2017 at 9:36 AM, Dirk Frederickx <di...@gmail.com>
wrote:

> Hi Juan,
>
> Adding markup support would indeed be an excellent extension for jspwiki.
>
>
> When changing the parser/render as you proposed, this would mean that
> for one jspwiki instance the backend storage would be changed from
> jspwiki-markup to markdown.
>
> I think this has some important limitations:
> - you can not use both markup styles on the same wiki instance
>   so users familiar with jspwiki-markup would be forced to switch
> - we need a solution for migrating a wiki repository between both markups
>   to allow smooth transitions
> - we'll need a rewrite the wysiwyg editors,  as they are converting
>   between html and jspwiki-markup
> - as you indicated we need some adaptions to the plain editor
>   but I believe this would mainly be configuration   (I can help with that)
>   (configuration to adapt the auto-completions, ...)
>
> ----
>
> Alternatively, we could look for a solution whereby a user can either use
> markdown
> or jspwiki-markup, whenever he/she choses to.
> In this case, the backend would continue to use jspwiki-markup;
> during editing the user can chose another type of input.
>
> The user would then select a "markdown editor" store in its
> UserPreferences.
> (similar as chosing a wysiwyg editor)
> Markdown would be used during editing, and converted to jspwiki-markup
> during saving.
> With live-preview the user will see immediately the rendered page.
>
> This approach is similar how the wysiwyg editors work,  translating between
> HTML
> and jspwiki-markup when saving the page.
>
>
> A topic to be addressed is the fact that markdown has a few more features
> than jspwiki-markup.
> For example, with markdown you can go to 6 levels with headers,  jspwiki
> has only 3 levels.
> I guess most of these cases would be solvable through the use of specific
> %%css-class constructs.
> To be validated.
>
> ----
>
> Finally, we could also opt for a solution to mix different markup styles in
> one page.
> You can write a page in jspwiki-markup,  but if you want to copy/paste some
> markdown, it can be put inside a PLUGIN or a %%dynamic-style.
>
> The syntax of a PLUGIN (server side markup conversion) may be a bit too
> complex
> for most users.
>
> [{MarkdownPlugin
>
> ####This Is a Header####
> }]
>
> Taking the markdown conversion to the browser-level (with javascript) would
> be feasible as well.
> (eg showdown http://www.showdownjs.com/  )
> %%markdown
>
> ####This Is a Header####
> /%
>
>
> ----
>
> I believe the 2nd option would be preferred.
> However,  a faster path to support markdown would be with a plugin or a
> dynamic style.
>
>
> br,
> dirk
>
>
>
>
> On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
> juanpablo.santos@gmail.com> wrote:
>
> > Hi!
> >
> > (last e-mail today, promise)
> >
> > At work we've built a POC regarding a tool to generate some static sites.
> > We're automating JBake site's generation from some assets, templates and
> > markdown content (b/c JBake understands markdown). For content authors we
> > were looking for a good web-based contents-editor, with versioning
> > capabilities, visually appealing, integrated with our ldap... Something
> > like JSPWiki O:-) but supporting markdown.
> >
> > Given that current master allows switching parsers/renderers, I've been
> > going at work with some customizations we needed to use JSPWiki for this
> > POC, whenever I've could look at the POC. Most of them are specific to
> our
> > infrastructure (workflows, page storage), but the Markdown support is
> > pretty agnostic to us so I asked if I could move it to JSPWiki, and I got
> > my ok :-)
> >
> > In order to simplify things regarding code transfer I'll be putting it
> on a
> > personal github repo (my company is already ok with that), AL licensed. I
> > have to extract it from our current code-base, so it maybe some days
> until
> > I put it on a github repo for review prior to bringing in to master, but
> > this is a heads up.
> >
> > Regarding the maturity of the development
> > * it is POC level, meaning is working and stable, but not feature
> complete.
> > It's enough to demonstrate that JSPWiki can parse/render Markdown, but
> > there are some rough edges
> > ** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash with
> a
> > link for all headings) that I might overlooked so there might be
> > differences
> > ** No toolbar support on editors (especially on plain editor). Current
> > toolbars aren't thought with multiple parsers on mind, and as the POC
> > end-users are more than capable of writing markdown with their bare hands
> > :-) I've preferred to focus on working the parser and the renderer than
> on
> > this. In order to be feature complete, this should be done, and I haven't
> > thought on a way of doing it yet, so any idea would be more than welcome
> > O:-) (probably extending the parsers with a new method to expose the
> markup
> > associated which each toolbar button, and present it through a new JSP or
> > who knows how)
> > * The parsing / rendering is handled with Flexmark (markdown parser, AL
> > licensed) and some extensions. It requires Java 7 (we are on Java 6).
> Time
> > to think on 2.11 O:-)?
> >
> >
> > br,
> > juan pablo
> >
>

Re: Markdown support

[Dirk Frederickx <di...@gmail.com>]

Hi Juan,

Adding markup support would indeed be an excellent extension for jspwiki.


When changing the parser/render as you proposed, this would mean that
for one jspwiki instance the backend storage would be changed from
jspwiki-markup to markdown.

I think this has some important limitations:
- you can not use both markup styles on the same wiki instance
  so users familiar with jspwiki-markup would be forced to switch
- we need a solution for migrating a wiki repository between both markups
  to allow smooth transitions
- we'll need a rewrite the wysiwyg editors,  as they are converting
  between html and jspwiki-markup
- as you indicated we need some adaptions to the plain editor
  but I believe this would mainly be configuration   (I can help with that)
  (configuration to adapt the auto-completions, ...)

----

Alternatively, we could look for a solution whereby a user can either use
markdown
or jspwiki-markup, whenever he/she choses to.
In this case, the backend would continue to use jspwiki-markup;
during editing the user can chose another type of input.

The user would then select a "markdown editor" store in its
UserPreferences.
(similar as chosing a wysiwyg editor)
Markdown would be used during editing, and converted to jspwiki-markup
during saving.
With live-preview the user will see immediately the rendered page.

This approach is similar how the wysiwyg editors work,  translating between
HTML
and jspwiki-markup when saving the page.


A topic to be addressed is the fact that markdown has a few more features
than jspwiki-markup.
For example, with markdown you can go to 6 levels with headers,  jspwiki
has only 3 levels.
I guess most of these cases would be solvable through the use of specific
%%css-class constructs.
To be validated.

----

Finally, we could also opt for a solution to mix different markup styles in
one page.
You can write a page in jspwiki-markup,  but if you want to copy/paste some
markdown, it can be put inside a PLUGIN or a %%dynamic-style.

The syntax of a PLUGIN (server side markup conversion) may be a bit too
complex
for most users.

[{MarkdownPlugin

####This Is a Header####
}]

Taking the markdown conversion to the browser-level (with javascript) would
be feasible as well.
(eg showdown http://www.showdownjs.com/  )
%%markdown

####This Is a Header####
/%


----

I believe the 2nd option would be preferred.
However,  a faster path to support markdown would be with a plugin or a
dynamic style.


br,
dirk




On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
juanpablo.santos@gmail.com> wrote:

> Hi!
>
> (last e-mail today, promise)
>
> At work we've built a POC regarding a tool to generate some static sites.
> We're automating JBake site's generation from some assets, templates and
> markdown content (b/c JBake understands markdown). For content authors we
> were looking for a good web-based contents-editor, with versioning
> capabilities, visually appealing, integrated with our ldap... Something
> like JSPWiki O:-) but supporting markdown.
>
> Given that current master allows switching parsers/renderers, I've been
> going at work with some customizations we needed to use JSPWiki for this
> POC, whenever I've could look at the POC. Most of them are specific to our
> infrastructure (workflows, page storage), but the Markdown support is
> pretty agnostic to us so I asked if I could move it to JSPWiki, and I got
> my ok :-)
>
> In order to simplify things regarding code transfer I'll be putting it on a
> personal github repo (my company is already ok with that), AL licensed. I
> have to extract it from our current code-base, so it maybe some days until
> I put it on a github repo for review prior to bringing in to master, but
> this is a heads up.
>
> Regarding the maturity of the development
> * it is POC level, meaning is working and stable, but not feature complete.
> It's enough to demonstrate that JSPWiki can parse/render Markdown, but
> there are some rough edges
> ** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash with a
> link for all headings) that I might overlooked so there might be
> differences
> ** No toolbar support on editors (especially on plain editor). Current
> toolbars aren't thought with multiple parsers on mind, and as the POC
> end-users are more than capable of writing markdown with their bare hands
> :-) I've preferred to focus on working the parser and the renderer than on
> this. In order to be feature complete, this should be done, and I haven't
> thought on a way of doing it yet, so any idea would be more than welcome
> O:-) (probably extending the parsers with a new method to expose the markup
> associated which each toolbar button, and present it through a new JSP or
> who knows how)
> * The parsing / rendering is handled with Flexmark (markdown parser, AL
> licensed) and some extensions. It requires Java 7 (we are on Java 6). Time
> to think on 2.11 O:-)?
>
>
> br,
> juan pablo
>