You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by sp...@apache.org on 2026/07/27 12:28:39 UTC
(tinkerpop) 01/06: Publish an agent-friendly Markdown docs mirror + llms.txt (agentdocsspec.com)
This is an automated email from the ASF dual-hosted git repository.
spmallette pushed a commit to branch afd
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit 314e5253dc7bdaed087311860261eacf7f1a9f74
Author: Stephen Mallette <st...@amazon.com>
AuthorDate: Fri Jul 24 17:26:39 2026 +0000
Publish an agent-friendly Markdown docs mirror + llms.txt (agentdocsspec.com)
TinkerPop's docs published only single, multi-MB HTML pages with no Markdown
and no llms.txt, so AI coding agents truncated on the reference guide or fell
back to stale training data. This adds a Markdown mirror of the docs — split
into agent-sized (<=50KB) cross-linked pages — plus an llms.txt discovery
index, rendered alongside the existing HTML from a single Gremlin execution.
Architecture (the "neutral tree"): the executed Gremlin examples are captured
in a format-neutral tab model and rendered to both backends, so HTML/Markdown
parity of the executed ==> output is guaranteed by construction. HTML output
is unchanged except for one added agent-discovery directive.
Extension (docs/tinkeradoc-extension):
- GremlinHighlighter: CodeRay highlighting + conum injection extracted from the
treeprocessor into an HTML-only rendering seam.
- NeutralTab / NeutralTabCodec: format-neutral tab model (label/language/kind/
plain text) with a dependency-free JSON codec.
- HtmlTabRenderer / MarkdownTabRenderer: per-backend renderers over the neutral
tabs; HTML reproduces the existing radio-tab widget byte-for-byte.
- GremlinExecutionCache: keyed store so live Gremlin runs once across the two
per-backend render passes (AsciidoctorJ re-runs the treeprocessor per
convert()).
- MarkdownConverter (tpmarkdown backend) + GremlinDocsConverterRegistry (SPI):
renders headings, prose, lists, admonitions, tables, images, and inline
formatting to GitHub-flavored Markdown; emits explicit <a id> anchors so
xrefs resolve; substitutes the x.y.z version (postprocessor output is
discarded for custom StringConverter backends); resolves images against
imagesdir; supports curated [llms-summary] section descriptions via a hidden
comment that never renders into HTML/Markdown body.
- MarkdownSplitter: size-driven, heading-aligned page splitting with cross-page
xref rewriting and per-page llms.txt pointer; UTF-8 byte budgeting.
- LlmsTxtGenerator: llms.txt index (H1 + summary + per-book sections), curated
descriptions preferred over a first-prose fallback, absolute-URL links.
- GremlinTreeprocessor / GremlinPostprocessor: emit neutral tabs, render by
backend, cache-aware execution; CodeRay cleanup made HTML-only, dead callout
patterns removed.
Build & publish:
- pom.xml: a tpmarkdown <execution> per book mirroring each html5 book; raised
docs-build heap.
- bin/process-docs.sh: portable server-readiness probe (nc-optional), splits
Markdown and generates llms.txt after render.
- bin/publish-docs.sh: stages the .md tree beside the HTML in SVN and writes a
site-root llms.txt with absolute URLs.
- docs/src/docinfo-footer.html: agent-discovery directive in the HTML DOM.
- bin/validate-llms-txt.sh: run `npx afdocs check` against the published site
or a local build.
Tests: 171 extension tests covering the neutral model, both renderers, the
codec, execution cache, splitter, llms.txt generator, and dual-backend parity.
Assisted-by: Claude Code:claude-opus-4-8
---
bin/process-docs.sh | 63 ++-
bin/publish-docs.sh | 32 ++
bin/validate-llms-txt.sh | 177 ++++++++
docs/src/docinfo-footer.html | 17 +
.../tinkeradoc/GremlinDocsConverterRegistry.java | 36 ++
.../tinkeradoc/GremlinExecutionCache.java | 94 +++++
.../tinkerpop/tinkeradoc/GremlinHighlighter.java | 117 ++++++
.../tinkerpop/tinkeradoc/GremlinPostprocessor.java | 37 +-
.../tinkerpop/tinkeradoc/GremlinTreeprocessor.java | 213 +++++-----
.../tinkerpop/tinkeradoc/HtmlTabRenderer.java | 67 +++
.../tinkerpop/tinkeradoc/LlmsTxtGenerator.java | 288 +++++++++++++
.../tinkerpop/tinkeradoc/MarkdownConverter.java | 451 ++++++++++++++++++++
.../tinkerpop/tinkeradoc/MarkdownSplitter.java | 461 +++++++++++++++++++++
.../tinkerpop/tinkeradoc/MarkdownTabRenderer.java | 103 +++++
.../apache/tinkerpop/tinkeradoc/NeutralTab.java | 84 ++++
.../tinkerpop/tinkeradoc/NeutralTabCodec.java | 221 ++++++++++
...ciidoctor.jruby.converter.spi.ConverterRegistry | 1 +
.../tinkeradoc/DualBackendIntegrationTest.java | 118 ++++++
.../tinkeradoc/GremlinExecutionCacheTest.java | 124 ++++++
.../tinkeradoc/GremlinHighlighterTest.java | 72 ++++
.../tinkerpop/tinkeradoc/HtmlTabRendererTest.java | 104 +++++
.../tinkerpop/tinkeradoc/LlmsTxtGeneratorTest.java | 137 ++++++
.../tinkeradoc/MarkdownConverterProbeTest.java | 180 ++++++++
.../tinkerpop/tinkeradoc/MarkdownSplitterTest.java | 202 +++++++++
.../tinkeradoc/MarkdownTabRendererTest.java | 113 +++++
.../tinkerpop/tinkeradoc/NeutralTabCodecTest.java | 113 +++++
.../tinkeradoc/PostprocessorBackendTest.java | 79 ++++
pom.xml | 275 ++++++++++++
28 files changed, 3860 insertions(+), 119 deletions(-)
diff --git a/bin/process-docs.sh b/bin/process-docs.sh
index 31bfd662f6..4bfa1bc863 100755
--- a/bin/process-docs.sh
+++ b/bin/process-docs.sh
@@ -34,6 +34,13 @@ set -e
cd "$(dirname "$0")/.."
TP_HOME="$(pwd)"
+# The docs build renders every book to both HTML and Markdown in a single Maven JVM (AsciidoctorJ
+# runs in-process), so the heap must hold the parsed AST + JRuby runtime for ~two dozen executions.
+# Raise the heap unless the caller already set one, to avoid OutOfMemoryError during rendering.
+if [[ "${MAVEN_OPTS:-}" != *-Xmx* ]]; then
+ export MAVEN_OPTS="${MAVEN_OPTS:-} -Xmx8g"
+fi
+
# ---------------------------------------------------------------------------
# Argument parsing
# ---------------------------------------------------------------------------
@@ -75,6 +82,49 @@ trap cleanup EXIT
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
+# ---------------------------------------------------------------------------
+# Port readiness probe
+# ---------------------------------------------------------------------------
+# Returns 0 if a TCP port on the given host is accepting connections. Prefer nc when it is on the
+# PATH, but fall back to bash's /dev/tcp so the build still works on hosts without netcat (some
+# minimal container images and sandboxes ship no nc). Both branches are silent.
+port_open() {
+ local host="$1" port="$2"
+ if command -v nc >/dev/null 2>&1; then
+ nc -z "${host}" "${port}" 2>/dev/null
+ else
+ (exec 3<>"/dev/tcp/${host}/${port}") 2>/dev/null && exec 3>&- 2>/dev/null
+ fi
+}
+
+# ---------------------------------------------------------------------------
+# Markdown split (agentdocsspec.com)
+# ---------------------------------------------------------------------------
+# After the Markdown mirror is rendered under target/docs/markdown/, split each rendered book into
+# agent-sized (<=50KB) pages and rewrite intra-book links. Runs MarkdownSplitter from the
+# tinkeradoc-extension classes (built by the asciidoc profile) with asciidoctorj on the classpath.
+split_markdown() {
+ local md_root="target/docs/markdown"
+ [ -d "${md_root}" ] || return 0
+ local ext_classes="docs/tinkeradoc-extension/target/classes"
+ if [ ! -d "${ext_classes}" ]; then
+ echo "WARNING: ${ext_classes} not found; skipping Markdown split."
+ return 0
+ fi
+ # asciidoctorj jars are only needed for the extension build, not for the splitter (pure Java),
+ # so the extension classes alone suffice on the classpath.
+ local books
+ books=$(find "${md_root}" -name index.md 2>/dev/null)
+ [ -z "${books}" ] && return 0
+ echo "Splitting Markdown books into agent-sized pages..."
+ # shellcheck disable=SC2086
+ java -cp "${ext_classes}" org.apache.tinkerpop.tinkeradoc.MarkdownSplitter ${books}
+
+ # Generate the llms.txt discovery index over the split pages (agentdocsspec.com).
+ echo "Generating llms.txt discovery index..."
+ java -cp "${ext_classes}" org.apache.tinkerpop.tinkeradoc.LlmsTxtGenerator "${md_root}"
+}
+
# ---------------------------------------------------------------------------
# Dry-run mode
# ---------------------------------------------------------------------------
@@ -87,7 +137,9 @@ if [ "${MODE}" == "dry" ]; then
cp -r docs/{static,stylesheets} target/postprocess-asciidoc/
cp -r docs/src/* target/postprocess-asciidoc/
mvn process-resources -pl . -Dasciidoc -Dgremlin.docs.dryrun=true
- exit $?
+ ec=$?
+ [ ${ec} -eq 0 ] && split_markdown
+ exit ${ec}
fi
# ---------------------------------------------------------------------------
@@ -226,7 +278,7 @@ mkdir -p target
# server to fail binding while the readiness check still passes -- the console
# then connects to the wrong service and WebSocket handshakes fail, dumping
# large stacktraces into the rendered docs.
-if nc -z localhost 8182 2>/dev/null; then
+if port_open 127.0.0.1 8182; then
echo "ERROR: Port 8182 is already in use by another process."
echo " Gremlin Server needs this port for the docs ':remote' examples."
echo " Identify the process with 'lsof -nP -iTCP:8182' and stop it,"
@@ -248,7 +300,7 @@ for i in $(seq 1 30); do
echo "ERROR: Gremlin Server process exited during startup. See target/gremlin-server-docs.log"
exit 1
fi
- if nc -z localhost 8182 2>/dev/null; then
+ if port_open 127.0.0.1 8182; then
echo " ready."
break
fi
@@ -262,7 +314,7 @@ for i in $(seq 1 30); do
done
# 6. Start Gephi mock (port 8080)
-if ! nc -z localhost 8080 2>/dev/null; then
+if ! port_open 127.0.0.1 8080; then
echo "Starting Gephi mock on port 8080..."
bin/gephi-mock.py > /dev/null 2>&1 &
GEPHI_MOCK_PID=$!
@@ -290,7 +342,8 @@ ec=$?
set -e
if [ ${ec} -eq 0 ]; then
- echo "Documentation build complete. Output: target/docs/htmlsingle/"
+ split_markdown
+ echo "Documentation build complete. Output: target/docs/htmlsingle/ and target/docs/markdown/"
else
echo "ERROR: Documentation build failed."
fi
diff --git a/bin/publish-docs.sh b/bin/publish-docs.sh
index 38ca6bccd5..06639ce989 100755
--- a/bin/publish-docs.sh
+++ b/bin/publish-docs.sh
@@ -64,6 +64,27 @@ mkdir -p ../dotnetdocs/_site
rm -rf ../dotnetdocs/_site/*
cp -R ../../gremlin-dotnet/src/_site/ ../dotnetdocs/_site/
+# Stage the agent-friendly Markdown mirror + llms.txt (agentdocsspec.com) alongside the HTML, so
+# each page is reachable at both docs/<version>/<path>.html and .../<path>.md, and the discovery
+# index is served from the site root at /llms.txt. Copy the whole tree into the HTML output area
+# before diffing so the existing docs diff/copy/add loop below carries the .md files too.
+#
+# Before staging, regenerate the per-version llms.txt (docs/<version>/llms.txt) with ABSOLUTE URLs:
+# the spec's `llms-txt-links-resolve` / `-links-markdown` checks only recognize full http(s):// URLs
+# (relative paths are treated as "no links"). The local build's llms.txt intentionally uses relative
+# links for offline inspection; publishing rewrites them to the canonical site URLs.
+SITE_DOCS_URL="https://tinkerpop.apache.org/docs"
+PUBLISH_EXT_CLASSES="../../docs/tinkeradoc-extension/target/classes"
+if [ -d ../docs/markdown ]; then
+ if [ -d "${PUBLISH_EXT_CLASSES}" ]; then
+ echo "Regenerating per-version llms.txt with absolute URLs..."
+ java -cp "${PUBLISH_EXT_CLASSES}" org.apache.tinkerpop.tinkeradoc.LlmsTxtGenerator \
+ --prefix "${SITE_DOCS_URL}/${VERSION}/" --out ../docs/markdown/llms.txt ../docs/markdown
+ fi
+ echo "Staging Markdown mirror into htmlsingle for publish..."
+ cp -R ../docs/markdown/. ../docs/htmlsingle/
+fi
+
diff -rq -I '^Last updated' docs/${VERSION}/ ../docs/htmlsingle/ | awk -f ../../bin/publish-docs.awk | sed 's/^\(.\) \//\1 /g' > ../publish-docs.docs
diff -rq -I 'Generated by javadoc' -I '^<meta name="date"' javadocs/${VERSION}/ ../site/apidocs/ | awk -f ../../bin/publish-docs.awk | sed 's/^\(.\) \//\1 /g' > ../publish-docs.javadocs
diff -rq -I 'Generated by jsdoc' -I '^<meta name="date"' jsdocs/${VERSION}/ ../jsdocs/doc/ | awk -f ../../bin/publish-docs.awk | sed 's/^\(.\) \//\1 /g' > ../publish-docs.jsdocs
@@ -103,6 +124,17 @@ do
fi
done
+# Generate the site-root /llms.txt (agentdocsspec.com discovery index) with ABSOLUTE links into the
+# versioned docs tree, matching the per-page "> ... see [llms.txt](/llms.txt)" pointer. Absolute
+# URLs are required for the spec's link-resolution checks. This index lives at the site root; the
+# per-version llms.txt (also absolute now) is published alongside the pages via the loop above.
+if [ -d "../docs/markdown" ] && [ -d "${PUBLISH_EXT_CLASSES}" ]; then
+ echo "Generating site-root llms.txt..."
+ java -cp "${PUBLISH_EXT_CLASSES}" org.apache.tinkerpop.tinkeradoc.LlmsTxtGenerator \
+ --prefix "${SITE_DOCS_URL}/${VERSION}/" --out llms.txt ../docs/markdown
+ ${SVN_CMD} add --force llms.txt 2>/dev/null || true
+fi
+
pushd "docs/${VERSION}/"; cat ../../../publish-docs.docs | awk '/^A/ {print $2}' | grep -v '.graffle$' | xargs --no-run-if-empty svn add --parents; popd
pushd "javadocs/${VERSION}/"; cat ../../../publish-docs.javadocs | awk '/^A/ {print $2}' | xargs --no-run-if-empty svn add --parents; popd
pushd "jsdocs/${VERSION}/"; cat ../../../publish-docs.jsdocs | awk '/^A/ {print $2}' | xargs --no-run-if-empty svn add --parents; popd
diff --git a/bin/validate-llms-txt.sh b/bin/validate-llms-txt.sh
new file mode 100755
index 0000000000..69fb42abe0
--- /dev/null
+++ b/bin/validate-llms-txt.sh
@@ -0,0 +1,177 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Validate the agent-friendly documentation (agentdocsspec.com) — the Markdown mirror, the
+# llms.txt discovery index, page sizes, link resolution, and the per-page directive — using the
+# spec's own tool, `npx afdocs check`.
+#
+# Usage:
+# bin/validate-llms-txt.sh # check the published current docs
+# # (https://tinkerpop.apache.org/docs/current/)
+# bin/validate-llms-txt.sh -v 3.7.7 # check a published version
+# bin/validate-llms-txt.sh --version 3.7.7 # (https://tinkerpop.apache.org/docs/3.7.7/)
+# bin/validate-llms-txt.sh -v local # check the LOCAL build: replicate the publish layout
+# # (markdown + html + images together) under
+# # target/docs/serve/, serve it, and check localhost
+#
+# Any extra arguments after the mode are passed through to `afdocs check` (e.g. --fixes, --verbose,
+# --format json, --skip-checks ...).
+#
+# NOTE: afdocs requires Node.js >= 22; this script warns if the active node is older. A plain local
+# static server cannot exercise the hosting-dependent checks (content-negotiation, redirects,
+# soft-404, cache headers, auth) — those are only meaningful against the real published site.
+
+set -e
+
+cd "$(dirname "$0")/.."
+TP_HOME="$(pwd)"
+
+BASE_URL="https://tinkerpop.apache.org/docs"
+VERSION="current"
+
+# ---------------------------------------------------------------------------
+# Argument parsing
+# ---------------------------------------------------------------------------
+PASSTHROUGH=()
+while [[ $# -gt 0 ]]; do
+ case "$1" in
+ -v|--version) VERSION="$2"; shift 2 ;;
+ -h|--help)
+ sed -n '19,40p' "$0" | sed 's/^# \{0,1\}//'
+ exit 0 ;;
+ *) PASSTHROUGH+=("$1"); shift ;;
+ esac
+done
+
+# ---------------------------------------------------------------------------
+# Node version advisory (afdocs requires >= 22)
+# ---------------------------------------------------------------------------
+if command -v node >/dev/null 2>&1; then
+ NODE_MAJOR=$(node -v | sed 's/^v//; s/\..*//')
+ if [ "${NODE_MAJOR:-0}" -lt 22 ]; then
+ echo "WARNING: afdocs requires Node.js >= 22 but found $(node -v)."
+ echo " Results may be unreliable. Install a newer node (e.g. 'mise install node@22')."
+ fi
+else
+ echo "ERROR: node/npx not found on PATH; afdocs needs Node.js >= 22."
+ exit 1
+fi
+
+# ---------------------------------------------------------------------------
+# Local mode: replicate the publish layout and serve it
+# ---------------------------------------------------------------------------
+HTTP_PID=""
+cleanup() {
+ set +e
+ [ -n "${HTTP_PID}" ] && kill "${HTTP_PID}" 2>/dev/null
+}
+trap cleanup EXIT INT TERM
+
+run_afdocs() {
+ local url="$1"; shift
+ local extra=("$@")
+ echo "Running: npx --yes afdocs check ${url} ${extra[*]} ${PASSTHROUGH[*]}"
+ echo "-----------------------------------------------------------------------"
+ npx --yes afdocs@latest check "${url}" "${extra[@]}" "${PASSTHROUGH[@]}"
+}
+
+if [ "${VERSION}" == "local" ]; then
+ MD_DIR="target/docs/markdown"
+ HTML_DIR="target/docs/htmlsingle"
+ SERVE_DIR="target/docs/serve"
+
+ if [ ! -d "${MD_DIR}" ] || [ ! -f "${MD_DIR}/llms.txt" ]; then
+ echo "ERROR: No local Markdown docs found at ${MD_DIR} (with llms.txt)."
+ echo " Generate them first, e.g.: bin/process-docs.sh --dryRun"
+ echo " (a --dryRun build produces the Markdown mirror + llms.txt without a live server;"
+ echo " a full 'bin/process-docs.sh' additionally fills in executed ==> output.)"
+ exit 1
+ fi
+
+ echo "Replicating publish layout into ${SERVE_DIR}/ (markdown + html + images together)..."
+ rm -rf "${SERVE_DIR}"
+ mkdir -p "${SERVE_DIR}"
+ # HTML first (brings images/, stylesheets, and the .html pages), then overlay the Markdown mirror
+ # so .md sits beside .html and llms.txt lands at the served root — mirroring bin/publish-docs.sh.
+ if [ -d "${HTML_DIR}" ]; then
+ cp -R "${HTML_DIR}/." "${SERVE_DIR}/"
+ else
+ echo "NOTE: ${HTML_DIR} not found; serving Markdown only (HTML/parity checks will be limited)."
+ fi
+ cp -R "${MD_DIR}/." "${SERVE_DIR}/"
+
+ # Pick a free port up front so we can bake the absolute base URL into llms.txt before serving.
+ PORT=$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1",0)); print(s.getsockname()[1]); s.close()')
+ LOCAL_URL="http://127.0.0.1:${PORT}/"
+
+ # Regenerate the served llms.txt with ABSOLUTE URLs (http://127.0.0.1:PORT/...). The spec's
+ # link-resolution/coverage checks only recognize full http(s):// links, so this mirrors what
+ # publish-docs.sh does with the canonical site URL and lets the local run exercise those checks.
+ EXT_CLASSES="docs/tinkeradoc-extension/target/classes"
+ if [ -d "${EXT_CLASSES}" ]; then
+ echo "Regenerating served llms.txt with absolute URLs for ${LOCAL_URL} ..."
+ java -cp "${EXT_CLASSES}" org.apache.tinkerpop.tinkeradoc.LlmsTxtGenerator \
+ --prefix "${LOCAL_URL}" --out "${SERVE_DIR}/llms.txt" "${SERVE_DIR}"
+ else
+ echo "NOTE: ${EXT_CLASSES} not found; serving llms.txt as-is (relative links won't satisfy"
+ echo " the link-resolution/coverage checks). Build the extension to enable absolute URLs."
+ fi
+
+ echo "Serving ${SERVE_DIR}/ at ${LOCAL_URL} ..."
+ # A threaded server: afdocs issues concurrent requests, and python's default single-threaded
+ # http.server can drop connections mid-response (undici in afdocs then aborts). ThreadingHTTPServer
+ # handles the concurrency cleanly.
+ python3 -c "
+import sys, http.server, socketserver, functools
+port = int(sys.argv[1]); root = sys.argv[2]
+handler = functools.partial(http.server.SimpleHTTPRequestHandler, directory=root)
+class T(socketserver.ThreadingMixIn, http.server.HTTPServer):
+ daemon_threads = True
+ allow_reuse_address = True
+T(('127.0.0.1', port), handler).serve_forever()
+" "${PORT}" "${SERVE_DIR}" >/dev/null 2>&1 &
+ HTTP_PID=$!
+
+ # Wait for readiness (loopback works within this single script invocation).
+ for i in $(seq 1 30); do
+ if curl -sf "http://127.0.0.1:${PORT}/llms.txt" >/dev/null 2>&1; then break; fi
+ if ! kill -0 "${HTTP_PID}" 2>/dev/null; then
+ echo "ERROR: local web server exited during startup."; exit 1
+ fi
+ [ "$i" -eq 30 ] && { echo "ERROR: local web server did not become ready."; exit 1; }
+ sleep 0.5
+ done
+
+ # Local-run options (each skippable by passing your own):
+ # --canonical-origin: tells afdocs the localhost origin is canonical, so the absolute llms.txt
+ # links (baked in above) count as same-origin and its link-resolution checks run.
+ # --max-concurrency 1 / --request-delay: afdocs' HTTP client (undici) can abort on many-page
+ # concurrent crawls against a lightweight local server; serial requests are reliable.
+ LOCAL_OPTS=()
+ [[ " ${PASSTHROUGH[*]} " == *" --canonical-origin "* ]] || LOCAL_OPTS+=(--canonical-origin "http://127.0.0.1:${PORT}")
+ [[ " ${PASSTHROUGH[*]} " == *" --max-concurrency "* ]] || LOCAL_OPTS+=(--max-concurrency 1)
+ [[ " ${PASSTHROUGH[*]} " == *" --request-delay "* ]] || LOCAL_OPTS+=(--request-delay 20)
+ run_afdocs "${LOCAL_URL}" "${LOCAL_OPTS[@]}"
+ echo "-----------------------------------------------------------------------"
+ echo "NOTE: hosting-dependent checks (content-negotiation, redirects, soft-404, cache headers,"
+ echo " auth) are not meaningful against this local static server; validate those against"
+ echo " the published site."
+else
+ run_afdocs "${BASE_URL}/${VERSION}/"
+fi
diff --git a/docs/src/docinfo-footer.html b/docs/src/docinfo-footer.html
new file mode 100644
index 0000000000..2ad47a7649
--- /dev/null
+++ b/docs/src/docinfo-footer.html
@@ -0,0 +1,17 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+<!-- Agent-facing discovery directive (agentdocsspec.com llms-txt-directive-html). Plain,
+ server-rendered, not display:none, so it survives HTML-to-Markdown conversion. -->
+<p class="agent-docs-index">For AI agents: a documentation index is available at <a href="/llms.txt">/llms.txt</a>.</p>
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinDocsConverterRegistry.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinDocsConverterRegistry.java
new file mode 100644
index 0000000000..064d997082
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinDocsConverterRegistry.java
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.Asciidoctor;
+import org.asciidoctor.jruby.converter.spi.ConverterRegistry;
+
+/**
+ * SPI entry point that registers the TinkerPop Markdown backend converter with AsciidoctorJ. This
+ * is discovered automatically (via {@code META-INF/services}) by the asciidoctor-maven-plugin, so
+ * a build execution with {@code <backend>tpmarkdown</backend>} renders through
+ * {@link MarkdownConverter}.
+ */
+public class GremlinDocsConverterRegistry implements ConverterRegistry {
+
+ @Override
+ public void register(final Asciidoctor asciidoctor) {
+ asciidoctor.javaConverterRegistry().register(MarkdownConverter.class);
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinExecutionCache.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinExecutionCache.java
new file mode 100644
index 0000000000..faa6838bb4
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinExecutionCache.java
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * A keyed store of executed {@link NeutralTab} groups, so live Gremlin runs exactly once even
+ * though the document is rendered for multiple backends (HTML and Markdown).
+ * <p>
+ * AsciidoctorJ cannot re-target a loaded document's backend, so each backend is a separate
+ * {@code convert()} that re-runs the treeprocessor. The first pass to reach a given Gremlin block
+ * executes it and stores the resulting neutral tabs here; every later pass reads the cached tabs
+ * and skips the console. Because both backends render from the <em>same</em> cached neutral data,
+ * the executed {@code ==>} output is identical across HTML and Markdown by construction.
+ * <p>
+ * The production instance is {@link #SHARED}, backed by a concurrent map so it survives across the
+ * separate Asciidoctor runtimes the asciidoctor-maven-plugin creates for each per-book
+ * {@code <execution>} within one build JVM. Tests inject a private instance to avoid cross-test
+ * leakage.
+ */
+class GremlinExecutionCache {
+
+ /**
+ * Process-wide cache used by the SPI-registered treeprocessor in production. Keys embed the
+ * source document identity and the block's ordinal (see {@link #key}), so entries from
+ * different books or positions never collide.
+ */
+ static final GremlinExecutionCache SHARED = new GremlinExecutionCache(new ConcurrentHashMap<>());
+
+ private final Map<String, List<NeutralTab>> store;
+
+ /** Creates a cache with its own private (non-shared) backing map, for tests. */
+ GremlinExecutionCache() {
+ this(new ConcurrentHashMap<>());
+ }
+
+ GremlinExecutionCache(final Map<String, List<NeutralTab>> store) {
+ this.store = store;
+ }
+
+ /**
+ * Builds a stable cache key for a Gremlin block. The key must be identical across the two
+ * backend passes (which parse the same source in the same order) yet distinct across documents
+ * and block positions, because a block's console output depends on the stateful console session
+ * up to that point.
+ *
+ * @param documentId a stable identity for the source document (e.g. its {@code docfile})
+ * @param ordinal the block's 1-based position among Gremlin blocks in the document
+ * @param source the block's source text, hashed in as an integrity guard
+ */
+ static String key(final String documentId, final int ordinal, final String source) {
+ return documentId + "#" + ordinal + "#" + (source == null ? 0 : source.hashCode());
+ }
+
+ boolean contains(final String key) {
+ return store.containsKey(key);
+ }
+
+ List<NeutralTab> get(final String key) {
+ return store.get(key);
+ }
+
+ void put(final String key, final List<NeutralTab> tabs) {
+ store.put(key, tabs);
+ }
+
+ /** Removes all entries. Intended for test isolation. */
+ void clear() {
+ store.clear();
+ }
+
+ int size() {
+ return store.size();
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinHighlighter.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinHighlighter.java
new file mode 100644
index 0000000000..aef6bae566
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinHighlighter.java
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.ast.StructuralNode;
+
+import java.util.logging.Logger;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * Turns plain source text (a console transcript or a code sample) into CodeRay-highlighted HTML,
+ * re-injecting AsciiDoc callout markers as conum spans.
+ * <p>
+ * This is the HTML-only rendering seam extracted from {@link GremlinTreeprocessor}: the executor
+ * produces format-neutral plain text and callout markers, and this class is the one place that
+ * knows how to render that neutrally-held content into HTML. It relies on the CodeRay highlighter
+ * bundled with AsciidoctorJ, accessed through the JRuby runtime obtained from an AST node.
+ */
+class GremlinHighlighter {
+
+ private static final Logger LOG = Logger.getLogger(GremlinHighlighter.class.getName());
+
+ private static final Pattern TRAILING_CALLOUTS = Pattern.compile("\\s*((<\\d+>\\s*)+)$");
+ private static final Pattern CALLOUT_NUMBER = Pattern.compile("<(\\d+)>");
+
+ private org.jruby.runtime.builtin.IRubyObject coderayEncoder;
+ private org.jruby.Ruby rubyRuntime;
+
+ /**
+ * Highlights {@code source} as {@code lang} using CodeRay via the JRuby runtime bundled with
+ * AsciidoctorJ. Callout markers ({@code <1>}) at line ends are stripped before highlighting and
+ * re-injected afterward as conum HTML, matching the output Asciidoctor produces for callouts.
+ *
+ * @param parent an AST node used to locate the JRuby runtime
+ * @param lang the source language passed to CodeRay
+ * @param source the plain source text to highlight (may contain trailing callout markers)
+ * @return highlighted HTML, or escaped plain text if highlighting is unavailable
+ */
+ String highlight(final StructuralNode parent, final String lang, final String source) {
+ if (source == null || source.isEmpty()) return "";
+ // Strip callouts before highlighting, re-inject after
+ final String[] lines = source.split("\\r?\\n");
+ final String[] calloutMarkers = new String[lines.length];
+ final StringBuilder cleanSource = new StringBuilder();
+ for (int i = 0; i < lines.length; i++) {
+ final Matcher m = TRAILING_CALLOUTS.matcher(lines[i]);
+ if (m.find()) {
+ calloutMarkers[i] = m.group(1);
+ if (i > 0) cleanSource.append("\n");
+ cleanSource.append(lines[i], 0, m.start());
+ } else {
+ calloutMarkers[i] = null;
+ if (i > 0) cleanSource.append("\n");
+ cleanSource.append(lines[i]);
+ }
+ }
+
+ final String highlighted = doHighlight(parent, lang, cleanSource.toString());
+
+ // Re-inject callouts as HTML conums
+ final String[] highlightedLines = highlighted.split("\\n", -1);
+ final StringBuilder result = new StringBuilder();
+ for (int i = 0; i < highlightedLines.length; i++) {
+ if (i > 0) result.append("\n");
+ result.append(highlightedLines[i]);
+ if (i < calloutMarkers.length && calloutMarkers[i] != null) {
+ final Matcher nums = CALLOUT_NUMBER.matcher(calloutMarkers[i]);
+ while (nums.find()) {
+ result.append(" <span class=\"comment\">//</span> <b class=\"conum\">(")
+ .append(nums.group(1)).append(")</b>");
+ }
+ }
+ }
+ return result.toString();
+ }
+
+ private String doHighlight(final StructuralNode parent, final String lang, final String source) {
+ try {
+ if (rubyRuntime == null) {
+ rubyRuntime = org.asciidoctor.jruby.internal.JRubyRuntimeContext.get(parent);
+ if (rubyRuntime == null) return escapeHtml(source);
+ coderayEncoder = rubyRuntime.evalScriptlet(
+ "require 'coderay'; CodeRay::Duo[:groovy, :html, :css => :class]");
+ }
+ final org.jruby.RubyString rubySource = org.jruby.RubyString.newString(rubyRuntime, source);
+ final org.jruby.runtime.builtin.IRubyObject result = coderayEncoder.callMethod(
+ rubyRuntime.getCurrentContext(), "highlight", rubySource);
+ return result != null ? result.asJavaString() : escapeHtml(source);
+ } catch (final Exception e) {
+ LOG.warning("CodeRay highlighting failed, falling back to plain: " + e.getMessage());
+ return escapeHtml(source);
+ }
+ }
+
+ static String escapeHtml(final String text) {
+ if (text == null) return "";
+ return text.replace("&", "&").replace("<", "<")
+ .replace(">", ">").replace("\"", """);
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinPostprocessor.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinPostprocessor.java
index eb66ba4768..e56c9f2f69 100644
--- a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinPostprocessor.java
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinPostprocessor.java
@@ -24,20 +24,13 @@ import org.asciidoctor.extension.Postprocessor;
import java.util.regex.Pattern;
/**
- * Postprocessor that applies callout fixes, removes empty comment spans,
- * and replaces x.y.z version placeholders with the actual TinkerPop version.
+ * Postprocessor that finalizes rendered output for both backends. It replaces {@code x.y.z} version
+ * placeholders with the actual TinkerPop version (applies to HTML and Markdown alike) and, for the
+ * HTML backend only, removes the empty comment spans CodeRay emits.
*/
public class GremlinPostprocessor extends Postprocessor {
- // Matches <i class="conum"> or <b class="conum"> (with possible other classes)
- private static final Pattern CONUM_PATTERN = Pattern.compile(
- "<([ib])\\s+class=\"conum\"");
-
- // Matches // preceding a callout marker, wraps in hide-when-copy span
- private static final Pattern COMMENT_BEFORE_CONUM_PATTERN = Pattern.compile(
- "//\\s*(<[ib] class=\"conum)");
-
- // Matches empty comment spans from CodeRay: <span class="comment">/* */</span>
+ // Matches empty comment spans from CodeRay: <span class="comment">/* */</span>. HTML-only.
private static final Pattern EMPTY_COMMENT_SPAN_PATTERN = Pattern.compile(
"<span class=\"comment\">/\\*\\s*\\*/</span>");
@@ -45,10 +38,14 @@ public class GremlinPostprocessor extends Postprocessor {
public String process(final Document document, final String output) {
String result = output;
- // 1. Remove empty comment spans from CodeRay
- result = EMPTY_COMMENT_SPAN_PATTERN.matcher(result).replaceAll("");
+ // 1. Remove empty comment spans from CodeRay — an HTML-highlighting artifact that has no
+ // place in Markdown output, so only strip it for the HTML backend.
+ if (isHtmlBackend(document)) {
+ result = EMPTY_COMMENT_SPAN_PATTERN.matcher(result).replaceAll("");
+ }
- // 2. Replace x.y.z with actual version
+ // 2. Replace x.y.z with the actual version. This applies to every backend so both the HTML
+ // and the Markdown mirror carry the concrete version string.
final String version = resolveVersion(document);
if (version != null) {
result = result.replace("x.y.z", version);
@@ -57,6 +54,18 @@ public class GremlinPostprocessor extends Postprocessor {
return result;
}
+ /**
+ * Whether the active backend is an HTML backend. The Markdown backend ({@code tpmarkdown}) must
+ * skip HTML-specific cleanups.
+ */
+ private static boolean isHtmlBackend(final Document document) {
+ final Object backend = document.getAttribute("backend");
+ // Default to HTML when unknown so existing HTML behavior is never weakened.
+ if (backend == null) return true;
+ final String b = backend.toString();
+ return b.startsWith("html") || b.equals("xhtml") || b.equals("html5");
+ }
+
private String resolveVersion(final Document document) {
Object version = document.getAttribute("tinkerpop-version");
if (version != null) {
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinTreeprocessor.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinTreeprocessor.java
index b2274df899..0106f33dba 100644
--- a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinTreeprocessor.java
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/GremlinTreeprocessor.java
@@ -71,45 +71,73 @@ public class GremlinTreeprocessor extends Treeprocessor {
private int gremlinBlockCount;
private final StatementExecutor executor;
private final TabbedHtmlBuilder tabBuilder;
+ private final GremlinHighlighter highlighter;
+ private final HtmlTabRenderer htmlTabRenderer;
+ private final MarkdownTabRenderer markdownTabRenderer;
+ private final GremlinExecutionCache executionCache;
private final ConsoleRestartHandler restartHandler;
private ConsoleRestartHandler activeRestartHandler;
private String currentGraph;
+ private String documentId;
private List<String> currentExcludedPlugins;
/**
* Creates a GremlinTreeprocessor that processes blocks without executing them (dry-run mode).
+ * Uses the shared process-wide execution cache (production/SPI path).
*/
public GremlinTreeprocessor() {
- this((StatementExecutor) null, null);
+ this((StatementExecutor) null, null, GremlinExecutionCache.SHARED);
}
/**
- * Creates a GremlinTreeprocessor that executes blocks against the provided console.
+ * Creates a GremlinTreeprocessor that executes blocks against the provided console. Uses the
+ * shared process-wide execution cache (production path) so live execution happens once across
+ * the per-backend render passes.
*
* @param console the GremlinConsole to execute statements against, or null for dry-run
*/
public GremlinTreeprocessor(final GremlinConsole console) {
- this(console == null ? null : statement -> console.execute(statement), null);
+ this(console == null ? null : statement -> console.execute(statement), null,
+ GremlinExecutionCache.SHARED);
}
/**
- * Creates a GremlinTreeprocessor with a custom statement executor for testing.
+ * Creates a GremlinTreeprocessor with a custom statement executor for testing. Uses a private
+ * execution cache so cached tabs do not leak between tests.
*
* @param executor the executor to use, or null for dry-run
*/
GremlinTreeprocessor(final StatementExecutor executor) {
- this(executor, null);
+ this(executor, null, new GremlinExecutionCache());
}
/**
- * Creates a GremlinTreeprocessor with a custom statement executor and restart handler.
+ * Creates a GremlinTreeprocessor with a custom statement executor and restart handler for
+ * testing. Uses a private execution cache so cached tabs do not leak between tests.
*
* @param executor the executor to use, or null for dry-run
* @param restartHandler the handler invoked when plugin exclusions change, or null to ignore
*/
GremlinTreeprocessor(final StatementExecutor executor, final ConsoleRestartHandler restartHandler) {
+ this(executor, restartHandler, new GremlinExecutionCache());
+ }
+
+ /**
+ * Creates a GremlinTreeprocessor with an explicit execution cache. Tests use this to inject a
+ * private cache so cached tabs do not leak between tests.
+ *
+ * @param executor the executor to use, or null for dry-run
+ * @param restartHandler the handler invoked when plugin exclusions change, or null to ignore
+ * @param executionCache the cache of executed neutral tab groups, keyed per block
+ */
+ GremlinTreeprocessor(final StatementExecutor executor, final ConsoleRestartHandler restartHandler,
+ final GremlinExecutionCache executionCache) {
this.executor = executor;
this.tabBuilder = new TabbedHtmlBuilder();
+ this.highlighter = new GremlinHighlighter();
+ this.htmlTabRenderer = new HtmlTabRenderer(this.highlighter, this.tabBuilder);
+ this.markdownTabRenderer = new MarkdownTabRenderer();
+ this.executionCache = executionCache;
this.restartHandler = restartHandler;
}
@@ -117,12 +145,15 @@ public class GremlinTreeprocessor extends Treeprocessor {
private GremlinConsole lazyConsole;
private Path consoleHomePath;
private boolean dryRun;
+ private boolean markdownMode;
private boolean sugarLoaded;
@Override
public Document process(final Document document) {
gremlinBlockCount = 0;
currentGraph = null;
+ documentId = resolveDocumentId(document);
+ markdownMode = isMarkdownBackend(document);
final Object dryRunAttr = document.getAttribute("gremlin-docs-dryrun");
dryRun = dryRunAttr != null && !"false".equals(dryRunAttr.toString());
@@ -155,6 +186,21 @@ public class GremlinTreeprocessor extends Treeprocessor {
return document;
}
+ /**
+ * Resolves a stable identity for the source document, used to namespace execution-cache keys so
+ * blocks from different books or files never collide. Prefers the {@code docfile} attribute (the
+ * absolute source path, identical across backend passes); falls back to the document title, then
+ * a constant, both of which still combine with the block ordinal and source hash in the key.
+ */
+ private static String resolveDocumentId(final Document document) {
+ final Object docfile = document.getAttribute("docfile");
+ if (docfile != null && !docfile.toString().isEmpty()) {
+ return docfile.toString();
+ }
+ final String title = document.getTitle();
+ return title != null && !title.isEmpty() ? title : "<no-docfile>";
+ }
+
/**
* Starts the console on demand when the first gremlin block is encountered.
*/
@@ -271,106 +317,43 @@ public class GremlinTreeprocessor extends Treeprocessor {
final List<StructuralNode> blocks = parent.getBlocks();
final Block gremlinBlock = (Block) blocks.get(startIndex);
- final String consoleOutput = buildConsoleOutput(gremlinBlock, dryRun);
- final List<TabbedHtmlBuilder.Tab> tabs = new ArrayList<>();
- tabs.add(TabbedHtmlBuilder.consoleTabHighlighted("groovy",
- highlightAsGroovy(parent, consoleOutput)));
- // Add second tab with clean source code (no prompts/output)
- tabs.add(TabbedHtmlBuilder.codeTabHighlighted("groovy",
- highlightAsGroovy(parent, gremlinBlock.getSource())));
-
- // Consume consecutive [source,<lang>] sibling blocks as manual tabs (FR-5)
+ // Walk consecutive [source,<lang>] sibling blocks first (FR-5). This is pure AST work with
+ // no execution: it fixes the block range to replace and captures each sibling's language
+ // and source, which are needed whether the console output comes fresh or from the cache.
int lastIndex = startIndex;
+ final List<String[]> siblingSources = new ArrayList<>(); // [lang, source]
for (int j = startIndex + 1; j < blocks.size(); j++) {
final StructuralNode sibling = blocks.get(j);
if (isManualTabBlock(sibling)) {
final Block sourceBlock = (Block) sibling;
- final String lang = getSourceLanguage(sourceBlock);
- tabs.add(TabbedHtmlBuilder.codeTabHighlighted(lang,
- highlightAsSource(parent, lang, sourceBlock.getSource())));
+ siblingSources.add(new String[]{getSourceLanguage(sourceBlock), sourceBlock.getSource()});
lastIndex = j;
} else {
break;
}
}
- final String html = tabBuilder.build(tabs);
- replaceWithPassBlock(parent, startIndex, lastIndex, html);
- return startIndex;
- }
-
- /**
- * Highlights source code using CodeRay via the JRuby runtime bundled with AsciidoctorJ.
- */
- private String highlightAsGroovy(final StructuralNode parent, final String source) {
- return highlightAsSource(parent, "groovy", source);
- }
-
- private String highlightAsSource(final StructuralNode parent, final String lang, final String source) {
- if (source == null || source.isEmpty()) return "";
- // Strip callouts before highlighting, re-inject after
- final String[] lines = source.split("\\r?\\n");
- final String[] calloutMarkers = new String[lines.length];
- final StringBuilder cleanSource = new StringBuilder();
- for (int i = 0; i < lines.length; i++) {
- final java.util.regex.Matcher m = java.util.regex.Pattern
- .compile("\\s*((<\\d+>\\s*)+)$").matcher(lines[i]);
- if (m.find()) {
- calloutMarkers[i] = m.group(1);
- if (i > 0) cleanSource.append("\n");
- cleanSource.append(lines[i], 0, m.start());
- } else {
- calloutMarkers[i] = null;
- if (i > 0) cleanSource.append("\n");
- cleanSource.append(lines[i]);
- }
- }
-
- String highlighted = doHighlight(parent, lang, cleanSource.toString());
-
- // Re-inject callouts as HTML conums
- final String[] highlightedLines = highlighted.split("\\n", -1);
- final StringBuilder result = new StringBuilder();
- for (int i = 0; i < highlightedLines.length; i++) {
- if (i > 0) result.append("\n");
- result.append(highlightedLines[i]);
- if (i < calloutMarkers.length && calloutMarkers[i] != null) {
- final java.util.regex.Matcher nums = java.util.regex.Pattern
- .compile("<(\\d+)>").matcher(calloutMarkers[i]);
- while (nums.find()) {
- result.append(" <span class=\"comment\">//</span> <b class=\"conum\">(")
- .append(nums.group(1)).append(")</b>");
- }
- }
- }
- return result.toString();
- }
-
- private org.jruby.runtime.builtin.IRubyObject coderayEncoder;
- private org.jruby.Ruby rubyRuntime;
-
- private String doHighlight(final StructuralNode parent, final String lang, final String source) {
- try {
- if (rubyRuntime == null) {
- rubyRuntime = org.asciidoctor.jruby.internal.JRubyRuntimeContext.get(parent);
- if (rubyRuntime == null) return escapeHtml(source);
- coderayEncoder = rubyRuntime.evalScriptlet(
- "require 'coderay'; CodeRay::Duo[:groovy, :html, :css => :class]");
+ // Execute once: on the first backend pass this runs Gremlin and caches the neutral tabs;
+ // on later backend passes the cache hit skips the console entirely, so live execution
+ // happens exactly once and both backends render from identical executed output.
+ final String cacheKey = GremlinExecutionCache.key(documentId, gremlinBlockCount, gremlinBlock.getSource());
+ final List<NeutralTab> tabs;
+ if (executionCache.contains(cacheKey)) {
+ tabs = executionCache.get(cacheKey);
+ } else {
+ final String consoleOutput = buildConsoleOutput(gremlinBlock, dryRun);
+ tabs = new ArrayList<>();
+ tabs.add(NeutralTab.console("groovy", consoleOutput));
+ // Second tab with clean source code (no prompts/output)
+ tabs.add(NeutralTab.source("groovy", gremlinBlock.getSource()));
+ for (final String[] sibling : siblingSources) {
+ tabs.add(NeutralTab.source(sibling[0], sibling[1]));
}
- final org.jruby.RubyString rubySource = org.jruby.RubyString.newString(rubyRuntime, source);
- final org.jruby.runtime.builtin.IRubyObject result = coderayEncoder.callMethod(
- rubyRuntime.getCurrentContext(), "highlight", rubySource);
- return result != null ? result.asJavaString() : escapeHtml(source);
- } catch (final Exception e) {
- LOG.warning("CodeRay highlighting failed, falling back to plain: " + e.getMessage());
- return escapeHtml(source);
+ executionCache.put(cacheKey, tabs);
}
- }
- private static String escapeHtml(final String text) {
- if (text == null) return "";
- return text.replace("&", "&").replace("<", "<")
- .replace(">", ">").replace("\"", """);
+ emitNeutralTabGroup(parent, startIndex, lastIndex, tabs);
+ return startIndex;
}
/**
@@ -379,7 +362,7 @@ public class GremlinTreeprocessor extends Treeprocessor {
*/
private int processStandaloneTabGroup(final StructuralNode parent, final int startIndex) {
final List<StructuralNode> blocks = parent.getBlocks();
- final List<TabbedHtmlBuilder.Tab> tabs = new ArrayList<>();
+ final List<NeutralTab> tabs = new ArrayList<>();
int lastIndex = startIndex;
for (int j = startIndex; j < blocks.size(); j++) {
@@ -387,19 +370,44 @@ public class GremlinTreeprocessor extends Treeprocessor {
if (isStandaloneTabBlock(block) || isManualTabBlock(block)) {
final Block sourceBlock = (Block) block;
final String lang = getSourceLanguage(sourceBlock);
- tabs.add(TabbedHtmlBuilder.codeTabHighlighted(lang,
- highlightAsSource(parent, lang, sourceBlock.getSource())));
+ tabs.add(NeutralTab.source(lang, sourceBlock.getSource()));
lastIndex = j;
} else {
break;
}
}
- final String html = tabBuilder.build(tabs);
- replaceWithPassBlock(parent, startIndex, lastIndex, html);
+ emitNeutralTabGroup(parent, startIndex, lastIndex, tabs);
return startIndex;
}
+ /**
+ * Renders a neutral tab group and replaces the source blocks with the result. The neutral tabs
+ * are round-tripped through {@link NeutralTabCodec} (serialize then parse) before rendering, so
+ * the JSON payload is the single source of truth for the rendered output. This exercises the
+ * codec on every real block and is the seam where a future execution pass will persist the JSON
+ * into a neutral custom block for a separate render pass to consume.
+ */
+ private void emitNeutralTabGroup(final StructuralNode parent, final int startIndex,
+ final int endIndex, final List<NeutralTab> tabs) {
+ final String json = NeutralTabCodec.serialize(tabs);
+ final List<NeutralTab> resolvedTabs = NeutralTabCodec.parse(json);
+ final String rendered = markdownMode
+ ? markdownTabRenderer.render(resolvedTabs)
+ : htmlTabRenderer.render(parent, resolvedTabs);
+ replaceWithPassBlock(parent, startIndex, endIndex, rendered);
+ }
+
+ /**
+ * Whether the active backend is the TinkerPop Markdown backend. Read from the {@code backend}
+ * attribute, which is available at treeprocessor time (verified) and lets one execution feed
+ * both the HTML and Markdown render passes.
+ */
+ private static boolean isMarkdownBackend(final Document document) {
+ final Object backend = document.getAttribute("backend");
+ return backend != null && backend.toString().startsWith("tpmarkdown");
+ }
+
/**
* Checks if a block is a [source,<lang>] listing block with a supported language (not tab-annotated).
*/
@@ -441,16 +449,21 @@ public class GremlinTreeprocessor extends Treeprocessor {
}
/**
- * Replaces blocks from startIndex to endIndex (inclusive) with a pass block containing raw HTML.
+ * Replaces blocks from startIndex to endIndex (inclusive) with a pass block containing the
+ * pre-rendered content (raw HTML for the HTML backend, raw Markdown for the Markdown backend).
+ * The block is created with a {@code :raw} content model so its body is emitted verbatim by the
+ * active converter without further AsciiDoc substitution.
*/
private void replaceWithPassBlock(final StructuralNode parent, final int startIndex, final int endIndex,
- final String html) {
+ final String content) {
final List<StructuralNode> blocks = parent.getBlocks();
for (int j = endIndex; j >= startIndex; j--) {
blocks.remove(j);
}
final Map<String, Object> attrs = new HashMap<>();
- final Block passBlock = createBlock(parent, "pass", html, attrs);
+ final Map<Object, Object> options = new HashMap<>();
+ options.put("content_model", ":raw");
+ final Block passBlock = createBlock(parent, "pass", content, attrs, options);
blocks.add(startIndex, passBlock);
}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/HtmlTabRenderer.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/HtmlTabRenderer.java
new file mode 100644
index 0000000000..3f9f667d69
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/HtmlTabRenderer.java
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.ast.StructuralNode;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Renders a list of format-neutral {@link NeutralTab}s into the CSS-only tabbed HTML widget.
+ * <p>
+ * This is the HTML backend for the neutral tab model: it highlights each tab's plain-text content
+ * with {@link GremlinHighlighter} (CodeRay + conum injection) and hands the highlighted panels to
+ * {@link TabbedHtmlBuilder}, which builds the radio-tab structure. The output is byte-for-byte the
+ * same HTML the treeprocessor produced when it highlighted inline, so existing rendered docs are
+ * unaffected. A parallel Markdown renderer will consume the same {@link NeutralTab}s later.
+ */
+class HtmlTabRenderer {
+
+ private final GremlinHighlighter highlighter;
+ private final TabbedHtmlBuilder tabBuilder;
+
+ HtmlTabRenderer(final GremlinHighlighter highlighter, final TabbedHtmlBuilder tabBuilder) {
+ this.highlighter = highlighter;
+ this.tabBuilder = tabBuilder;
+ }
+
+ /**
+ * Highlights and renders the given neutral tabs into a complete tab-group HTML string.
+ *
+ * @param parent the AST node used to locate the JRuby runtime for highlighting
+ * @param tabs the neutral tabs to render
+ * @return the tab-group HTML
+ */
+ String render(final StructuralNode parent, final List<NeutralTab> tabs) {
+ final List<TabbedHtmlBuilder.Tab> htmlTabs = new ArrayList<>(tabs.size());
+ for (final NeutralTab tab : tabs) {
+ // The existing pipeline highlighted every tab (console and source) as groovy-flavored
+ // CodeRay regardless of the tab's declared language, so reproduce that exactly by
+ // passing the tab language through to the highlighter (which uses groovy internally).
+ final String highlighted = highlighter.highlight(parent, tab.getLanguage(), tab.getContent());
+ if (tab.getKind() == NeutralTab.Kind.CONSOLE) {
+ htmlTabs.add(TabbedHtmlBuilder.consoleTabHighlighted(tab.getLanguage(), highlighted));
+ } else {
+ htmlTabs.add(TabbedHtmlBuilder.codeTabHighlighted(tab.getLanguage(), highlighted));
+ }
+ }
+ return tabBuilder.build(htmlTabs);
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/LlmsTxtGenerator.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/LlmsTxtGenerator.java
new file mode 100644
index 0000000000..ea7a5dd138
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/LlmsTxtGenerator.java
@@ -0,0 +1,288 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Comparator;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.logging.Logger;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+/**
+ * Generates an {@code llms.txt} discovery index (per agentdocsspec.com / llmstxt.org) over the split
+ * Markdown mirror. The index is a single file with an H1 project name, a summary blockquote, and one
+ * H2 section per book listing every page as {@code [title](path): description}. It links every page
+ * so coverage is 100% (the spec wants >=95%), and is kept well under the 50,000-character cap.
+ */
+class LlmsTxtGenerator {
+
+ private static final Logger LOG = Logger.getLogger(LlmsTxtGenerator.class.getName());
+
+ /** Spec size cap for llms.txt: pass under 50,000 characters. */
+ static final int SIZE_CAP = 50_000;
+
+ private static final Pattern HEADING = Pattern.compile("^#{1,6} +(.*)$");
+ private static final Pattern ANCHOR = Pattern.compile("^<a id=\"[^\"]+\"></a>$");
+ private static final Pattern LLMS_POINTER_LINE = Pattern.compile("^> For the complete documentation index.*$");
+ // Author-curated summary carried as a hidden comment by MarkdownConverter: <!-- llms-summary: ... -->
+ private static final Pattern LLMS_SUMMARY_COMMENT = Pattern.compile("^<!-- llms-summary: (.*) -->$");
+ // Strip inline Markdown/HTML noise from a description snippet.
+ private static final Pattern INLINE_LINK = Pattern.compile("\\[([^\\]]+)\\]\\([^)]*\\)");
+ private static final Pattern INLINE_CODE = Pattern.compile("`([^`]*)`");
+ private static final Pattern HTML_TAG = Pattern.compile("<[^>]+>");
+
+ /** One page's index metadata. */
+ static final class Entry {
+ final String relativePath; // e.g. "dev/io/graphson.md"
+ final String title;
+ final String description;
+
+ Entry(final String relativePath, final String title, final String description) {
+ this.relativePath = relativePath;
+ this.title = title;
+ this.description = description;
+ }
+ }
+
+ private final String projectName;
+ private final String summary;
+ private final String linkPrefix;
+
+ LlmsTxtGenerator(final String projectName, final String summary) {
+ this(projectName, summary, "");
+ }
+
+ /**
+ * @param linkPrefix prepended to every page path in the emitted links. Empty for an index that
+ * sits in the same directory as the pages (local build); set to e.g.
+ * {@code docs/3.7.7/} when the index is written to the site root so links
+ * resolve to the versioned docs tree.
+ */
+ LlmsTxtGenerator(final String projectName, final String summary, final String linkPrefix) {
+ this.projectName = projectName;
+ this.summary = summary;
+ this.linkPrefix = linkPrefix == null ? "" : linkPrefix;
+ }
+
+ /**
+ * Renders the llms.txt body from entries grouped by book. Books are the top path segment (e.g.
+ * {@code dev/io} -> section "dev/io"); within a section, the book landing page ({@code index.md})
+ * is listed first, then the rest in path order.
+ */
+ String generate(final List<Entry> entries) {
+ final StringBuilder sb = new StringBuilder();
+ sb.append("# ").append(projectName).append("\n\n");
+ sb.append("> ").append(summary).append("\n\n");
+
+ // Group by book (the directory portion of the relative path).
+ final Map<String, List<Entry>> byBook = new LinkedHashMap<>();
+ for (final Entry e : entries) {
+ final String book = bookOf(e.relativePath);
+ byBook.computeIfAbsent(book, k -> new ArrayList<>()).add(e);
+ }
+
+ for (final Map.Entry<String, List<Entry>> section : byBook.entrySet()) {
+ sb.append("## ").append(section.getKey()).append("\n\n");
+ final List<Entry> pages = section.getValue();
+ pages.sort(landingFirst());
+ for (final Entry e : pages) {
+ sb.append("- [").append(e.title).append("](").append(linkPrefix).append(e.relativePath).append(')');
+ if (e.description != null && !e.description.isEmpty()) {
+ sb.append(": ").append(e.description);
+ }
+ sb.append('\n');
+ }
+ sb.append('\n');
+ }
+ return sb.toString();
+ }
+
+ private static Comparator<Entry> landingFirst() {
+ return Comparator.comparing((Entry e) -> e.relativePath.endsWith("/index.md")
+ || e.relativePath.equals("index.md") ? 0 : 1)
+ .thenComparing(e -> e.relativePath);
+ }
+
+ private static String bookOf(final String relativePath) {
+ final int slash = relativePath.lastIndexOf('/');
+ return slash < 0 ? "." : relativePath.substring(0, slash);
+ }
+
+ // ---- filesystem driver -------------------------------------------------
+
+ /**
+ * Scans {@code markdownRoot} for {@code .md} pages, builds an entry per page (title from its
+ * first heading, description from its first prose paragraph), writes {@code llms.txt} at the
+ * root, and returns the number of pages indexed.
+ */
+ int generateInto(final Path markdownRoot) throws IOException {
+ return generateInto(markdownRoot, markdownRoot.resolve("llms.txt"));
+ }
+
+ /**
+ * Scans {@code markdownRoot} and writes the index to {@code outFile} (which may be outside the
+ * markdown tree, e.g. the site root when publishing with a versioned {@code linkPrefix}).
+ */
+ int generateInto(final Path markdownRoot, final Path outFile) throws IOException {
+ final List<Entry> entries = new ArrayList<>();
+ try (Stream<Path> walk = Files.walk(markdownRoot)) {
+ final List<Path> mdFiles = walk
+ .filter(Files::isRegularFile)
+ .filter(p -> p.getFileName().toString().endsWith(".md"))
+ .sorted()
+ .collect(Collectors.toList());
+ for (final Path md : mdFiles) {
+ final String rel = markdownRoot.relativize(md).toString().replace('\\', '/');
+ final String content = new String(Files.readAllBytes(md), StandardCharsets.UTF_8);
+ entries.add(new Entry(rel, extractTitle(content, rel), extractDescription(content)));
+ }
+ }
+ final String body = generate(entries);
+ Files.write(outFile, body.getBytes(StandardCharsets.UTF_8));
+ if (body.length() > SIZE_CAP) {
+ LOG.warning("llms.txt is " + body.length() + " chars, over the " + SIZE_CAP
+ + "-char cap; consider a two-level (per-book) index");
+ }
+ LOG.info("Wrote " + outFile + " indexing " + entries.size() + " pages (" + body.length() + " chars)");
+ return entries.size();
+ }
+
+ /** First heading text, or the file name (sans .md) if the page has no heading. */
+ static String extractTitle(final String content, final String relativePath) {
+ for (final String line : content.split("\n", -1)) {
+ final Matcher m = HEADING.matcher(line);
+ if (m.matches()) return m.group(1).trim();
+ }
+ final String name = relativePath.substring(relativePath.lastIndexOf('/') + 1);
+ return name.endsWith(".md") ? name.substring(0, name.length() - 3) : name;
+ }
+
+ /**
+ * The page description for the index. Prefers an author-curated {@code <!-- llms-summary: ... -->}
+ * comment (emitted by {@link MarkdownConverter} from a section's {@code [llms-summary="..."]}
+ * attribute) when the page has one; otherwise falls back to the first prose paragraph: the first
+ * non-blank line after the first heading that is not itself structural (anchor, heading, pointer,
+ * fence, image, table). Inline links/code/HTML are flattened and the result trimmed to one
+ * sentence.
+ */
+ static String extractDescription(final String content) {
+ final String curated = extractCuratedSummary(content);
+ if (!curated.isEmpty()) return curated;
+
+ final String[] lines = content.split("\n", -1);
+ boolean seenHeading = false;
+ boolean inFence = false;
+ for (final String raw : lines) {
+ final String line = raw.trim();
+ if (!seenHeading) {
+ if (HEADING.matcher(line).matches()) seenHeading = true;
+ continue;
+ }
+ // Skip the entire fenced code block, not just its opening delimiter.
+ if (line.startsWith("```")) {
+ inFence = !inFence;
+ continue;
+ }
+ if (inFence) continue;
+ if (line.isEmpty()) continue;
+ if (ANCHOR.matcher(line).matches() || HEADING.matcher(line).matches()
+ || LLMS_POINTER_LINE.matcher(line).matches()
+ || line.startsWith("").matcher(s);
+ if (m.find() && m.start() + 1 <= cap) {
+ return s.substring(0, m.start() + 1);
+ }
+ return s.length() <= cap ? s : s.substring(0, cap).trim() + "…";
+ }
+
+ /**
+ * CLI entry point:
+ * {@code LlmsTxtGenerator [--prefix P] [--out FILE] <markdown-root> [projectName] [summary]}.
+ * Scans the root for {@code .md} pages and writes the index to {@code --out} (default
+ * {@code <markdown-root>/llms.txt}); {@code --prefix} is prepended to every page link (e.g.
+ * {@code docs/3.7.7/} for a site-root index).
+ */
+ public static void main(final String[] args) throws IOException {
+ String prefix = "";
+ String out = null;
+ final List<String> pos = new ArrayList<>();
+ for (int i = 0; i < args.length; i++) {
+ if ("--prefix".equals(args[i]) && i + 1 < args.length) {
+ prefix = args[++i];
+ } else if ("--out".equals(args[i]) && i + 1 < args.length) {
+ out = args[++i];
+ } else {
+ pos.add(args[i]);
+ }
+ }
+ if (pos.isEmpty()) {
+ System.err.println("usage: LlmsTxtGenerator [--prefix P] [--out FILE] <markdown-root> [projectName] [summary]");
+ System.exit(2);
+ }
+ final Path root = Path.of(pos.get(0));
+ final String project = pos.size() > 1 ? pos.get(1) : "Apache TinkerPop Documentation";
+ final String summary = pos.size() > 2 ? pos.get(2)
+ : "Documentation for Apache TinkerPop, a graph computing framework and the Gremlin graph traversal language.";
+ final Path outFile = out != null ? Path.of(out) : root.resolve("llms.txt");
+ new LlmsTxtGenerator(project, summary, prefix).generateInto(root, outFile);
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownConverter.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownConverter.java
new file mode 100644
index 0000000000..3d9dae7fb3
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownConverter.java
@@ -0,0 +1,451 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.ast.Block;
+import org.asciidoctor.ast.ContentNode;
+import org.asciidoctor.ast.Document;
+import org.asciidoctor.ast.List;
+import org.asciidoctor.ast.ListItem;
+import org.asciidoctor.ast.Section;
+import org.asciidoctor.ast.StructuralNode;
+import org.asciidoctor.ast.Table;
+import org.asciidoctor.ast.Row;
+import org.asciidoctor.ast.Cell;
+import org.asciidoctor.converter.ConverterFor;
+import org.asciidoctor.converter.StringConverter;
+
+import java.util.Map;
+
+/**
+ * A GitHub-Flavored-Markdown backend for the TinkerPop docs. Registered for the {@code tpmarkdown}
+ * backend, it walks the resolved AST and emits Markdown, so the same executed document renders to
+ * both HTML (built-in backend) and Markdown from a single Gremlin execution (see
+ * {@link GremlinExecutionCache}).
+ * <p>
+ * Gremlin tab groups arrive as {@code pass} blocks whose content the treeprocessor has already
+ * rendered to Markdown (via {@link MarkdownTabRenderer}); those pass through verbatim. Prose,
+ * headings, lists, admonitions, and simple tables are converted here. Complex tables that GFM pipe
+ * syntax cannot represent fall back to a fenced text rendering.
+ */
+@ConverterFor("tpmarkdown")
+public class MarkdownConverter extends StringConverter {
+
+ /**
+ * Section attribute holding an author-curated one-line summary for the llms.txt index, e.g.
+ * {@code [llms-summary="..."]} on a section header. Emitted only as a hidden Markdown comment;
+ * never rendered into the page body of either backend.
+ */
+ static final String LLMS_SUMMARY_ATTR = "llms-summary";
+
+ public MarkdownConverter(final String backend, final Map<String, Object> opts) {
+ super(backend, opts);
+ }
+
+ @Override
+ public String convert(final ContentNode node, final String transform, final Map<Object, Object> opts) {
+ if (node == null) return "";
+ final String name = transform != null ? transform : node.getNodeName();
+
+ // Debug hook: print every (transform,nodeName) the converter is asked to render.
+ if (System.getProperty("tinkeradoc.md.debug") != null) {
+ System.out.println("[MD-CONV] transform=" + transform + " nodeName=" + node.getNodeName());
+ }
+
+ switch (name) {
+ case "document":
+ case "embedded":
+ return convertDocument((Document) node);
+ case "section":
+ return convertSection((Section) node);
+ case "paragraph":
+ return blockContent((StructuralNode) node) + "\n";
+ case "ulist":
+ return convertList((List) node, false);
+ case "olist":
+ return convertList((List) node, true);
+ case "dlist":
+ return convertDescriptionList((org.asciidoctor.ast.DescriptionList) node);
+ case "colist":
+ // Callout list: the tab renderers already inline callouts as // (n); a stray colist
+ // elsewhere renders as a simple numbered list of its item texts.
+ return convertList((List) node, true);
+ case "listing":
+ case "literal":
+ return convertListing((Block) node);
+ case "pass":
+ // Treeprocessor-produced Markdown (e.g. a gremlin tab group) — emit verbatim.
+ return ((Block) node).getContent() + "\n";
+ case "admonition":
+ return convertAdmonition((Block) node);
+ case "table":
+ return convertTable((Table) node);
+ case "image":
+ return convertImage((StructuralNode) node);
+ case "inline_quoted":
+ return convertInlineQuoted((org.asciidoctor.ast.PhraseNode) node);
+ case "inline_anchor":
+ return convertInlineAnchor((org.asciidoctor.ast.PhraseNode) node);
+ case "inline_break":
+ return ((org.asciidoctor.ast.PhraseNode) node).getText() + "\n";
+ case "inline_callout":
+ // Callouts inside gremlin tab groups are handled by the tab renderers; a stray
+ // inline callout elsewhere renders as its number in parentheses.
+ return "(" + ((org.asciidoctor.ast.PhraseNode) node).getText() + ")";
+ case "inline_image":
+ return convertInlineImage((org.asciidoctor.ast.PhraseNode) node);
+ case "inline_footnote":
+ case "inline_indexterm":
+ return inlineText(node);
+ default:
+ return convertFallback(node);
+ }
+ }
+
+ /** Renders an {@code inline_quoted} phrase (bold/italic/monospace/etc.) as Markdown. */
+ private String convertInlineQuoted(final org.asciidoctor.ast.PhraseNode node) {
+ final String text = node.getText() == null ? "" : node.getText();
+ final String type = node.getType();
+ if (type == null) return text;
+ switch (type) {
+ case "strong": return "**" + text + "**";
+ case "emphasis": return "_" + text + "_";
+ case "monospaced": return "`" + text + "`";
+ case "mark": return text;
+ case "superscript": return "<sup>" + text + "</sup>";
+ case "subscript": return "<sub>" + text + "</sub>";
+ default: return text;
+ }
+ }
+
+ /** Renders an {@code inline_anchor} (link/xref) as a Markdown link, or bare text if no target. */
+ private String convertInlineAnchor(final org.asciidoctor.ast.PhraseNode node) {
+ final String text = node.getText();
+ final Object target = node.getTarget();
+ if (target == null) return text == null ? "" : text;
+ final String label = (text == null || text.isEmpty()) ? target.toString() : text;
+ return "[" + label + "](" + target + ")";
+ }
+
+ private String inlineText(final ContentNode node) {
+ if (node instanceof org.asciidoctor.ast.PhraseNode) {
+ final String t = ((org.asciidoctor.ast.PhraseNode) node).getText();
+ return t == null ? "" : t;
+ }
+ return "";
+ }
+
+ private String convertDocument(final Document doc) {
+ final String content = doc.getContent() == null ? "" : doc.getContent().toString();
+ final String title = doc.getDoctitle();
+ final String assembled;
+ // In a book, the document title is realized as a level-0 section inside the content, so it
+ // already appears as an H1 there (matching the HTML backend's single <h1 class="sect0">).
+ // In an article, the title is not part of the content, so emit it as the leading H1.
+ if (title != null && !title.isEmpty() && !isBook(doc)) {
+ final StringBuilder head = new StringBuilder();
+ appendAnchor(head, doc.getId());
+ head.append("# ").append(title).append("\n\n").append(content);
+ assembled = head.toString();
+ } else {
+ assembled = content;
+ }
+ // Substitute the x.y.z version placeholder here rather than in the shared postprocessor:
+ // AsciidoctorJ discards a postprocessor's return value for a custom StringConverter backend,
+ // so the Markdown file would otherwise keep the literal x.y.z. The HTML backend still relies
+ // on the postprocessor for the same substitution.
+ return substituteVersion(doc, assembled);
+ }
+
+ /** Replaces {@code x.y.z} with the resolved TinkerPop version, if available. */
+ private static String substituteVersion(final Document doc, final String text) {
+ final Object version = firstNonNull(doc.getAttribute("tinkerpop-version"), doc.getAttribute("revnumber"));
+ return version == null ? text : text.replace("x.y.z", version.toString());
+ }
+
+ private static Object firstNonNull(final Object a, final Object b) {
+ return a != null ? a : b;
+ }
+
+ private static boolean isBook(final Document doc) {
+ final Object doctype = doc.getAttribute("doctype");
+ return "book".equals(String.valueOf(doctype));
+ }
+
+ private String convertSection(final Section section) {
+ final StringBuilder sb = new StringBuilder();
+ final int level = section.getLevel();
+ final StringBuilder hashes = new StringBuilder();
+ for (int i = 0; i <= level; i++) hashes.append('#');
+ // Emit an explicit HTML anchor carrying the section's AsciiDoc id so xrefs of the form
+ // [label](#the-id) resolve exactly (GFM renderers honor raw HTML anchors). This preserves
+ // the ids verbatim, so no xref fragment remapping is needed, and it survives the WS2 page
+ // split (each page keeps its headings' anchors).
+ appendAnchor(sb, section.getId());
+ sb.append(hashes).append(' ').append(section.getTitle()).append("\n\n");
+ appendLlmsSummary(sb, section.getAttribute(LLMS_SUMMARY_ATTR));
+ sb.append(section.getContent());
+ return sb.toString();
+ }
+
+ /** Emits {@code <a id="..."></a>} on its own line when the node has a non-empty id. */
+ private static void appendAnchor(final StringBuilder sb, final String id) {
+ if (id != null && !id.isEmpty()) {
+ sb.append("<a id=\"").append(id).append("\"></a>\n");
+ }
+ }
+
+ /**
+ * Emits a curated section summary as an HTML comment ({@code <!-- llms-summary: ... -->}) that
+ * {@link LlmsTxtGenerator} reads for the page description. It is invisible in rendered Markdown
+ * (and absent from HTML — the source attribute never reaches the HTML backend as content), so it
+ * carries author-written index copy without polluting the page body. Authored in AsciiDoc as
+ * {@code [llms-summary="..."]} on a section header.
+ */
+ private static void appendLlmsSummary(final StringBuilder sb, final Object summary) {
+ if (summary == null) return;
+ final String s = summary.toString().trim();
+ if (s.isEmpty()) return;
+ // Neutralize any comment-closer in author text so the comment can't be broken out of.
+ sb.append("<!-- llms-summary: ").append(s.replace("-->", "-->")).append(" -->\n\n");
+ }
+
+ private String convertList(final List list, final boolean ordered) {
+ final StringBuilder sb = new StringBuilder();
+ int index = 1;
+ for (final StructuralNode item : list.getItems()) {
+ final ListItem li = (ListItem) item;
+ final String marker = ordered ? (index++ + ".") : "-";
+ final String text = li.hasText() && li.getText() != null ? li.getText() : "";
+ sb.append(marker).append(' ').append(text).append('\n');
+ // Attached blocks (continuation paragraphs, nested lists) are converted individually and
+ // indented. Converting each child block once — rather than calling li.getContent(), which
+ // re-renders the item's whole subtree — avoids the quadratic/exponential blow-up that a
+ // repeated getContent() causes on deep or dense structures.
+ for (final StructuralNode child : li.getBlocks()) {
+ sb.append(indent(child.convert()));
+ }
+ }
+ sb.append('\n');
+ return sb.toString();
+ }
+
+ /**
+ * Renders a description (definition) list as {@code **term**} followed by an indented
+ * description. Each term and description is converted once via its own node, never through a
+ * blanket {@code getContent()} on the list (which re-renders and blows up on dense lists).
+ */
+ private String convertDescriptionList(final org.asciidoctor.ast.DescriptionList list) {
+ final StringBuilder sb = new StringBuilder();
+ for (final org.asciidoctor.ast.DescriptionListEntry entry : list.getItems()) {
+ for (final ListItem term : entry.getTerms()) {
+ final String t = term.hasText() && term.getText() != null ? term.getText() : "";
+ sb.append("**").append(t).append("**\n");
+ }
+ final ListItem desc = entry.getDescription();
+ if (desc != null) {
+ if (desc.hasText() && desc.getText() != null) {
+ sb.append(indent(desc.getText())).append('\n');
+ }
+ for (final StructuralNode child : desc.getBlocks()) {
+ sb.append(indent(child.convert()));
+ }
+ }
+ sb.append('\n');
+ }
+ return sb.toString();
+ }
+
+ private String convertListing(final Block block) {
+ final String lang = languageOf(block);
+ final StringBuilder sb = new StringBuilder();
+ sb.append("```").append(lang).append('\n');
+ sb.append(block.getSource());
+ sb.append("\n```\n");
+ return sb.toString();
+ }
+
+ private String convertAdmonition(final Block block) {
+ final String style = block.getStyle();
+ final String label = style != null ? style.toUpperCase() : "NOTE";
+ final Object contentObj = block.getContent();
+ final String body = contentObj == null ? "" : contentObj.toString();
+ final StringBuilder sb = new StringBuilder();
+ for (final String line : body.split("\n", -1)) {
+ sb.append("> **").append(label).append(":** ").append(line).append('\n');
+ }
+ return sb.toString();
+ }
+
+ private String convertTable(final Table table) {
+ // Only simple tables map cleanly to GFM pipe syntax. Fall back to a fenced text dump when
+ // the table has multiple columns spanning or nested block content that pipes can't hold.
+ try {
+ final StringBuilder sb = new StringBuilder();
+ final java.util.List<Row> header = table.getHeader();
+ final java.util.List<Row> body = table.getBody();
+ final int cols = columnCount(table);
+ if (cols == 0) return "";
+
+ if (!header.isEmpty()) {
+ appendRow(sb, header.get(0), cols);
+ appendSeparator(sb, cols);
+ } else {
+ // GFM requires a header row; synthesize a blank one so the table is valid.
+ appendBlankHeader(sb, cols);
+ appendSeparator(sb, cols);
+ }
+ for (final Row row : body) {
+ appendRow(sb, row, cols);
+ }
+ sb.append('\n');
+ return sb.toString();
+ } catch (final RuntimeException e) {
+ return convertTableFallback(table);
+ }
+ }
+
+ private String convertTableFallback(final Table table) {
+ final StringBuilder sb = new StringBuilder("```\n");
+ for (final Row row : table.getBody()) {
+ boolean first = true;
+ for (final Cell cell : row.getCells()) {
+ if (!first) sb.append(" | ");
+ sb.append(cellText(cell));
+ first = false;
+ }
+ sb.append('\n');
+ }
+ sb.append("```\n");
+ return sb.toString();
+ }
+
+ private String convertImage(final StructuralNode node) {
+ final Object alt = node.getAttribute("alt");
+ return ") + ")\n";
+ }
+
+ /** Renders an inline image ({@code image:foo.png[alt]}) as a Markdown inline image. */
+ private String convertInlineImage(final org.asciidoctor.ast.PhraseNode node) {
+ final Object alt = node.getAttribute("alt");
+ // Inline images carry their file in getTarget(), not the "target" attribute (unlike block
+ // images), so prefer getTarget() and fall back to the attribute.
+ Object target = node.getTarget();
+ if (target == null) target = node.getAttribute("target");
+ return " + ")";
+ }
+
+ /**
+ * Resolves an image target against the document's {@code imagesdir}, matching the HTML backend
+ * (which prepends {@code imagesdir} to every image src). Split pages of a book all live in the
+ * book's directory, so the per-book {@code imagesdir} (e.g. {@code ../images}) is the correct
+ * relative prefix for every page. Absolute or URL targets are left untouched.
+ */
+ private static String imagePath(final ContentNode node, final Object target) {
+ if (target == null) return "";
+ final String t = target.toString();
+ if (t.isEmpty() || t.startsWith("/") || t.matches("^[a-zA-Z][a-zA-Z0-9+.-]*://.*")) {
+ return t;
+ }
+ final Object dir = node.getDocument() == null ? null : node.getDocument().getAttribute("imagesdir");
+ if (dir == null || dir.toString().isEmpty()) return t;
+ final String d = dir.toString();
+ return d.endsWith("/") ? d + t : d + "/" + t;
+ }
+
+ /**
+ * Fallback for node types without a dedicated case. It converts the node's own text (if any) and
+ * then each child block individually. It deliberately does <em>not</em> call {@code getContent()}
+ * on the node: for container nodes {@code getContent()} re-renders the whole subtree, and when a
+ * parent then also renders that content the work multiplies per nesting level, which produced an
+ * out-of-memory blow-up on dense books. Converting each child exactly once is linear.
+ */
+ private String convertFallback(final ContentNode node) {
+ if (!(node instanceof StructuralNode)) {
+ return "";
+ }
+ final StructuralNode sn = (StructuralNode) node;
+ final StringBuilder sb = new StringBuilder();
+ if (sn instanceof Block) {
+ final Object src = ((Block) sn).getSource();
+ if (src != null && !src.toString().isEmpty()) {
+ sb.append(src).append('\n');
+ }
+ }
+ for (final StructuralNode child : sn.getBlocks()) {
+ sb.append(child.convert());
+ }
+ return sb.toString();
+ }
+
+ private static String blockContent(final StructuralNode node) {
+ final Object content = node.getContent();
+ return content == null ? "" : content.toString();
+ }
+
+ private static String languageOf(final Block block) {
+ final Object lang = block.getAttribute("language");
+ return lang == null ? "" : lang.toString();
+ }
+
+ private int columnCount(final Table table) {
+ if (!table.getHeader().isEmpty()) return table.getHeader().get(0).getCells().size();
+ if (!table.getBody().isEmpty()) return table.getBody().get(0).getCells().size();
+ return 0;
+ }
+
+ private void appendRow(final StringBuilder sb, final Row row, final int cols) {
+ sb.append('|');
+ final java.util.List<Cell> cells = row.getCells();
+ for (int i = 0; i < cols; i++) {
+ final String text = i < cells.size() ? cellText(cells.get(i)) : "";
+ sb.append(' ').append(text.replace("\n", " ")).append(" |");
+ }
+ sb.append('\n');
+ }
+
+ private void appendBlankHeader(final StringBuilder sb, final int cols) {
+ sb.append('|');
+ for (int i = 0; i < cols; i++) sb.append(" |");
+ sb.append('\n');
+ }
+
+ private void appendSeparator(final StringBuilder sb, final int cols) {
+ sb.append('|');
+ for (int i = 0; i < cols; i++) sb.append(" --- |");
+ sb.append('\n');
+ }
+
+ private String cellText(final Cell cell) {
+ final String t = cell.getText();
+ return t == null ? "" : t;
+ }
+
+ private static String indent(final String content) {
+ final StringBuilder sb = new StringBuilder();
+ for (final String line : content.split("\n", -1)) {
+ if (line.isEmpty()) {
+ sb.append('\n');
+ } else {
+ sb.append(" ").append(line).append('\n');
+ }
+ }
+ return sb.toString();
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownSplitter.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownSplitter.java
new file mode 100644
index 0000000000..f00f3b94da
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownSplitter.java
@@ -0,0 +1,461 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.logging.Logger;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * Splits a single rendered Markdown book (as produced by {@link MarkdownConverter}) into
+ * agent-sized pages, each a named, heading-bounded section under the size budget, and rewrites
+ * intra-document anchor links to point across pages.
+ * <p>
+ * <b>Splitting rule (size-driven, heading-aligned).</b> Emit each page at the coarsest heading
+ * level that keeps it under the byte budget; descend a level (h2 → h3 → h4) only for subtrees that
+ * exceed the budget. Every page is a whole heading-bounded section, so page URLs stay semantic and
+ * stable. This is one uniform rule with no per-file special cases: small books stay as whole-h2
+ * pages, large books descend where needed.
+ * <p>
+ * The splitter operates purely on the rendered Markdown text. It relies on the
+ * {@code <a id="..."></a>} anchor {@link MarkdownConverter} emits immediately before each heading:
+ * the anchor id becomes both the page's file name (for pages that start a file) and the link
+ * fragment. Links of the form {@code [label](#anchor)} are rewritten to {@code [label](file.md#anchor)}
+ * when the target anchor lives on a different page.
+ */
+class MarkdownSplitter {
+
+ /** Default per-page budget in bytes (~50 KB), per the agent-docs spec's page-size check. */
+ static final int DEFAULT_BUDGET = 50_000;
+
+ /** Safety margin (bytes) held back from the packing budget to absorb estimate/render drift. */
+ private static final int PACK_SAFETY_MARGIN = 512;
+
+ private static final Pattern ANCHOR = Pattern.compile("^<a id=\"([^\"]+)\"></a>$");
+ private static final Pattern HEADING = Pattern.compile("^(#{1,6}) +(.*)$");
+ // Intra-document links: [label](#anchor). Capture label and anchor separately.
+ private static final Pattern INTRA_LINK = Pattern.compile("\\]\\(#([^)]+)\\)");
+
+ private static final Logger LOG = Logger.getLogger(MarkdownSplitter.class.getName());
+
+ private final int budget;
+ private final int packBudget;
+
+ MarkdownSplitter() {
+ this(DEFAULT_BUDGET);
+ }
+
+ MarkdownSplitter(final int budget) {
+ this.budget = budget;
+ // Reserve room in the packing budget for (a) the llms.txt pointer prepended to every page
+ // after packing, and (b) a small safety margin. The per-node byte estimate can drift a few
+ // bytes from the concatenated render (newline/preamble joins), and pages pack right up to
+ // the limit; the margin keeps the final UTF-8 page comfortably under the hard cap.
+ final int pointer = LLMS_POINTER.getBytes(java.nio.charset.StandardCharsets.UTF_8).length;
+ this.packBudget = Math.max(1, budget - pointer - PACK_SAFETY_MARGIN);
+ }
+
+ /**
+ * Splits a rendered book file in place: reads {@code bookFile} (e.g. {@code dev/io/index.md}),
+ * writes the resulting pages into its parent directory, and replaces the original with the
+ * landing page. Pages that still exceed the budget (indivisible single-heading leaves) are
+ * reported so silent truncation never masquerades as full coverage.
+ *
+ * @param bookFile the rendered single-file book to split
+ * @return the file names written into the book's directory
+ */
+ List<String> splitFile(final Path bookFile) throws IOException {
+ final String markdown = new String(Files.readAllBytes(bookFile), StandardCharsets.UTF_8);
+ final Path dir = bookFile.getParent();
+ final String indexName = bookFile.getFileName().toString();
+ final List<Page> pages = split(markdown, indexName);
+
+ final List<String> written = new ArrayList<>();
+ for (final Page page : pages) {
+ final Path out = dir.resolve(page.getFileName());
+ final byte[] bytes = page.getContent().getBytes(StandardCharsets.UTF_8);
+ Files.write(out, bytes);
+ written.add(page.getFileName());
+ if (bytes.length > budget) {
+ LOG.warning("Markdown page " + out + " is " + bytes.length
+ + " bytes, over the " + budget + "-byte budget (indivisible single-heading section)");
+ }
+ }
+ LOG.info("Split " + bookFile + " into " + written.size() + " pages: " + written);
+ return written;
+ }
+
+ /**
+ * CLI entry point: {@code MarkdownSplitter [--budget N] <book.md> [<book.md> ...]}. Each named
+ * rendered book file is split in place into agent-sized pages in its own directory.
+ */
+ public static void main(final String[] args) throws IOException {
+ int budget = DEFAULT_BUDGET;
+ final List<String> files = new ArrayList<>();
+ for (int i = 0; i < args.length; i++) {
+ if ("--budget".equals(args[i]) && i + 1 < args.length) {
+ budget = Integer.parseInt(args[++i]);
+ } else {
+ files.add(args[i]);
+ }
+ }
+ if (files.isEmpty()) {
+ System.err.println("usage: MarkdownSplitter [--budget N] <book.md> [<book.md> ...]");
+ System.exit(2);
+ }
+ final MarkdownSplitter splitter = new MarkdownSplitter(budget);
+ for (final String f : files) {
+ final Path p = Path.of(f);
+ if (Files.isRegularFile(p)) {
+ splitter.splitFile(p);
+ } else {
+ System.err.println("skip (not a file): " + f);
+ }
+ }
+ }
+
+ /** A rendered page: its file name (without directory), and its Markdown body. */
+ static final class Page {
+ final String fileName;
+ final String content;
+
+ Page(final String fileName, final String content) {
+ this.fileName = fileName;
+ this.content = content;
+ }
+
+ String getFileName() {
+ return fileName;
+ }
+
+ String getContent() {
+ return content;
+ }
+ }
+
+ /** A parsed heading unit: its level, anchor id (may be null), and the raw block of lines. */
+ private static final class Node {
+ final int level;
+ final String anchor;
+ final List<String> lines = new ArrayList<>();
+ final List<Node> children = new ArrayList<>();
+ int byteSize; // size of this node's own lines plus descendants, computed lazily
+
+ Node(final int level, final String anchor) {
+ this.level = level;
+ this.anchor = anchor;
+ }
+ }
+
+ /**
+ * Splits {@code markdown} into pages. The first page always carries the file name
+ * {@code indexFileName} (typically {@code index.md}) and holds the document preamble plus the
+ * top-level structure that fits; deeper sections that individually exceed the budget become
+ * their own pages named from their anchor id.
+ *
+ * @param markdown the full rendered Markdown book
+ * @param indexFileName the file name for the landing page (e.g. {@code index.md})
+ * @return the ordered list of pages (always at least one)
+ */
+ List<Page> split(final String markdown, final String indexFileName) {
+ final Node root = parse(markdown);
+ computeSizes(root);
+
+ // Decide which nodes start their own page. The root's own lines (preamble) always live on
+ // the index page. Each top-level child is placed on the index page if it fits; otherwise it
+ // (or its children, recursively) become separate pages.
+ final Map<String, String> anchorToFile = new LinkedHashMap<>();
+ final List<PagePlan> plans = new ArrayList<>();
+ final PagePlan index = new PagePlan(indexFileName);
+ plans.add(index);
+
+ // Record every anchor's home page as we assign, so links can be rewritten afterward.
+ assignRootAnchors(root, index, anchorToFile);
+ final PageCursor cursor = new PageCursor(index);
+ cursor.used = preambleSize(root);
+ planChildren(root, cursor, plans, anchorToFile);
+
+ // Render each planned page, rewrite cross-page links, and prepend the llms.txt pointer.
+ final List<Page> pages = new ArrayList<>();
+ for (final PagePlan plan : plans) {
+ final String body = plan.render();
+ final String rewritten = rewriteLinks(body, plan.fileName, anchorToFile);
+ pages.add(new Page(plan.fileName, LLMS_POINTER + rewritten));
+ }
+ return pages;
+ }
+
+ /**
+ * The agent-facing directive prepended to every page (agentdocsspec.com {@code
+ * llms-txt-directive-md} check): a top-of-page blockquote pointing at the site-root index.
+ */
+ static final String LLMS_POINTER =
+ "> For the complete documentation index, see [llms.txt](/llms.txt)\n\n";
+
+ // ---- parsing -----------------------------------------------------------
+
+ private Node parse(final String markdown) {
+ final String[] lines = markdown.split("\n", -1);
+ final Node root = new Node(0, null);
+ final java.util.Deque<Node> stack = new java.util.ArrayDeque<>();
+ stack.push(root);
+
+ String pendingAnchor = null;
+ boolean inFence = false;
+ for (int i = 0; i < lines.length; i++) {
+ final String line = lines[i];
+ // Track fenced code blocks: a line starting with ``` toggles fence state. Lines inside a
+ // fence are always content, never headings — otherwise shell/YAML/properties comment
+ // lines like "# Spark Configuration" would be mistaken for section headings and split
+ // the page mid-code-block.
+ if (line.startsWith("```")) {
+ inFence = !inFence;
+ if (pendingAnchor != null) {
+ stack.peek().lines.add("<a id=\"" + pendingAnchor + "\"></a>");
+ pendingAnchor = null;
+ }
+ stack.peek().lines.add(line);
+ continue;
+ }
+ final Matcher am = inFence ? null : ANCHOR.matcher(line);
+ if (am != null && am.matches()) {
+ pendingAnchor = am.group(1);
+ // The anchor line belongs to the heading node it precedes; hold it.
+ continue;
+ }
+ final Matcher hm = inFence ? null : HEADING.matcher(line);
+ if (hm != null && hm.matches()) {
+ final int level = hm.group(1).length();
+ final Node node = new Node(level, pendingAnchor);
+ if (pendingAnchor != null) {
+ node.lines.add("<a id=\"" + pendingAnchor + "\"></a>");
+ }
+ node.lines.add(line);
+ pendingAnchor = null;
+ // Pop until the top of stack is a strictly shallower node (its parent).
+ while (stack.peek().level >= level) stack.pop();
+ stack.peek().children.add(node);
+ stack.push(node);
+ } else {
+ // A dangling anchor not followed by a heading: keep it as content.
+ if (pendingAnchor != null) {
+ stack.peek().lines.add("<a id=\"" + pendingAnchor + "\"></a>");
+ pendingAnchor = null;
+ }
+ stack.peek().lines.add(line);
+ }
+ }
+ return root;
+ }
+
+ private int computeSizes(final Node node) {
+ int size = ownSize(node);
+ for (final Node c : node.children) size += computeSizes(c);
+ node.byteSize = size;
+ return size;
+ }
+
+ /**
+ * The size of a node's own lines (heading + leading content) in UTF-8 bytes, excluding children.
+ * Uses UTF-8 byte length (not {@code String.length()}, which counts UTF-16 chars) because pages
+ * are written and size-checked as UTF-8; docs contain multi-byte characters (™, →, é, …), so a
+ * char count would under-estimate and let a page slip over the byte budget.
+ */
+ private static int ownSize(final Node node) {
+ int size = 0;
+ for (final String l : node.lines) {
+ size += l.getBytes(java.nio.charset.StandardCharsets.UTF_8).length + 1;
+ }
+ return size;
+ }
+
+ /** The byte size of the document preamble (root's own lines) that always leads the index page. */
+ private static int preambleSize(final Node root) {
+ return ownSize(root);
+ }
+
+ // ---- page planning -----------------------------------------------------
+
+ private static final class PagePlan {
+ final String fileName;
+ final List<Node> nodes = new ArrayList<>();
+ final List<String> preambleLines = new ArrayList<>();
+
+ PagePlan(final String fileName) {
+ this.fileName = fileName;
+ }
+
+ String render() {
+ final StringBuilder sb = new StringBuilder();
+ for (final String l : preambleLines) sb.append(l).append('\n');
+ for (final Node n : nodes) renderNode(n, sb);
+ return sb.toString();
+ }
+
+ private void renderNode(final Node n, final StringBuilder sb) {
+ for (final String l : n.lines) sb.append(l).append('\n');
+ for (final Node c : n.children) renderNode(c, sb);
+ }
+ }
+
+ /** The root's own (pre-heading) lines and any anchors on them belong to the index page. */
+ private void assignRootAnchors(final Node root, final PagePlan index,
+ final Map<String, String> anchorToFile) {
+ index.preambleLines.addAll(root.lines);
+ // Anchors that appear in the preamble lines resolve to the index page.
+ for (final String line : root.lines) {
+ final Matcher am = ANCHOR.matcher(line);
+ if (am.matches()) anchorToFile.put(am.group(1), index.fileName);
+ }
+ }
+
+ /** Mutable pointer to the page currently being filled, plus its running byte size. */
+ private static final class PageCursor {
+ PagePlan page;
+ int used;
+
+ PageCursor(final PagePlan page) {
+ this.page = page;
+ }
+ }
+
+ /**
+ * Greedily packs each child of {@code parent} onto the current page, keeping pages as full as
+ * possible under the budget while never cutting mid-section:
+ * <ul>
+ * <li>If the child's whole subtree fits in the current page's remaining room, place it there.</li>
+ * <li>Else, if the whole subtree fits an empty page, start a fresh page for it.</li>
+ * <li>Else (the subtree alone exceeds the budget), open a page led by the child's own heading
+ * and descend a level, recursively planning its children.</li>
+ * </ul>
+ * The descend case is what turns an over-budget chapter into per-section pages.
+ */
+ private void planChildren(final Node parent, final PageCursor cursor,
+ final List<PagePlan> plans, final Map<String, String> anchorToFile) {
+ for (final Node child : parent.children) {
+ if (cursor.used + child.byteSize <= packBudget) {
+ placeWhole(child, cursor.page, anchorToFile);
+ cursor.used += child.byteSize;
+ } else if (child.byteSize <= packBudget) {
+ // Fits a page of its own: start a fresh page and move the cursor there.
+ final PagePlan page = newPage(child, plans);
+ placeWhole(child, page, anchorToFile);
+ cursor.page = page;
+ cursor.used = child.byteSize;
+ } else {
+ // Too big for any single page: lead a new page with the child's own heading, then
+ // descend into its children on that same page (they will overflow to more pages).
+ final PagePlan page = newPage(child, plans);
+ page.nodes.add(headOnly(child, page, anchorToFile));
+ final PageCursor childCursor = new PageCursor(page);
+ childCursor.used = ownSize(child);
+ planChildren(child, childCursor, plans, anchorToFile);
+ // Continue the outer loop on the last page the descent produced, so a small sibling
+ // after a big chapter can still share that page's leftover room.
+ cursor.page = childCursor.page;
+ cursor.used = childCursor.used;
+ }
+ }
+ }
+
+ private PagePlan newPage(final Node node, final List<PagePlan> plans) {
+ final PagePlan page = new PagePlan(fileNameFor(node, plans));
+ plans.add(page);
+ return page;
+ }
+
+ /** Places a node (and its whole subtree) onto a page, recording all its anchors' home. */
+ private void placeWhole(final Node node, final PagePlan page, final Map<String, String> anchorToFile) {
+ page.nodes.add(node);
+ recordAnchors(node, page.fileName, anchorToFile);
+ }
+
+ /**
+ * Returns a shallow copy of {@code node} holding only its own lines (heading + leading content),
+ * with its children dropped, so the node's heading leads its own page while its subsections are
+ * planned separately. Records the node's own anchor on the given page.
+ */
+ private Node headOnly(final Node node, final PagePlan page, final Map<String, String> anchorToFile) {
+ final Node head = new Node(node.level, node.anchor);
+ head.lines.addAll(node.lines);
+ if (node.anchor != null) anchorToFile.put(node.anchor, page.fileName);
+ return head;
+ }
+
+ private void recordAnchors(final Node node, final String fileName,
+ final Map<String, String> anchorToFile) {
+ if (node.anchor != null) anchorToFile.put(node.anchor, fileName);
+ for (final Node c : node.children) recordAnchors(c, fileName, anchorToFile);
+ }
+
+ private String fileNameFor(final Node node, final List<PagePlan> plans) {
+ final String base = node.anchor != null && !node.anchor.isEmpty()
+ ? sanitize(node.anchor) : "section";
+ String candidate = base + ".md";
+ int n = 2;
+ while (fileExists(plans, candidate)) {
+ candidate = base + "-" + (n++) + ".md";
+ }
+ return candidate;
+ }
+
+ private boolean fileExists(final List<PagePlan> plans, final String fileName) {
+ for (final PagePlan p : plans) if (p.fileName.equals(fileName)) return true;
+ return false;
+ }
+
+ private static String sanitize(final String anchor) {
+ // Anchor ids are already url-safe (letters, digits, _ and -), but guard against stray chars.
+ return anchor.replaceAll("[^A-Za-z0-9_.-]", "-");
+ }
+
+ // ---- link rewriting ----------------------------------------------------
+
+ /**
+ * Rewrites intra-document links {@code ](#anchor)} on {@code thisFile} so that anchors living on
+ * a different page become {@code ](otherFile.md#anchor)}. Anchors on the same page keep their
+ * bare {@code #anchor} form. Unknown anchors are left unchanged.
+ */
+ private String rewriteLinks(final String body, final String thisFile,
+ final Map<String, String> anchorToFile) {
+ final Matcher m = INTRA_LINK.matcher(body);
+ final StringBuffer sb = new StringBuffer();
+ while (m.find()) {
+ final String anchor = m.group(1);
+ final String home = anchorToFile.get(anchor);
+ final String replacement;
+ if (home == null || home.equals(thisFile)) {
+ replacement = "](#" + anchor + ")";
+ } else {
+ replacement = "](" + home + "#" + anchor + ")";
+ }
+ m.appendReplacement(sb, Matcher.quoteReplacement(replacement));
+ }
+ m.appendTail(sb);
+ return sb.toString();
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownTabRenderer.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownTabRenderer.java
new file mode 100644
index 0000000000..bb1685b956
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/MarkdownTabRenderer.java
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import java.util.List;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * Renders a list of format-neutral {@link NeutralTab}s into Markdown. This is the Markdown backend
+ * for the neutral tab model, parallel to {@link HtmlTabRenderer}.
+ * <p>
+ * HTML uses a CSS radio-tab widget; Markdown has no tabs, so each tab is flattened to a labeled
+ * fenced code block emitted in order. The console tab renders as a {@code text} fence (it is an
+ * interactive transcript with {@code gremlin>} prompts and {@code ==>} results, not compilable
+ * source); source tabs render with their language as the fence info string. AsciiDoc callout
+ * markers ({@code <1>}) in a tab's content are converted to inline {@code // (n)} comments, since
+ * Markdown fenced blocks have no callout facility.
+ */
+class MarkdownTabRenderer {
+
+ private static final Pattern TRAILING_CALLOUTS = Pattern.compile("\\s*((<\\d+>\\s*)+)$");
+ private static final Pattern CALLOUT_NUMBER = Pattern.compile("<(\\d+)>");
+
+ /**
+ * Renders the given neutral tabs as a sequence of labeled fenced code blocks.
+ *
+ * @param tabs the neutral tabs to render
+ * @return the Markdown for the tab group (no trailing newline)
+ */
+ String render(final List<NeutralTab> tabs) {
+ final StringBuilder sb = new StringBuilder();
+ for (int i = 0; i < tabs.size(); i++) {
+ if (i > 0) sb.append("\n\n");
+ final NeutralTab tab = tabs.get(i);
+ // A single Groovy console+source pair (the common case) needs no per-tab heading; extra
+ // language variants are labeled so the reader can tell them apart without tab chrome.
+ sb.append("**").append(tab.getLabel()).append("**\n\n");
+ final String fenceLang = tab.getKind() == NeutralTab.Kind.CONSOLE ? "text" : fenceLang(tab.getLanguage());
+ sb.append("```").append(fenceLang).append("\n");
+ sb.append(convertCallouts(tab.getContent()));
+ sb.append("\n```");
+ }
+ return sb.toString();
+ }
+
+ /**
+ * Maps a tab language to a Markdown fence info string. Unknown/blank languages produce a bare
+ * fence.
+ */
+ private static String fenceLang(final String language) {
+ return language == null ? "" : language.trim();
+ }
+
+ /**
+ * Converts trailing AsciiDoc callout markers on each line ({@code <1>}) into inline
+ * {@code // (n)} comments. Non-trailing angle brackets are left untouched.
+ */
+ static String convertCallouts(final String content) {
+ if (content == null || content.isEmpty()) return "";
+ final String[] lines = content.split("\n", -1);
+ final StringBuilder result = new StringBuilder();
+ for (int i = 0; i < lines.length; i++) {
+ if (i > 0) result.append("\n");
+ result.append(convertCalloutLine(lines[i]));
+ }
+ return result.toString();
+ }
+
+ private static String convertCalloutLine(final String line) {
+ final Matcher m = TRAILING_CALLOUTS.matcher(line);
+ if (!m.find()) return line;
+ final String prefix = stripTrailing(line.substring(0, m.start()));
+ final StringBuilder sb = new StringBuilder(prefix);
+ final Matcher nums = CALLOUT_NUMBER.matcher(m.group(1));
+ while (nums.find()) {
+ sb.append(" // (").append(nums.group(1)).append(')');
+ }
+ return sb.toString();
+ }
+
+ private static String stripTrailing(final String s) {
+ int end = s.length();
+ while (end > 0 && Character.isWhitespace(s.charAt(end - 1))) end--;
+ return s.substring(0, end);
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/NeutralTab.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/NeutralTab.java
new file mode 100644
index 0000000000..2aa66b82f5
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/NeutralTab.java
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+/**
+ * A format-neutral representation of one tab in a Gremlin example: a label, a language, a kind,
+ * and the tab's content as <em>plain text</em> (no highlighting, no HTML). Callout markers are
+ * retained inline in the content as AsciiDoc {@code <n>} markers; each backend renderer decides
+ * how to present them (HTML conums, Markdown comments, etc.).
+ * <p>
+ * This is the seam that lets a single execution of the live Gremlin console feed multiple output
+ * backends: the executor produces {@code NeutralTab}s, and {@link HtmlTabRenderer} (and, later, a
+ * Markdown renderer) turn them into their respective formats.
+ */
+class NeutralTab {
+
+ /** Whether a tab holds an interactive console transcript or a clean code sample. */
+ enum Kind {
+ /** A {@code gremlin>}-prefixed transcript including executed results. */
+ CONSOLE,
+ /** A clean code sample with no prompts or results. */
+ SOURCE
+ }
+
+ private final String label;
+ private final String language;
+ private final Kind kind;
+ private final String content;
+
+ NeutralTab(final String label, final String language, final Kind kind, final String content) {
+ this.label = label;
+ this.language = language;
+ this.kind = kind;
+ this.content = content;
+ }
+
+ /**
+ * Creates a console-transcript tab. Its label matches the existing HTML output, i.e.
+ * {@code console (<lang>)}.
+ */
+ static NeutralTab console(final String language, final String content) {
+ return new NeutralTab("console (" + language + ")", language, Kind.CONSOLE, content);
+ }
+
+ /**
+ * Creates a clean code-sample tab. Its label is the language identifier, matching the existing
+ * HTML output.
+ */
+ static NeutralTab source(final String language, final String content) {
+ return new NeutralTab(language, language, Kind.SOURCE, content);
+ }
+
+ String getLabel() {
+ return label;
+ }
+
+ String getLanguage() {
+ return language;
+ }
+
+ Kind getKind() {
+ return kind;
+ }
+
+ String getContent() {
+ return content;
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/NeutralTabCodec.java b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/NeutralTabCodec.java
new file mode 100644
index 0000000000..5d2e4db39a
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/java/org/apache/tinkerpop/tinkeradoc/NeutralTabCodec.java
@@ -0,0 +1,221 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Serializes and deserializes a list of {@link NeutralTab}s as a compact JSON array. This is the
+ * on-the-wire form of the neutral tab model that the execution pass writes into a neutral custom
+ * block and that each backend converter reads back.
+ * <p>
+ * The schema is fixed and internal: an array of objects
+ * <code>{"label":…,"language":…,"kind":"CONSOLE"|"SOURCE","content":…}</code>. Because the schema
+ * is known and small, this codec is intentionally self-contained (no external JSON dependency).
+ * It is a strict round-trip: {@code parse(serialize(tabs))} reproduces the input exactly, including
+ * arbitrary content with quotes, backslashes, and control characters.
+ */
+final class NeutralTabCodec {
+
+ private NeutralTabCodec() {
+ }
+
+ /**
+ * Serializes tabs to a JSON array string.
+ */
+ static String serialize(final List<NeutralTab> tabs) {
+ final StringBuilder sb = new StringBuilder();
+ sb.append('[');
+ for (int i = 0; i < tabs.size(); i++) {
+ if (i > 0) sb.append(',');
+ final NeutralTab tab = tabs.get(i);
+ sb.append('{');
+ appendMember(sb, "label", tab.getLabel(), true);
+ appendMember(sb, "language", tab.getLanguage(), false);
+ appendMember(sb, "kind", tab.getKind().name(), false);
+ appendMember(sb, "content", tab.getContent(), false);
+ sb.append('}');
+ }
+ sb.append(']');
+ return sb.toString();
+ }
+
+ /**
+ * Parses a JSON array string (as produced by {@link #serialize}) back into tabs.
+ *
+ * @throws IllegalArgumentException if the input is not well-formed for this schema
+ */
+ static List<NeutralTab> parse(final String json) {
+ final Parser parser = new Parser(json);
+ final List<NeutralTab> tabs = new ArrayList<>();
+ parser.skipWs();
+ parser.expect('[');
+ parser.skipWs();
+ if (parser.peek() == ']') {
+ parser.next();
+ return tabs;
+ }
+ while (true) {
+ parser.skipWs();
+ tabs.add(parseTab(parser));
+ parser.skipWs();
+ final char c = parser.next();
+ if (c == ']') break;
+ if (c != ',') throw parser.error("expected ',' or ']'");
+ }
+ return tabs;
+ }
+
+ private static NeutralTab parseTab(final Parser parser) {
+ parser.expect('{');
+ String label = null;
+ String language = null;
+ String kind = null;
+ String content = null;
+ parser.skipWs();
+ if (parser.peek() == '}') {
+ parser.next();
+ throw parser.error("empty tab object");
+ }
+ while (true) {
+ parser.skipWs();
+ final String key = parser.parseString();
+ parser.skipWs();
+ parser.expect(':');
+ parser.skipWs();
+ final String value = parser.parseString();
+ switch (key) {
+ case "label": label = value; break;
+ case "language": language = value; break;
+ case "kind": kind = value; break;
+ case "content": content = value; break;
+ default: throw parser.error("unexpected key: " + key);
+ }
+ parser.skipWs();
+ final char c = parser.next();
+ if (c == '}') break;
+ if (c != ',') throw parser.error("expected ',' or '}'");
+ }
+ if (label == null || language == null || kind == null || content == null) {
+ throw parser.error("missing member in tab object");
+ }
+ return new NeutralTab(label, language, NeutralTab.Kind.valueOf(kind), content);
+ }
+
+ private static void appendMember(final StringBuilder sb, final String key, final String value,
+ final boolean first) {
+ if (!first) sb.append(',');
+ writeString(sb, key);
+ sb.append(':');
+ writeString(sb, value);
+ }
+
+ private static void writeString(final StringBuilder sb, final String s) {
+ sb.append('"');
+ for (int i = 0; i < s.length(); i++) {
+ final char c = s.charAt(i);
+ switch (c) {
+ case '"': sb.append("\\\""); break;
+ case '\\': sb.append("\\\\"); break;
+ case '\n': sb.append("\\n"); break;
+ case '\r': sb.append("\\r"); break;
+ case '\t': sb.append("\\t"); break;
+ case '\b': sb.append("\\b"); break;
+ case '\f': sb.append("\\f"); break;
+ default:
+ if (c < 0x20) {
+ sb.append(String.format("\\u%04x", (int) c));
+ } else {
+ sb.append(c);
+ }
+ }
+ }
+ sb.append('"');
+ }
+
+ /** A minimal recursive-descent scanner for the fixed neutral-tab JSON schema. */
+ private static final class Parser {
+ private final String s;
+ private int pos;
+
+ Parser(final String s) {
+ this.s = s;
+ }
+
+ void skipWs() {
+ while (pos < s.length()) {
+ final char c = s.charAt(pos);
+ if (c == ' ' || c == '\t' || c == '\n' || c == '\r') pos++;
+ else break;
+ }
+ }
+
+ char peek() {
+ if (pos >= s.length()) throw error("unexpected end of input");
+ return s.charAt(pos);
+ }
+
+ char next() {
+ if (pos >= s.length()) throw error("unexpected end of input");
+ return s.charAt(pos++);
+ }
+
+ void expect(final char expected) {
+ final char c = next();
+ if (c != expected) throw error("expected '" + expected + "' but found '" + c + "'");
+ }
+
+ String parseString() {
+ expect('"');
+ final StringBuilder sb = new StringBuilder();
+ while (true) {
+ final char c = next();
+ if (c == '"') break;
+ if (c == '\\') {
+ final char esc = next();
+ switch (esc) {
+ case '"': sb.append('"'); break;
+ case '\\': sb.append('\\'); break;
+ case '/': sb.append('/'); break;
+ case 'n': sb.append('\n'); break;
+ case 'r': sb.append('\r'); break;
+ case 't': sb.append('\t'); break;
+ case 'b': sb.append('\b'); break;
+ case 'f': sb.append('\f'); break;
+ case 'u':
+ if (pos + 4 > s.length()) throw error("truncated \\u escape");
+ final String hex = s.substring(pos, pos + 4);
+ pos += 4;
+ sb.append((char) Integer.parseInt(hex, 16));
+ break;
+ default: throw error("invalid escape: \\" + esc);
+ }
+ } else {
+ sb.append(c);
+ }
+ }
+ return sb.toString();
+ }
+
+ IllegalArgumentException error(final String message) {
+ return new IllegalArgumentException("Invalid neutral-tab JSON at position " + pos + ": " + message);
+ }
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/main/resources/META-INF/services/org.asciidoctor.jruby.converter.spi.ConverterRegistry b/docs/tinkeradoc-extension/src/main/resources/META-INF/services/org.asciidoctor.jruby.converter.spi.ConverterRegistry
new file mode 100644
index 0000000000..95289be075
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/main/resources/META-INF/services/org.asciidoctor.jruby.converter.spi.ConverterRegistry
@@ -0,0 +1 @@
+org.apache.tinkerpop.tinkeradoc.GremlinDocsConverterRegistry
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/DualBackendIntegrationTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/DualBackendIntegrationTest.java
new file mode 100644
index 0000000000..48e634970a
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/DualBackendIntegrationTest.java
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.Asciidoctor;
+import org.asciidoctor.Options;
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * End-to-end test of the dual-backend pipeline: one document, rendered to HTML and to Markdown from
+ * a SINGLE Gremlin execution via the shared {@link GremlinExecutionCache}. Verifies that the
+ * executed {@code ==>} output appears identically in both backends (parity) and that each backend's
+ * tab rendering is correct.
+ */
+public class DualBackendIntegrationTest {
+
+ private static final class RecordingExecutor implements GremlinTreeprocessor.StatementExecutor {
+ final List<String> statements = new ArrayList<>();
+
+ @Override
+ public String execute(final String statement) {
+ statements.add(statement);
+ // The real Gremlin console echoes the submitted statement on the first line and puts
+ // ==> results on subsequent lines; the treeprocessor drops that echo line. Mimic that
+ // shape so the executed result survives into the rendered console transcript.
+ if (statement.equals("g.V(1)")) return statement + "\n==>v[1]";
+ return statement + "\n==>ok";
+ }
+ }
+
+ private static final String DOC =
+ "= Guide\n\n== Basics\n\nSome intro text.\n\n"
+ + "[gremlin-groovy,modern]\n----\ng.V(1)\n----\n";
+
+ @Test
+ public void oneExecutionFeedsBothBackends() {
+ final RecordingExecutor executor = new RecordingExecutor();
+ final GremlinExecutionCache cache = new GremlinExecutionCache();
+
+ final String html;
+ final String markdown;
+
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.javaConverterRegistry().register(MarkdownConverter.class);
+
+ // Pass 1: HTML backend executes Gremlin and populates the cache.
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(new GremlinTreeprocessor(executor, null, cache));
+ html = asciidoctor.convert(DOC, Options.builder().backend("html5").build());
+ final int executedAfterHtml = executor.statements.size();
+
+ // Pass 2: Markdown backend must reuse the cache and execute nothing more.
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(new GremlinTreeprocessor(executor, null, cache));
+ markdown = asciidoctor.convert(DOC, Options.builder().backend("tpmarkdown").build());
+ final int executedAfterMd = executor.statements.size();
+
+ assertThat("Markdown pass must not re-execute Gremlin", executedAfterMd, is(executedAfterHtml));
+ }
+
+ // HTML backend: tab widget with the executed result. CodeRay highlights the numeric index,
+ // so the result renders as v[<span class="integer">1</span>] rather than literal "v[1]".
+ assertThat(html, containsString("section class=\"tabs"));
+ assertThat(html, containsString("==>v["));
+ assertThat(html, containsString("<span class=\"integer\">1</span>"));
+
+ // Markdown backend: heading, prose, and a fenced console block with the executed result
+ // preserved verbatim (no highlighting).
+ assertThat(markdown, containsString("## Basics"));
+ assertThat(markdown, containsString("Some intro text."));
+ assertThat(markdown, containsString("**console (groovy)**"));
+ assertThat(markdown, containsString("```text"));
+ assertThat(markdown, containsString("gremlin> g.V(1)"));
+ assertThat(markdown, containsString("==>v[1]"));
+
+ // Parity: the executed ==> output appears in BOTH backends (HTML highlights the integer,
+ // Markdown keeps it plain, but the same executed value is present in each).
+ assertThat("executed output present in HTML", html.contains("==>v["), is(true));
+ assertThat("executed output present in Markdown", markdown.contains("==>v[1]"), is(true));
+ }
+
+ @Test
+ public void markdownDoesNotContainHtmlTabWidget() {
+ final RecordingExecutor executor = new RecordingExecutor();
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.javaConverterRegistry().register(MarkdownConverter.class);
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(
+ new GremlinTreeprocessor(executor, null, new GremlinExecutionCache()));
+ final String markdown = asciidoctor.convert(DOC, Options.builder().backend("tpmarkdown").build());
+ assertThat(markdown.contains("class=\"tabs"), is(false));
+ assertThat(markdown.contains("CodeRay"), is(false));
+ }
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/GremlinExecutionCacheTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/GremlinExecutionCacheTest.java
new file mode 100644
index 0000000000..8e666be8d7
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/GremlinExecutionCacheTest.java
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.Asciidoctor;
+import org.asciidoctor.Options;
+import org.junit.Test;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Verifies the execute-once guarantee: when the same document is rendered twice (as the two backend
+ * passes do), live Gremlin execution happens only on the first pass and the second pass is served
+ * entirely from {@link GremlinExecutionCache}. This is the core of the mechanism that lets HTML and
+ * Markdown be produced from a single console session.
+ */
+public class GremlinExecutionCacheTest {
+
+ /** Records every executed statement so we can assert the second pass executes nothing. */
+ private static final class RecordingExecutor implements GremlinTreeprocessor.StatementExecutor {
+ final List<String> statements = new ArrayList<>();
+
+ @Override
+ public String execute(final String statement) {
+ statements.add(statement);
+ return "==>ok";
+ }
+ }
+
+ private static final String DOC =
+ "= Test\n\n[gremlin-groovy,modern]\n----\ng.V(1)\n----\n\n"
+ + "[gremlin-groovy,modern]\n----\ng.V(2)\n----\n";
+
+ @Test
+ public void secondRenderPassExecutesNothingWhenCacheShared() {
+ final RecordingExecutor executor = new RecordingExecutor();
+ final GremlinExecutionCache cache = new GremlinExecutionCache();
+
+ // Two processors sharing one cache simulate the two per-backend render passes: the plugin
+ // creates a fresh treeprocessor per convert(), but the SHARED cache persists between them.
+ final GremlinTreeprocessor pass1 = new GremlinTreeprocessor(executor, null, cache);
+ final GremlinTreeprocessor pass2 = new GremlinTreeprocessor(executor, null, cache);
+
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(pass1);
+ asciidoctor.convert(DOC, Options.builder().build());
+ final int afterPass1 = executor.statements.size();
+ assertThat("first pass must execute Gremlin", afterPass1 > 0, is(true));
+
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(pass2);
+ asciidoctor.convert(DOC, Options.builder().build());
+ final int afterPass2 = executor.statements.size();
+
+ assertThat("second pass must execute nothing (served from cache)",
+ afterPass2, is(afterPass1));
+ assertThat("both gremlin blocks cached", cache.size(), is(2));
+ }
+ }
+
+ @Test
+ public void separateCachesDoNotShareAndBothExecute() {
+ final RecordingExecutor executor = new RecordingExecutor();
+
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(new GremlinTreeprocessor(executor, null,
+ new GremlinExecutionCache()));
+ asciidoctor.convert(DOC, Options.builder().build());
+ final int afterPass1 = executor.statements.size();
+
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(new GremlinTreeprocessor(executor, null,
+ new GremlinExecutionCache()));
+ asciidoctor.convert(DOC, Options.builder().build());
+
+ assertThat("distinct caches re-execute", executor.statements.size() > afterPass1, is(true));
+ }
+ }
+
+ @Test
+ public void cachedContentDrivesRenderedOutput() {
+ final RecordingExecutor executor = new RecordingExecutor();
+ final GremlinExecutionCache cache = new GremlinExecutionCache();
+ final GremlinTreeprocessor pass1 = new GremlinTreeprocessor(executor, null, cache);
+ final GremlinTreeprocessor pass2 = new GremlinTreeprocessor(executor, null, cache);
+
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(pass1);
+ final String html1 = asciidoctor.convert(DOC, Options.builder().build());
+
+ asciidoctor.unregisterAllExtensions();
+ asciidoctor.javaExtensionRegistry().treeprocessor(pass2);
+ final String html2 = asciidoctor.convert(DOC, Options.builder().build());
+
+ // Both passes render the tab widget, and the second (cache-served) pass is equivalent.
+ assertThat(html1, containsString("section class=\"tabs"));
+ assertThat(html2, containsString("section class=\"tabs"));
+ }
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/GremlinHighlighterTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/GremlinHighlighterTest.java
new file mode 100644
index 0000000000..4aad7682b9
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/GremlinHighlighterTest.java
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.junit.Test;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.CoreMatchers.not;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Unit tests for {@link GremlinHighlighter}. With a {@code null} parent node no JRuby runtime is
+ * available, so {@code highlight()} falls back to HTML-escaped plain text; that path exercises the
+ * callout stripping/re-injection logic without needing a live Asciidoctor context. The full
+ * CodeRay-highlighted path is covered end-to-end by {@link IntegrationTest}.
+ */
+public class GremlinHighlighterTest {
+
+ private final GremlinHighlighter highlighter = new GremlinHighlighter();
+
+ @Test
+ public void shouldReturnEmptyForNullOrEmptySource() {
+ assertThat(highlighter.highlight(null, "groovy", null), is(""));
+ assertThat(highlighter.highlight(null, "groovy", ""), is(""));
+ }
+
+ @Test
+ public void shouldEscapeHtmlInFallbackPath() {
+ final String result = highlighter.highlight(null, "java", "List<String> x = new ArrayList<>()");
+ assertThat(result, containsString("List<String> x = new ArrayList<>()"));
+ }
+
+ @Test
+ public void shouldReinjectTrailingCalloutAsConum() {
+ final String result = highlighter.highlight(null, "groovy", "g.V() <1>");
+ assertThat(result, containsString("<b class=\"conum\">(1)</b>"));
+ // The raw callout marker must not survive in the output.
+ assertThat(result, not(containsString("<1>")));
+ }
+
+ @Test
+ public void shouldReinjectMultipleCalloutsOnSameLine() {
+ final String result = highlighter.highlight(null, "groovy", "g.V() <1> <2>");
+ assertThat(result, containsString("<b class=\"conum\">(1)</b>"));
+ assertThat(result, containsString("<b class=\"conum\">(2)</b>"));
+ }
+
+ @Test
+ public void shouldNotTreatMidLineAngleBracketsAsCallouts() {
+ // Only trailing <n> markers are callouts; a mid-line comparison must be escaped, not injected.
+ final String result = highlighter.highlight(null, "groovy", "x < 1");
+ assertThat(result, containsString("x < 1"));
+ assertThat(result, not(containsString("conum")));
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/HtmlTabRendererTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/HtmlTabRendererTest.java
new file mode 100644
index 0000000000..d75da871ad
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/HtmlTabRendererTest.java
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Unit tests for {@link HtmlTabRenderer} and the {@link NeutralTab} model. A {@code null} parent
+ * node yields no JRuby runtime, so highlighting falls back to HTML-escaped plain text; that is
+ * enough to verify the renderer produces the same tab-widget structure, labels, and kinds the
+ * treeprocessor produced when it built {@link TabbedHtmlBuilder.Tab}s directly. The full
+ * CodeRay-highlighted path is covered end-to-end by {@link IntegrationTest}.
+ */
+public class HtmlTabRendererTest {
+
+ private HtmlTabRenderer renderer;
+
+ @Before
+ public void setUp() {
+ renderer = new HtmlTabRenderer(new GremlinHighlighter(), new TabbedHtmlBuilder());
+ }
+
+ @Test
+ public void consoleFactorySetsLabelLanguageAndKind() {
+ final NeutralTab tab = NeutralTab.console("groovy", "gremlin> g.V()");
+ assertThat(tab.getLabel(), is("console (groovy)"));
+ assertThat(tab.getLanguage(), is("groovy"));
+ assertThat(tab.getKind(), is(NeutralTab.Kind.CONSOLE));
+ assertThat(tab.getContent(), is("gremlin> g.V()"));
+ }
+
+ @Test
+ public void sourceFactorySetsLabelLanguageAndKind() {
+ final NeutralTab tab = NeutralTab.source("python", "g.V()");
+ assertThat(tab.getLabel(), is("python"));
+ assertThat(tab.getLanguage(), is("python"));
+ assertThat(tab.getKind(), is(NeutralTab.Kind.SOURCE));
+ }
+
+ @Test
+ public void rendersConsolePlusSourceTabsWithCorrectLabels() {
+ final List<NeutralTab> tabs = Arrays.asList(
+ NeutralTab.console("groovy", "gremlin> g.V(1)"),
+ NeutralTab.source("groovy", "g.V(1)"));
+ final String html = renderer.render(null, tabs);
+ assertThat(html, containsString("<section class=\"tabs tabs-2\">"));
+ assertThat(html, containsString("class=\"tab-label-1\">console (groovy)</label>"));
+ assertThat(html, containsString("class=\"tab-label-2\">groovy</label>"));
+ }
+
+ @Test
+ public void rendersMultiLanguageTabGroup() {
+ final List<NeutralTab> tabs = Arrays.asList(
+ NeutralTab.console("groovy", "gremlin> g.V(1)"),
+ NeutralTab.source("groovy", "g.V(1)"),
+ NeutralTab.source("java", "g.V(1)"),
+ NeutralTab.source("python", "g.V(1)"));
+ final String html = renderer.render(null, tabs);
+ assertThat(html, containsString("<section class=\"tabs tabs-4\">"));
+ assertThat(html, containsString("class=\"tab-label-3\">java</label>"));
+ assertThat(html, containsString("class=\"tab-label-4\">python</label>"));
+ }
+
+ @Test
+ public void escapesHtmlInSourceContentViaFallbackPath() {
+ final List<NeutralTab> tabs = Collections.singletonList(
+ NeutralTab.source("java", "List<String> x = new ArrayList<>()"));
+ final String html = renderer.render(null, tabs);
+ assertThat(html, containsString("List<String> x = new ArrayList<>()"));
+ }
+
+ @Test
+ public void producesCodeRayPreBlocks() {
+ final List<NeutralTab> tabs = Collections.singletonList(
+ NeutralTab.source("java", "g.V()"));
+ final String html = renderer.render(null, tabs);
+ assertThat(html, containsString("<pre class=\"CodeRay highlight\"><code data-lang=\"java\">"));
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/LlmsTxtGeneratorTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/LlmsTxtGeneratorTest.java
new file mode 100644
index 0000000000..42995012f1
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/LlmsTxtGeneratorTest.java
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.List;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Tests for {@link LlmsTxtGenerator}: the llms.txt discovery-index format and its title/description
+ * extraction.
+ */
+public class LlmsTxtGeneratorTest {
+
+ private final LlmsTxtGenerator gen =
+ new LlmsTxtGenerator("Apache TinkerPop Documentation", "Docs for a graph computing framework.");
+
+ @Test
+ public void emitsH1AndSummaryBlockquote() {
+ final String out = gen.generate(Arrays.asList(
+ new LlmsTxtGenerator.Entry("dev/io/index.md", "IO Reference", "How IO works.")));
+ assertThat(out, containsString("# Apache TinkerPop Documentation\n"));
+ assertThat(out, containsString("> Docs for a graph computing framework.\n"));
+ }
+
+ @Test
+ public void groupsPagesByBookAsSectionsWithLinks() {
+ final List<LlmsTxtGenerator.Entry> entries = Arrays.asList(
+ new LlmsTxtGenerator.Entry("dev/io/index.md", "IO Reference", "IO overview."),
+ new LlmsTxtGenerator.Entry("dev/io/graphson.md", "GraphSON", "A JSON format."),
+ new LlmsTxtGenerator.Entry("reference/index.md", "Reference", "The reference."));
+ final String out = gen.generate(entries);
+ assertThat(out, containsString("## dev/io\n"));
+ assertThat(out, containsString("## reference\n"));
+ assertThat(out, containsString("- [GraphSON](dev/io/graphson.md): A JSON format."));
+ assertThat(out, containsString("- [IO Reference](dev/io/index.md): IO overview."));
+ }
+
+ @Test
+ public void landingPageListedFirstInItsSection() {
+ final List<LlmsTxtGenerator.Entry> entries = Arrays.asList(
+ new LlmsTxtGenerator.Entry("dev/io/graphson.md", "GraphSON", "g"),
+ new LlmsTxtGenerator.Entry("dev/io/index.md", "IO Reference", "i"));
+ final String out = gen.generate(entries);
+ final int idx = out.indexOf("dev/io/index.md");
+ final int gs = out.indexOf("dev/io/graphson.md");
+ assertThat("landing page must come first", idx < gs, is(true));
+ }
+
+ @Test
+ public void linkPrefixIsPrependedToEveryPagePath() {
+ final LlmsTxtGenerator prefixed = new LlmsTxtGenerator(
+ "Apache TinkerPop Documentation", "Docs.", "docs/3.7.7/");
+ final String out = prefixed.generate(Arrays.asList(
+ new LlmsTxtGenerator.Entry("dev/io/graphson.md", "GraphSON", "A JSON format.")));
+ assertThat(out, containsString("- [GraphSON](docs/3.7.7/dev/io/graphson.md): A JSON format."));
+ }
+
+ @Test
+ public void extractsTitleFromFirstHeading() {
+ final String content = "> pointer\n\n<a id=\"graphson\"></a>\n# GraphSON\n\nGraphSON is a JSON format.\n";
+ assertThat(LlmsTxtGenerator.extractTitle(content, "dev/io/graphson.md"), is("GraphSON"));
+ }
+
+ @Test
+ public void titleFallsBackToFileNameWhenNoHeading() {
+ assertThat(LlmsTxtGenerator.extractTitle("just prose, no heading\n", "dev/io/orphan.md"),
+ is("orphan"));
+ }
+
+ @Test
+ public void prefersCuratedLlmsSummaryOverProse() {
+ final String content = "> For the complete documentation index, see [llms.txt](/llms.txt)\n\n"
+ + "<a id=\"combine-step\"></a>\n### Combine Step\n\n"
+ + "<!-- llms-summary: Combine two lists element-wise; handy for zipping results. -->\n\n"
+ + "The combine()-step merges the incoming list with the provided list argument.\n";
+ assertThat(LlmsTxtGenerator.extractDescription(content),
+ is("Combine two lists element-wise; handy for zipping results."));
+ }
+
+ @Test
+ public void curatedSummaryUsedVerbatimNotTruncatedToOneSentence() {
+ final String content = "<a id=\"x\"></a>\n# X\n\n"
+ + "<!-- llms-summary: First clause. Second clause stays too. -->\n\nProse.\n";
+ // Unlike the prose heuristic (one sentence), curated text is used as authored.
+ assertThat(LlmsTxtGenerator.extractDescription(content),
+ is("First clause. Second clause stays too."));
+ }
+
+ @Test
+ public void fallsBackToProseWhenNoCuratedSummary() {
+ final String content = "<a id=\"x\"></a>\n# X\n\nPlain prose description here. More detail.\n";
+ assertThat(LlmsTxtGenerator.extractDescription(content), is("Plain prose description here."));
+ }
+
+ @Test
+ public void extractsDescriptionFromFirstProseParagraph() {
+ final String content = "> For the complete documentation index, see [llms.txt](/llms.txt)\n\n"
+ + "<a id=\"graphson\"></a>\n# GraphSON\n\n"
+ + "GraphSON is a `JSON`-based [format](x.html) designed for humans. More detail follows.\n";
+ final String desc = LlmsTxtGenerator.extractDescription(content);
+ assertThat(desc, is("GraphSON is a JSON-based format designed for humans."));
+ }
+
+ @Test
+ public void descriptionSkipsStructuralLinesLikeFencesAndImages() {
+ final String content = "<a id=\"x\"></a>\n# X\n\n\n\n```java\ncode\n```\n\nActual prose here.\n";
+ assertThat(LlmsTxtGenerator.extractDescription(content), is("Actual prose here."));
+ }
+
+ @Test
+ public void emptyDescriptionWhenNoProse() {
+ final String content = "<a id=\"x\"></a>\n# X\n\n```\ncode\n```\n";
+ assertThat(LlmsTxtGenerator.extractDescription(content), is(""));
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownConverterProbeTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownConverterProbeTest.java
new file mode 100644
index 0000000000..0eaf4ce214
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownConverterProbeTest.java
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.Asciidoctor;
+import org.asciidoctor.Options;
+import org.asciidoctor.SafeMode;
+import org.junit.Test;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.CoreMatchers.not;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Probe for how {@link MarkdownConverter} handles core node types, including the crucial case of a
+ * {@code pass} block whose raw content (produced by the treeprocessor for a gremlin tab group) must
+ * survive verbatim into the Markdown output.
+ */
+public class MarkdownConverterProbeTest {
+
+ private String toMarkdown(final String adoc) {
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.javaConverterRegistry().register(MarkdownConverter.class);
+ return asciidoctor.convert(adoc,
+ Options.builder().safe(SafeMode.UNSAFE).backend("tpmarkdown").build());
+ }
+ }
+
+ @Test
+ public void descriptionListRendersLinearlyWithoutBlowup() {
+ // Regression guard: a description (definition) list previously fell through to a blanket
+ // getContent() fallback that re-rendered subtrees, producing an output that grew far beyond
+ // its input and exhausted the heap on dense books. Output must stay proportional to input.
+ final StringBuilder src = new StringBuilder("= D\n\n");
+ for (int i = 0; i < 200; i++) {
+ src.append("term").append(i).append(":: definition text for entry ").append(i).append("\n");
+ }
+ final String md = toMarkdown(src.toString());
+ // Linear: a dense dlist must not explode (input ~7KB; allow generous headroom but bar blowup).
+ assertThat("dlist output must stay proportional to input, was " + md.length(),
+ md.length() < src.length() * 5, is(true));
+ assertThat(md, containsString("**term0**"));
+ assertThat(md, containsString("definition text for entry 0"));
+ assertThat(md, containsString("**term199**"));
+ }
+
+ @Test
+ public void nestedListRendersAllItemsOnce() {
+ final String md = toMarkdown("= D\n\n* a\n** a1\n** a2\n* b\n");
+ assertThat(md, containsString("- a"));
+ assertThat(md, containsString("- b"));
+ assertThat(md, containsString("a1"));
+ assertThat(md, containsString("a2"));
+ // No item may be duplicated by re-conversion.
+ assertThat(countOccurrences(md, "- b"), is(1));
+ }
+
+ private static int countOccurrences(final String haystack, final String needle) {
+ int c = 0, i = 0;
+ while ((i = haystack.indexOf(needle, i)) >= 0) { c++; i += needle.length(); }
+ return c;
+ }
+
+ @Test
+ public void rendersHeadingsAndParagraphs() {
+ final String md = toMarkdown("= Title\n\n== Section One\n\nHello world.\n\n=== Sub\n\nMore text.\n");
+ assertThat(md, containsString("# Title"));
+ assertThat(md, containsString("## Section One"));
+ assertThat(md, containsString("### Sub"));
+ assertThat(md, containsString("Hello world."));
+ }
+
+ @Test
+ public void llmsSummaryAttributeBecomesHiddenCommentNotBodyText() {
+ // A curated [llms-summary="..."] on a section is emitted as a hidden HTML comment for the
+ // llms.txt generator, and must NOT appear as visible body prose.
+ final String md = toMarkdown("= T\n\n[llms-summary=\"Combine two lists element-wise.\"]\n"
+ + "== Combine Step\n\nThe combine() step merges lists.\n");
+ assertThat(md, containsString("<!-- llms-summary: Combine two lists element-wise. -->"));
+ // The section's real prose still renders.
+ assertThat(md, containsString("The combine() step merges lists."));
+ // The summary text appears ONLY inside the comment, not as a standalone paragraph.
+ assertThat(md.replace("<!-- llms-summary: Combine two lists element-wise. -->", ""),
+ not(containsString("Combine two lists element-wise.")));
+ }
+
+ @Test
+ public void noLlmsSummaryCommentWhenAttributeAbsent() {
+ final String md = toMarkdown("= T\n\n== Plain Step\n\nSome prose.\n");
+ assertThat(md, not(containsString("llms-summary")));
+ }
+
+ @Test
+ public void emitsExplicitAnchorFromSectionId() {
+ // An explicit AsciiDoc id must surface as an HTML anchor immediately before the heading so
+ // xrefs of the form [label](#the-id) resolve.
+ final String md = toMarkdown("= T\n\n[#my-anchor]\n== Custom Section\n\nBody.\n");
+ assertThat(md, containsString("<a id=\"my-anchor\"></a>"));
+ assertThat(md, containsString("## Custom Section"));
+ }
+
+ @Test
+ public void emitsAutoGeneratedAnchorFromSectionId() {
+ // Even without an explicit id, AsciiDoc auto-generates section ids (e.g. _custom_section);
+ // these must be anchored too so auto-xrefs resolve after the page split.
+ final String md = toMarkdown("= T\n\n== Custom Section\n\nBody.\n");
+ assertThat(md, containsString("<a id=\"_custom_section\"></a>"));
+ }
+
+ @Test
+ public void passBlockContentSurvivesVerbatim() {
+ // Simulate what the treeprocessor emits for a gremlin tab group in markdown mode: a pass
+ // block whose body is pre-rendered Markdown. It must appear unchanged in the output.
+ final String adoc = "= T\n\n++++\n**console (groovy)**\n\n```text\ngremlin> g.V()\n```\n++++\n";
+ final String md = toMarkdown(adoc);
+ assertThat(md, containsString("**console (groovy)**"));
+ assertThat(md, containsString("```text\ngremlin> g.V()\n```"));
+ }
+
+ private String toMarkdownWithImagesdir(final String adoc, final String imagesdir) {
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.javaConverterRegistry().register(MarkdownConverter.class);
+ return asciidoctor.convert(adoc, Options.builder().safe(SafeMode.UNSAFE).backend("tpmarkdown")
+ .attributes(org.asciidoctor.Attributes.builder().attribute("imagesdir", imagesdir).build())
+ .build());
+ }
+ }
+
+ @Test
+ public void blockImageTargetPrefixedWithImagesdir() {
+ final String md = toMarkdownWithImagesdir("= T\n\nimage::match-step.png[match step]\n", "../images");
+ assertThat(md, containsString(""));
+ }
+
+ @Test
+ public void inlineImageTargetPrefixedWithImagesdir() {
+ final String md = toMarkdownWithImagesdir("= T\n\nSee image:logo.png[Logo] here.\n", "../../images");
+ assertThat(md, containsString(""));
+ }
+
+ @Test
+ public void absoluteAndUrlImageTargetsAreNotPrefixed() {
+ final String abs = toMarkdownWithImagesdir("= T\n\nimage::/static/x.png[x]\n", "../images");
+ assertThat(abs, containsString(""));
+ final String url = toMarkdownWithImagesdir("= T\n\nimage::https://ex.com/y.png[y]\n", "../images");
+ assertThat(url, containsString(""));
+ }
+
+ @Test
+ public void rendersUnorderedList() {
+ final String md = toMarkdown("= T\n\n* one\n* two\n* three\n");
+ assertThat(md, containsString("- one"));
+ assertThat(md, containsString("- two"));
+ }
+
+ @Test
+ public void rendersInlineFormatting() {
+ final String md = toMarkdown("= T\n\nSome *bold* and `code` and _italic_.\n");
+ assertThat(md, containsString("`code`"));
+ assertThat(md, containsString("**bold**"));
+ assertThat(md, containsString("_italic_"));
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownSplitterTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownSplitterTest.java
new file mode 100644
index 0000000000..0476726672
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownSplitterTest.java
@@ -0,0 +1,202 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.junit.Test;
+
+import java.util.List;
+import java.util.stream.Collectors;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.hasItem;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.CoreMatchers.not;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Tests for {@link MarkdownSplitter}: the size-driven, heading-aligned page splitter and its
+ * cross-page link rewriting.
+ */
+public class MarkdownSplitterTest {
+
+ private static String heading(final String id, final int level, final String title) {
+ final StringBuilder h = new StringBuilder();
+ for (int i = 0; i < level; i++) h.append('#');
+ return "<a id=\"" + id + "\"></a>\n" + h + " " + title + "\n";
+ }
+
+ private static String filler(final int bytes) {
+ final StringBuilder sb = new StringBuilder();
+ while (sb.length() < bytes) sb.append("lorem ipsum dolor sit amet consectetur\n");
+ return sb.toString();
+ }
+
+ private static List<String> fileNames(final List<MarkdownSplitter.Page> pages) {
+ return pages.stream().map(MarkdownSplitter.Page::getFileName).collect(Collectors.toList());
+ }
+
+ @Test
+ public void smallBookStaysSinglePage() {
+ final String md = heading("_io_reference", 1, "IO Reference") + "\nsome intro\n\n"
+ + heading("graphml", 1, "GraphML") + "\nshort content\n";
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(50_000).split(md, "index.md");
+ assertThat(pages.size(), is(1));
+ assertThat(pages.get(0).getFileName(), is("index.md"));
+ assertThat(pages.get(0).getContent(), containsString("# GraphML"));
+ }
+
+ @Test
+ public void oversizedSectionBecomesOwnPage() {
+ final String md = heading("_io_reference", 1, "IO Reference") + "\nintro\n\n"
+ + heading("graphml", 1, "GraphML") + "\nshort\n\n"
+ + heading("graphson", 1, "GraphSON") + "\n" + filler(60_000);
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(50_000).split(md, "index.md");
+ final List<String> names = fileNames(pages);
+ // The big GraphSON chapter must be split off into graphson.md; index keeps the small parts.
+ assertThat(names, hasItem("index.md"));
+ assertThat(names, hasItem("graphson.md"));
+ final MarkdownSplitter.Page index = pages.stream()
+ .filter(p -> p.getFileName().equals("index.md")).findFirst().orElseThrow(AssertionError::new);
+ assertThat(index.getContent(), containsString("# GraphML"));
+ assertThat(index.getContent(), not(containsString(filler(60_000).substring(0, 200))));
+ }
+
+ @Test
+ public void everyDivisiblePageUnderBudget() {
+ // A, B, C each have small children? No — here each is a single leaf heading. B alone is
+ // 60KB of direct content, which is indivisible: it must occupy one page even though that
+ // page exceeds the budget (there is no finer heading to split on). Every OTHER page must
+ // be under budget.
+ final String md = heading("book", 1, "Book") + "\nintro\n\n"
+ + heading("a", 1, "A") + "\n" + filler(40_000)
+ + heading("b", 1, "B") + "\n" + filler(60_000)
+ + heading("c", 1, "C") + "\n" + filler(20_000);
+ final int budget = 50_000;
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(budget).split(md, "index.md");
+ for (final MarkdownSplitter.Page p : pages) {
+ final boolean indivisible = p.getFileName().equals("b.md");
+ if (!indivisible) {
+ assertThat(p.getFileName() + " over budget: " + p.getContent().length(),
+ p.getContent().length() <= budget, is(true));
+ }
+ }
+ // The indivisible 60KB leaf is isolated on its own page (so it doesn't bloat neighbors).
+ assertThat(fileNames(pages), hasItem("b.md"));
+ }
+
+ @Test
+ public void renderedPageIncludingPointerStaysUnderBudget() {
+ // Every divisible page's FINAL content (which includes the prepended llms.txt pointer) must
+ // stay within the budget: the splitter reserves room for the pointer when packing.
+ final int budget = 50_000;
+ final StringBuilder md = new StringBuilder(heading("book", 1, "Book") + "\nintro\n\n");
+ // Many medium sections that would pack right up against the budget.
+ for (int i = 0; i < 8; i++) {
+ md.append(heading("sec-" + i, 1, "Section " + i)).append('\n').append(filler(12_000));
+ }
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(budget).split(md.toString(), "index.md");
+ for (final MarkdownSplitter.Page p : pages) {
+ assertThat(p.getFileName() + " (incl pointer) = " + p.getContent().length(),
+ p.getContent().length() <= budget, is(true));
+ assertThat(p.getContent(), containsString("[llms.txt](/llms.txt)"));
+ }
+ }
+
+ @Test
+ public void indivisibleOversizedLeafIsIsolatedOnOwnPage() {
+ final String md = heading("book", 1, "Book") + "\nintro\n\n"
+ + heading("small", 1, "Small") + "\nshort\n\n"
+ + heading("huge", 1, "Huge") + "\n" + filler(70_000);
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(50_000).split(md, "index.md");
+ final MarkdownSplitter.Page huge = pages.stream()
+ .filter(p -> p.getFileName().equals("huge.md")).findFirst().orElseThrow(AssertionError::new);
+ // The huge leaf page contains only the Huge section, not the small one.
+ assertThat(huge.getContent(), containsString("# Huge"));
+ assertThat(huge.getContent(), not(containsString("# Small")));
+ }
+
+ @Test
+ public void descendsToChildrenWhenChapterTooBig() {
+ // A chapter (90KB total) that exceeds the budget descends to its h2 children. Greedy packing
+ // keeps pages full: the chapter page leads with the chapter heading + as many whole child
+ // sections as fit; the remainder overflow to further pages. So we get chapter.md (heading +
+ // Section One) and a second page for Section Two — never a mid-section cut.
+ final String md = heading("book", 1, "Book") + "\nintro\n\n"
+ + heading("chapter", 1, "Chapter") + "\nlead-in\n\n"
+ + heading("sec-one", 2, "Section One") + "\n" + filler(45_000)
+ + heading("sec-two", 2, "Section Two") + "\n" + filler(45_000);
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(50_000).split(md, "index.md");
+ final List<String> names = fileNames(pages);
+ assertThat(names, hasItem("chapter.md"));
+ // Section Two overflows to its own page (named from its anchor); each h2 stays whole.
+ assertThat(names, hasItem("sec-two.md"));
+ final MarkdownSplitter.Page chapter = pages.stream()
+ .filter(p -> p.getFileName().equals("chapter.md")).findFirst().orElseThrow(AssertionError::new);
+ assertThat(chapter.getContent(), containsString("# Chapter"));
+ assertThat(chapter.getContent(), containsString("## Section One"));
+ assertThat(chapter.getContent(), not(containsString("## Section Two")));
+ }
+
+ @Test
+ public void rewritesCrossPageLinksAndKeepsSamePageBare() {
+ final String md = heading("book", 1, "Book") + "\nintro\n\n"
+ + heading("graphml", 1, "GraphML") + "\nSee [GraphSON](#graphson) and [self](#graphml).\n\n"
+ + heading("graphson", 1, "GraphSON") + "\n" + filler(60_000)
+ + "Back to [GraphML](#graphml).\n";
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(50_000).split(md, "index.md");
+
+ final MarkdownSplitter.Page index = pages.stream()
+ .filter(p -> p.getFileName().equals("index.md")).findFirst().orElseThrow(AssertionError::new);
+ final MarkdownSplitter.Page graphson = pages.stream()
+ .filter(p -> p.getFileName().equals("graphson.md")).findFirst().orElseThrow(AssertionError::new);
+
+ // GraphML lives on index; its link to GraphSON (another page) is rewritten; self-link stays bare.
+ assertThat(index.getContent(), containsString("[GraphSON](graphson.md#graphson)"));
+ assertThat(index.getContent(), containsString("[self](#graphml)"));
+ // GraphSON page links back to GraphML on the index page.
+ assertThat(graphson.getContent(), containsString("[GraphML](index.md#graphml)"));
+ }
+
+ @Test
+ public void hashCommentsInsideCodeFenceAreNotTreatedAsHeadings() {
+ // A code fence containing shell/properties comment lines that begin with '#' must NOT be
+ // parsed as section headings (which would split the page mid-code-block and produce generic
+ // "section.md" file names). The whole fenced block stays with its owning section.
+ final String md = heading("book", 1, "Book") + "\nintro\n\n"
+ + heading("config", 1, "Config") + "\nHere is a config file:\n\n"
+ + "```properties\n# Spark Configuration\nspark.master=local[4]\n# another comment\n"
+ + "spark.executor.memory=1g\n```\n\nAfter the block.\n";
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(50_000).split(md, "index.md");
+ final List<String> names = fileNames(pages);
+ // Only index.md — no page split off a code comment, no generic "section*.md".
+ assertThat(names.stream().noneMatch(n -> n.startsWith("section")), is(true));
+ final String body = pages.get(0).getContent();
+ // The fenced block and its comments remain intact and are not promoted to headings.
+ assertThat(body, containsString("# Spark Configuration"));
+ assertThat(body, containsString("```properties"));
+ assertThat(body, containsString("After the block."));
+ }
+
+ @Test
+ public void unknownAnchorsLeftUnchanged() {
+ final String md = heading("book", 1, "Book") + "\nSee [external](#not-a-heading).\n";
+ final List<MarkdownSplitter.Page> pages = new MarkdownSplitter(50_000).split(md, "index.md");
+ assertThat(pages.get(0).getContent(), containsString("[external](#not-a-heading)"));
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownTabRendererTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownTabRendererTest.java
new file mode 100644
index 0000000000..34d8e61dd7
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/MarkdownTabRendererTest.java
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.CoreMatchers.not;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Unit tests for {@link MarkdownTabRenderer}: the Markdown backend for the neutral tab model.
+ */
+public class MarkdownTabRendererTest {
+
+ private MarkdownTabRenderer renderer;
+
+ @Before
+ public void setUp() {
+ renderer = new MarkdownTabRenderer();
+ }
+
+ @Test
+ public void consoleTabRendersAsTextFence() {
+ final String md = renderer.render(Collections.singletonList(
+ NeutralTab.console("groovy", "gremlin> g.V()\n==>v[1]")));
+ assertThat(md, containsString("**console (groovy)**"));
+ assertThat(md, containsString("```text\ngremlin> g.V()\n==>v[1]\n```"));
+ }
+
+ @Test
+ public void sourceTabRendersWithLanguageFence() {
+ final String md = renderer.render(Collections.singletonList(
+ NeutralTab.source("java", "g.V(1)")));
+ assertThat(md, containsString("**java**"));
+ assertThat(md, containsString("```java\ng.V(1)\n```"));
+ }
+
+ @Test
+ public void multipleTabsEmittedInOrderSeparatedByBlankLines() {
+ final List<NeutralTab> tabs = Arrays.asList(
+ NeutralTab.console("groovy", "gremlin> g.V(1)"),
+ NeutralTab.source("groovy", "g.V(1)"),
+ NeutralTab.source("python", "g.V(1)"));
+ final String md = renderer.render(tabs);
+ final int console = md.indexOf("**console (groovy)**");
+ final int groovy = md.indexOf("**groovy**");
+ final int python = md.indexOf("**python**");
+ assertThat(console >= 0 && groovy > console && python > groovy, is(true));
+ // Console must be a text fence; python a python fence.
+ assertThat(md, containsString("```text\n"));
+ assertThat(md, containsString("```python\n"));
+ }
+
+ @Test
+ public void trailingCalloutBecomesInlineComment() {
+ final String md = renderer.render(Collections.singletonList(
+ NeutralTab.source("java", "g.V(1).out() <1>")));
+ assertThat(md, containsString("g.V(1).out() // (1)"));
+ assertThat(md, not(containsString("<1>")));
+ }
+
+ @Test
+ public void multipleTrailingCalloutsBecomeMultipleComments() {
+ final String md = renderer.render(Collections.singletonList(
+ NeutralTab.source("java", "g.V() <1> <2>")));
+ assertThat(md, containsString("g.V() // (1) // (2)"));
+ }
+
+ @Test
+ public void midLineAngleBracketsAreNotTreatedAsCallouts() {
+ final String md = renderer.render(Collections.singletonList(
+ NeutralTab.source("java", "List<String> x")));
+ assertThat(md, containsString("List<String> x"));
+ assertThat(md, not(containsString("// (")));
+ }
+
+ @Test
+ public void emptyContentProducesEmptyFenceBody() {
+ final String md = renderer.render(Collections.singletonList(
+ NeutralTab.console("groovy", "")));
+ assertThat(md, containsString("```text\n\n```"));
+ }
+
+ @Test
+ public void calloutConversionPerLine() {
+ final String md = renderer.render(Collections.singletonList(
+ NeutralTab.source("groovy", "a() <1>\nb()\nc() <2>")));
+ assertThat(md, containsString("a() // (1)\nb()\nc() // (2)"));
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/NeutralTabCodecTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/NeutralTabCodecTest.java
new file mode 100644
index 0000000000..5c48c72172
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/NeutralTabCodecTest.java
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Round-trip tests for {@link NeutralTabCodec}. The codec is the on-the-wire form of the neutral
+ * tab model, so lossless round-tripping is the correctness foundation for rendering both backends
+ * from a single Gremlin execution.
+ */
+public class NeutralTabCodecTest {
+
+ private static void assertRoundTrip(final List<NeutralTab> tabs) {
+ final String json = NeutralTabCodec.serialize(tabs);
+ final List<NeutralTab> parsed = NeutralTabCodec.parse(json);
+ assertThat(parsed.size(), is(tabs.size()));
+ for (int i = 0; i < tabs.size(); i++) {
+ final NeutralTab a = tabs.get(i);
+ final NeutralTab b = parsed.get(i);
+ assertThat(b.getLabel(), is(a.getLabel()));
+ assertThat(b.getLanguage(), is(a.getLanguage()));
+ assertThat(b.getKind(), is(a.getKind()));
+ assertThat(b.getContent(), is(a.getContent()));
+ }
+ }
+
+ @Test
+ public void shouldRoundTripEmptyList() {
+ assertRoundTrip(Collections.emptyList());
+ assertThat(NeutralTabCodec.serialize(Collections.emptyList()), is("[]"));
+ }
+
+ @Test
+ public void shouldRoundTripConsolePlusSource() {
+ assertRoundTrip(Arrays.asList(
+ NeutralTab.console("groovy", "gremlin> g.V(1)\n==>v[1]"),
+ NeutralTab.source("groovy", "g.V(1)")));
+ }
+
+ @Test
+ public void shouldRoundTripMultiLanguageTabs() {
+ assertRoundTrip(Arrays.asList(
+ NeutralTab.console("groovy", "gremlin> g.V()"),
+ NeutralTab.source("groovy", "g.V()"),
+ NeutralTab.source("java", "g.V()"),
+ NeutralTab.source("python", "g.V()")));
+ }
+
+ @Test
+ public void shouldRoundTripContentWithQuotesAndBackslashes() {
+ assertRoundTrip(Collections.singletonList(
+ NeutralTab.source("groovy", "x = \"a \\\"quoted\\\" path C:\\\\tmp\"")));
+ }
+
+ @Test
+ public void shouldRoundTripContentWithNewlinesTabsAndControlChars() {
+ assertRoundTrip(Collections.singletonList(
+ NeutralTab.console("groovy", "line1\n\tindented\r\nline3end")));
+ }
+
+ @Test
+ public void shouldRoundTripContentWithCalloutMarkers() {
+ assertRoundTrip(Collections.singletonList(
+ NeutralTab.source("java", "g.V() <1>\ng.E() <2> <3>")));
+ }
+
+ @Test
+ public void shouldRoundTripUnicodeContent() {
+ assertRoundTrip(Collections.singletonList(
+ NeutralTab.source("groovy", "name = 'Résumé — 日本語 ✓'")));
+ }
+
+ @Test
+ public void shouldProduceExpectedJsonShape() {
+ final String json = NeutralTabCodec.serialize(Collections.singletonList(
+ NeutralTab.source("java", "g.V()")));
+ assertThat(json, is("[{\"label\":\"java\",\"language\":\"java\",\"kind\":\"SOURCE\",\"content\":\"g.V()\"}]"));
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void shouldRejectMalformedJson() {
+ NeutralTabCodec.parse("[{\"label\":\"x\"}]");
+ }
+
+ @Test(expected = IllegalArgumentException.class)
+ public void shouldRejectNonArray() {
+ NeutralTabCodec.parse("{\"label\":\"x\"}");
+ }
+}
diff --git a/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/PostprocessorBackendTest.java b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/PostprocessorBackendTest.java
new file mode 100644
index 0000000000..b998f3fd64
--- /dev/null
+++ b/docs/tinkeradoc-extension/src/test/java/org/apache/tinkerpop/tinkeradoc/PostprocessorBackendTest.java
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.tinkerpop.tinkeradoc;
+
+import org.asciidoctor.Asciidoctor;
+import org.asciidoctor.Attributes;
+import org.asciidoctor.Options;
+import org.junit.Test;
+
+import static org.hamcrest.CoreMatchers.containsString;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.CoreMatchers.not;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+/**
+ * Verifies the postprocessor behaves correctly per backend: the {@code x.y.z} version placeholder
+ * is substituted in BOTH HTML and Markdown output, while the CodeRay empty-comment-span cleanup is
+ * applied only to HTML.
+ */
+public class PostprocessorBackendTest {
+
+ private String convert(final String backend) {
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.javaConverterRegistry().register(MarkdownConverter.class);
+ asciidoctor.javaExtensionRegistry().postprocessor(GremlinPostprocessor.class);
+ final String input = "= Test\n\nDownload version x.y.z from the repo.\n";
+ final Options options = Options.builder()
+ .backend(backend)
+ .attributes(Attributes.builder().attribute("tinkerpop-version", "3.7.7").build())
+ .build();
+ return asciidoctor.convert(input, options);
+ }
+ }
+
+ @Test
+ public void versionSubstitutedInHtml() {
+ final String html = convert("html5");
+ assertThat(html, containsString("3.7.7"));
+ assertThat(html, not(containsString("x.y.z")));
+ }
+
+ @Test
+ public void versionSubstitutedInMarkdown() {
+ final String md = convert("tpmarkdown");
+ assertThat(md, containsString("3.7.7"));
+ assertThat(md, not(containsString("x.y.z")));
+ }
+
+ @Test
+ public void codeRayCleanupSkippedForMarkdown() {
+ // A literal CodeRay empty-comment-span in Markdown content must be left untouched (the
+ // cleanup is an HTML artifact); in practice Markdown never contains these, but this pins
+ // the backend gate so the cleanup cannot silently corrupt Markdown.
+ try (final Asciidoctor asciidoctor = Asciidoctor.Factory.create()) {
+ asciidoctor.javaConverterRegistry().register(MarkdownConverter.class);
+ asciidoctor.javaExtensionRegistry().postprocessor(GremlinPostprocessor.class);
+ // A passthrough block carries the span verbatim into the output.
+ final String input = "= T\n\n++++\nkeep<span class=\"comment\">/* */</span>me\n++++\n";
+ final String md = asciidoctor.convert(input, Options.builder().backend("tpmarkdown").build());
+ assertThat(md, containsString("<span class=\"comment\">/* */</span>"));
+ }
+ }
+}
diff --git a/pom.xml b/pom.xml
index 6e33f515f0..30b97396d3 100644
--- a/pom.xml
+++ b/pom.xml
@@ -972,6 +972,10 @@ limitations under the License.
<!-- the output directory for the asciidoctor plugin - finalized documents can be found here -->
<htmlsingle.output.dir>${project.basedir}/target/docs/htmlsingle</htmlsingle.output.dir>
+
+ <!-- the output directory for the agent-friendly Markdown mirror (agentdocsspec.com),
+ rendered by the tpmarkdown backend from the same executed neutral tree -->
+ <markdown.output.dir>${project.basedir}/target/docs/markdown</markdown.output.dir>
<asciidoctor.style.dir>${asciidoc.input.dir}/stylesheets</asciidoctor.style.dir>
<!-- GremlinTreeprocessor attributes - set via CLI in bin/process-docs.sh -->
@@ -1251,6 +1255,34 @@ limitations under the License.
</attributes>
</configuration>
</execution>
+ <!-- Agent-friendly Markdown mirror of the io-book (agentdocsspec.com pilot).
+ Renders through the custom tpmarkdown backend from the SAME executed
+ neutral tree as the html5 execution above, so live Gremlin runs once
+ (see GremlinExecutionCache) and the executed output matches the HTML. -->
+ <execution>
+ <id>io-book-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/dev/io</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/dev/io</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>book</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
<execution>
<id>provider-book</id>
<phase>generate-resources</phase>
@@ -1431,6 +1463,249 @@ limitations under the License.
</attributes>
</configuration>
</execution>
+ <!-- Agent-friendly Markdown mirrors (agentdocsspec.com) for the remaining
+ books/tutorials; each renders via tpmarkdown from the same executed neutral tree
+ as its html5 counterpart above. -->
+ <execution>
+ <id>reference-book-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/reference</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/reference</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>book</doctype>
+ <attributes>
+ <imagesdir>../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>upgrade-book-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/upgrade</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/upgrade</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>book</doctype>
+ <attributes>
+ <imagesdir>../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>developer-book-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/dev/developer</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/dev/developer</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>book</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>future-book-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/dev/future</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/dev/future</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>book</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>provider-book-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/dev/provider</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/dev/provider</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>book</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>recipe-book-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/recipes</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/recipes</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>book</doctype>
+ <attributes>
+ <imagesdir>../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>tutorial-getting-started-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/tutorials/getting-started</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/tutorials/getting-started</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>article</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>tutorial-the-gremlin-console-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/tutorials/the-gremlin-console</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/tutorials/the-gremlin-console</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>article</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>tutorial-gremlin-language-variants-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/tutorials/gremlin-language-variants</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/tutorials/gremlin-language-variants</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>article</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
+ <execution>
+ <id>tutorial-gremlins-anatomy-markdown</id>
+ <phase>generate-resources</phase>
+ <goals>
+ <goal>process-asciidoc</goal>
+ </goals>
+ <configuration>
+ <sourceDirectory>${asciidoc.input.dir}/tutorials/gremlins-anatomy</sourceDirectory>
+ <sourceDocumentName>index.asciidoc</sourceDocumentName>
+ <outputDirectory>${markdown.output.dir}/tutorials/gremlins-anatomy</outputDirectory>
+ <backend>tpmarkdown</backend>
+ <doctype>article</doctype>
+ <attributes>
+ <imagesdir>../../images</imagesdir>
+ <encoding>UTF-8</encoding>
+ <outfilesuffix>.md</outfilesuffix>
+ <basedir>${project.basedir}</basedir>
+ <gremlin-docs-console-home>${gremlin.docs.console.home}</gremlin-docs-console-home>
+ <gremlin-docs-hadoop-libs>${gremlin.docs.hadoop.libs}</gremlin-docs-hadoop-libs>
+ <gremlin-docs-dryrun>${gremlin.docs.dryrun}</gremlin-docs-dryrun>
+ <tinkerpop-version>${project.version}</tinkerpop-version>
+ </attributes>
+ </configuration>
+ </execution>
</executions>
</plugin>
</plugins>