(apisix-website) 01/01: fix(llms): keep the curated index, put the generated one after it

[we...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

moonming pushed a commit to branch fix/llms-keep-curated
in repository https://gitbox.apache.org/repos/asf/apisix-website.git

commit 76ac1a70fab7795970573e7745194b383a448816
Author: Ming Wen <we...@apache.org>
AuthorDate: Tue Jul 28 10:26:01 2026 +0800

    fix(llms): keep the curated index, put the generated one after it
    
    The site already ships a hand-written llms.txt (website/static/llms.txt):
    ~48 entries grouped by topic — Core Documentation, Authentication &
    Security, Traffic Control, AI Gateway, … — each with a sentence on what it
    is for. The generated index was silently replacing it during the overlay,
    trading that editorial judgement for a flat list of 1,186 URLs.
    
    An agent gets more from "here are the twenty pages that matter, and what
    each is for" than from everything in alphabetical order. So the curated
    list now leads, and the generated full index follows under its own
    heading. The curated file stays the place to edit the front section.
---
 next/scripts/generate-md-twins.mjs | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/next/scripts/generate-md-twins.mjs b/next/scripts/generate-md-twins.mjs
index 3d686917866..5eedba7a15d 100644
--- a/next/scripts/generate-md-twins.mjs
+++ b/next/scripts/generate-md-twins.mjs
@@ -150,6 +150,20 @@ const group = (items, frag) => {
   return items.filter((p) => prefix.test(p.url));
 };
 
+/**
+ * The site ships a hand-curated llms.txt (website/static/llms.txt): ~48 entries
+ * grouped by topic, each with a sentence on what it is for. That editorial
+ * judgement is worth more to an agent than a flat list, so it stays at the top
+ * — trimmed of its own heading — and the generated full index follows it.
+ */
+const curatedFile = path.join(root, '..', 'website', 'static', 'llms.txt');
+const curated = fs.existsSync(curatedFile)
+  ? fs.readFileSync(curatedFile, 'utf8')
+    .replace(/^#[^\n]*\n+/, '')            // drop its H1; ours is the page title
+    .replace(/^>.*\n+/m, '')               // and its blurb; ours carries one
+    .trim()
+  : '';
+
 const llms = [
   '# Apache APISIX',
   '',
@@ -157,6 +171,8 @@ const llms = [
   '',
   'Every page below is available as Markdown — append `index.md` to any page URL.',
   '',
+  ...(curated ? [curated, ''] : []),
+  ...(curated ? ['# Full index', ''] : []),
   ...section('Documentation', group(en, '/docs/')),
   ...section('Learning center', group(en, '/learning-center/')),
   ...section('Blog', group(en, '/blog/')),