You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tooling.apache.org by ak...@apache.org on 2026/06/29 21:36:56 UTC
(tooling-agents) branch main updated (03fdf05 -> 437b335)
This is an automated email from the ASF dual-hosted git repository.
andrewmusselman pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-agents.git
from 03fdf05 Comment fix
new 5c12e53 asvs_orchestrate: restore issue/PR cross-reference wiring dropped in 7833bff
new 437b335 asvs_relevance_filter: match hyphenated and well-known security-doc names
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
ASVS/agents/asvs_orchestrate.py | 81 ++++++++++++++++++++++++++++++++++++
ASVS/agents/asvs_relevance_filter.py | 17 +++++++-
2 files changed, 96 insertions(+), 2 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
(tooling-agents) 02/02: asvs_relevance_filter: match hyphenated and well-known security-doc names
Posted by ak...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
andrewmusselman pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-agents.git
commit 437b3354983a6c0060483b65de388a3bc5fd1ef4
Author: Andrew Musselman <ak...@apache.org>
AuthorDate: Mon Jun 29 14:33:55 2026 -0700
asvs_relevance_filter: match hyphenated and well-known security-doc names
Discovery patterns and the _doc_priority tier-1 list were missing common
real-world naming variants, so a project's threat model could be skipped or
demoted purely by filename. Add (both to SOURCE_DOC_PATTERNS and tier-1):
- hyphenated threat models: threat-model.md/.rst, threat-modeling.md/.rst
(root and any depth) alongside the existing threatmodel / threat_model forms
- security-policy.md/.rst (root and any depth)
- RFC 9116 security.txt and .well-known/security.txt
Tier-2 substring match also broadened to catch threat-model. Matching is
lowercased on both sides, so case variants (THREAT_MODEL.md, SECURITY-POLICY.md)
are covered. This complements the link-follower (5ed3678): link-following
rescues docs a policy file points to, these patterns catch well-named docs
nobody links to. Names joined by hyphen, underscore, or concatenation are now
found and tier-1-promoted; a doc reachable by neither a pattern nor a link
remains out of scope (see the content-based-discovery follow-up.
---
ASVS/agents/asvs_relevance_filter.py | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/ASVS/agents/asvs_relevance_filter.py b/ASVS/agents/asvs_relevance_filter.py
index 09d2ec0..7ad1384 100644
--- a/ASVS/agents/asvs_relevance_filter.py
+++ b/ASVS/agents/asvs_relevance_filter.py
@@ -670,8 +670,18 @@ async def run(input_dict, tools):
"agents.md", "agents.rst",
"*/agents.md", "*/agents.rst",
"threatmodel.md", "threatmodel.rst", "threat_model.md", "threat_model.rst",
+ "threat-model.md", "threat-model.rst",
+ "threat-modeling.md", "threat-modeling.rst",
"*/threatmodel.md", "*/threatmodel.rst",
"*/threat_model.md", "*/threat_model.rst",
+ "*/threat-model.md", "*/threat-model.rst",
+ "*/threat-modeling.md", "*/threat-modeling.rst",
+ # security-policy named variants
+ "security-policy.md", "security-policy.rst",
+ "*/security-policy.md", "*/security-policy.rst",
+ # RFC 9116 well-known security contact/policy file
+ "security.txt", ".well-known/security.txt",
+ "*/.well-known/security.txt",
# docs/security tree — extensions required to skip
# __init__.py, helpers, etc. that happen to live alongside
"docs/security/*.md", "docs/security/*.rst",
@@ -753,11 +763,14 @@ async def run(input_dict, tools):
if basename in (
"agents.md", "agents.rst",
"security.md", "security.rst", "security",
- "threatmodel.md", "threat_model.md",
+ "security.txt", "security-policy.md", "security-policy.rst",
+ "threatmodel.md", "threat_model.md", "threat-model.md",
+ "threat-modeling.md",
):
return 1
# Tier 2 — documented security/threat models
- if "security_model" in full or "threatmodel" in full or "threat_model" in full:
+ if ("security_model" in full or "threatmodel" in full
+ or "threat_model" in full or "threat-model" in full):
return 2
# Tier 3 — sensitive-data / authn / authz / policies
if any(t in full for t in (
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org
(tooling-agents) 01/02: asvs_orchestrate: restore issue/PR cross-reference wiring dropped in 7833bff
Posted by ak...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
andrewmusselman pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tooling-agents.git
commit 5c12e53503f970f1a75d59e78f0542a2a83b11a4
Author: Andrew Musselman <ak...@apache.org>
AuthorDate: Mon Jun 29 14:33:29 2026 -0700
asvs_orchestrate: restore issue/PR cross-reference wiring dropped in 7833bff
The scans-style path + metadata.yml change (7833bff) inadvertently deleted
both cross-reference wiring blocks. They sat immediately adjacent to the two
regions that commit rewrote: the issue/PR snapshot fetch was right before the
old repo_path_segment line that the new path layout replaced, and the
compare-and-push block was right after Consolidation done where the metadata.yml
writer was inserted. Each edit swallowed one block as collateral, so runs since
7833bff produced no snapshot and no issues_cross_reference.md (caught on the
groovy run: no Issue/PR snapshot line in the log, grep -c asvs_fetch_issues_prs
returned 0).
Restores both blocks to their pre-7833bff form, keeping all of 7833bff's
path/metadata work:
- Fetch (after commit-hash resolution): snapshot open issues + PRs pinned to
commit_hash into issues_prs:{repo}@{sha}, with the issue/PR count logging.
- Compare (after consolidation, before the metadata writer): asvs_compare_open_issues_prs
returns report_md, which the orchestrator pushes as issues_cross_reference.md
via asvs_push_github, with the tracked/addressed/unaddressed summary logging.
Ordered before the metadata writer so the file exists when supporting_files
is enumerated, and sets cross_reference_done so that branch is no longer dead.
---
ASVS/agents/asvs_orchestrate.py | 81 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 81 insertions(+)
diff --git a/ASVS/agents/asvs_orchestrate.py b/ASVS/agents/asvs_orchestrate.py
index ae6b31f..de791a3 100644
--- a/ASVS/agents/asvs_orchestrate.py
+++ b/ASVS/agents/asvs_orchestrate.py
@@ -369,6 +369,35 @@ async def run(input_dict, tools):
source_id_path += f"/{source_path_prefix}"
source_id = f"{source_id_path} @ {commit_hash}"
+ # ---- Cross-reference: snapshot open issues & PRs (best-effort) ----
+ # Pin a snapshot of the repo's open issues and open PRs (with changed
+ # files) to this exact commit, stored in issues_prs:{repo}@{sha}, so
+ # the post-consolidation cross-reference compares findings against a
+ # fixed point (what was open as of commit_hash) rather than a moving
+ # branch. Keyed on the GitHub repo (repo_owner_name, no sub-path),
+ # which is also what the compare step reads. Best-effort: a failure
+ # here must not abort the audit.
+ try:
+ snap_resp = await gofannon_client.call(
+ agent_name="asvs_fetch_issues_prs",
+ input_dict={"inputText": json.dumps({
+ "repo": repo_owner_name,
+ "sha": commit_hash,
+ "token": source_token,
+ })},
+ )
+ _snap = json.loads(snap_resp.get("outputText", "{}"))
+ if "error" in _snap:
+ print(f" WARNING: issue/PR snapshot failed: {_snap['error']}", flush=True)
+ else:
+ _c = _snap.get("counts", {})
+ _done = "complete" if _snap.get("complete", True) else "INCOMPLETE"
+ print(f" Issue/PR snapshot ({_done}): {_c.get('issues','?')} issues, "
+ f"{_c.get('prs','?')} PRs @ {commit_hash}", flush=True)
+ except Exception as e:
+ print(f" WARNING: issue/PR snapshot call failed ({e}); "
+ f"cross-reference will be skipped", flush=True)
+
# Per-section reports go into CouchDB instead of GitHub. The
# previous approach committed each per-section report to the
# private repo, which surfaced one entry on the public
@@ -1379,6 +1408,58 @@ async def run(input_dict, tools):
else:
print(f" Consolidation done", flush=True)
+ # ---- Cross-reference findings vs open issues/PRs ------
+ # Reads the commit-pinned snapshot written before
+ # discovery and the structured findings persisted by
+ # consolidate; the compare agent RETURNS report_md and
+ # this orchestrator pushes it as issues_cross_reference.md.
+ # Always runs after a successful consolidation.
+ # Best-effort: never fail the run on it. Sets
+ # cross_reference_done so the metadata writer lists the
+ # file authoritatively.
+ cross_reference_done = False
+ try:
+ cmp_resp = await gofannon_client.call(
+ agent_name="asvs_compare_open_issues_prs",
+ input_dict={"inputText": json.dumps({
+ "repo": repo_owner_name,
+ "sha": commit_hash,
+ "reports_namespace": filtered_reports_namespace,
+ })},
+ )
+ cmp_out = json.loads(cmp_resp.get("outputText", "{}"))
+ if "error" in cmp_out:
+ print(f" Cross-reference skipped: {cmp_out['error']}", flush=True)
+ else:
+ s = cmp_out.get("summary", {})
+ print(f" Cross-reference: {s.get('tracked',0)} tracked, "
+ f"{s.get('addressed',0)} addressed by open PR, "
+ f"{s.get('unaddressed',0)} unaddressed "
+ f"(of {s.get('findings',0)})", flush=True)
+ report_md = cmp_out.get("report_md") or ""
+ if report_md:
+ try:
+ await gofannon_client.call(
+ agent_name="asvs_push_github",
+ input_dict={
+ "inputText": json.dumps({
+ "token": output_token,
+ "repo": output_repo,
+ "directory": push_directory,
+ "fileName": "issues_cross_reference.md",
+ }),
+ "fileContents": report_md,
+ "commitMessage": f"Add issue/PR cross-reference [{source_id}]",
+ },
+ )
+ cross_reference_done = True
+ print(f" Cross-reference report pushed", flush=True)
+ except Exception as _pe:
+ print(f" WARNING: could not push cross-reference report: {_pe}", flush=True)
+ except Exception as e:
+ print(f" WARNING: cross-reference call failed ({e})", flush=True)
+
+
# ---- Write metadata.yml into the leaf dir -------------
# Mirrors the scans-style manifest: project/repo/sha/date,
# models, asvs level, threat-model URL, findings_total, and
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@tooling.apache.org
For additional commands, e-mail: commits-help@tooling.apache.org