You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "rob-9 (via GitHub)" <gi...@apache.org> on 2026/08/12 19:11:35 UTC

[PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

rob-9 opened a new pull request, #1005:
URL: https://github.com/apache/flink-agents/pull/1005

   ### Purpose of change
   
   Closes #1003.
   
   Remote skill archives are trusted instructions and code: their `SKILL.md` content is consumed by the agent, and bundled scripts can become executable through the built-in tools. Until now, `Skills.fromUrl(...)` / `Skills.from_url(...)` accepted plain HTTP and extracted whatever arrived, without verifying an operator-provided integrity expectation.
   
   This change makes URL skill sources secure by default and adds optional integrity verification across the Java, Python, and YAML surfaces.
   
   - Reject plain HTTP URL skill sources by default; HTTPS is required.
   - Keep plain HTTP available only through explicitly named unsafe factory methods or the `allow_insecure_http` YAML option.
   - Let operators pin an expected SHA-256 digest, which is verified against the downloaded archive before extraction.
   - Enforce the transport and digest rules again at runtime, so sources arriving through plan serialization or YAML cannot bypass API-level validation.
   - Fail with a clear error on unsupported cross-protocol redirects and clean up temporary archives on every failure path.
   - Add a structured YAML `url_sources` list while preserving the existing `urls` shorthand for unpinned HTTPS sources.
   - Keep Java and Python aligned on configuration, security behavior, tests, documentation, and generated YAML schemas.
   
   Existing direct and same-protocol-redirecting HTTPS sources are unaffected. Existing plain HTTP sources must now opt in explicitly.
   
   ### Tests
   
   - Targeted Java API tests: 31 passed.
   - Targeted Java runtime tests: 38 passed.
   - Python tests covering the changed API, YAML package, and runtime skill paths: 151 passed.
   - Java/Python agent-plan cross-language tests: 9 passed, 1 skipped.
   - Java Spotless check passed.
   - Ruff passed on the changed code; the six remaining reported violations are pre-existing and unrelated. 
   - YAML schema consistency verified, including both generated schema copies and the updated contract blob hash.
   - Apache RAT license check passed.
   
   ### API
   
   Yes. The Java and Python skill factories now reject plain HTTP by default and add aligned methods for SHA-256 pinning and explicit HTTP opt-in:
   
   - Java: `fromUrlWithSha256`, `fromUrlUnsafe`, and `fromUrlUnsafeWithSha256`.
   - Python: `from_url_with_sha256`, `from_url_unsafe`, and `from_url_unsafe_with_sha256`.
   
   YAML gains a structured `url_sources` list whose entries accept `url`, optional `sha256`, and optional `allow_insecure_http`. The existing `urls` list remains available for ordinary HTTPS sources.
   
   ### Documentation
   
   - [ ] `doc-needed`
   - [ ] `doc-not-needed`
   - [x] `doc-included`
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   - [x] Yes
   - [ ] No
   
   Generated-by: OpenAI Codex (GPT-5)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3798482243


##########
api/src/main/java/org/apache/flink/agents/api/yaml/YamlLoader.java:
##########
@@ -173,8 +174,19 @@ public static Skills buildSkills(SkillsSpec spec) {
         for (String p : spec.getPaths()) {
             sources.add(new SkillSourceSpec("local", Map.of("path", p)));
         }
-        for (String u : spec.getUrls()) {
-            sources.add(new SkillSourceSpec("url", Map.of("url", u)));
+        for (String url : spec.getUrls()) {
+            sources.add(new SkillSourceSpec("url", Map.of("url", url)));

Review Comment:
   it wasn't deliberate. both YAML loaders now use the validated URL factories, so plain HTTP and malformed digests fail during loading while explicit HTTP opt-in remains supported.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#issuecomment-5471321328

   addressed comments and made another review pass.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3912223371


##########
api/src/main/java/org/apache/flink/agents/api/skills/SkillUrlUtils.java:
##########
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.agents.api.skills;
+
+import org.apache.flink.annotation.Internal;
+
+import java.net.MalformedURLException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.util.Locale;
+import java.util.regex.Pattern;
+
+/**
+ * Shared validation and redaction helpers for URL-backed skill sources. Internal contract shared
+ * with the runtime module; not a stable public API.
+ */
+@Internal
+public final class SkillUrlUtils {
+
+    private static final String REDACTED = "<redacted>";
+    private static final Pattern INVALID_PERCENT_ESCAPE = Pattern.compile("%(?![0-9a-fA-F]{2})");
+
+    private SkillUrlUtils() {}
+
+    /**
+     * Validate {@code url} and return its lowercase {@code http} or {@code https} scheme.
+     *
+     * @throws IllegalArgumentException if the URL is invalid or violates the transport policy.
+     */
+    public static String validate(String url, boolean allowInsecureHttp) {
+        if (url == null) {
+            throw new IllegalArgumentException("skill URL must not be null");
+        }
+        if (INVALID_PERCENT_ESCAPE.matcher(url).find()) {
+            throw new IllegalArgumentException("Invalid skill URL: " + redact(url));
+        }
+        URI uri;
+        try {
+            uri = URI.create(url);
+        } catch (IllegalArgumentException ignored) {
+            throw new IllegalArgumentException("Invalid skill URL: " + redact(url));
+        }
+        String scheme = uri.getScheme();
+        scheme = scheme == null ? "" : scheme.toLowerCase(Locale.ROOT);
+        if (!(scheme.equals("http") || scheme.equals("https"))) {
+            throw new IllegalArgumentException(
+                    "Only HTTP(S) skill URLs are supported: " + redact(url));
+        }
+        try {
+            uri = uri.parseServerAuthority();
+        } catch (URISyntaxException ignored) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host and, when present, a valid port: "
+                            + redact(url));
+        }
+        if (uri.getRawUserInfo() != null) {
+            throw new IllegalArgumentException(
+                    "Skill URL must not include user info: " + redact(url));
+        }
+        if (uri.getHost() == null || uri.getHost().isEmpty()) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host: " + redact(url));
+        }
+        if (uri.getPort() > 65535) {
+            throw new IllegalArgumentException(
+                    "Skill URL port must be between 0 and 65535: " + redact(url));
+        }
+        if (scheme.equals("http") && !allowInsecureHttp) {
+            throw new IllegalArgumentException(
+                    "Plain HTTP skill URLs are disabled by default; use HTTPS or explicitly allow"
+                            + " insecure HTTP for this source: "
+                            + redact(url));
+        }
+        return scheme;
+    }
+
+    /** Return {@code url} without user info, query parameters, or a fragment. */
+    public static String redact(String url) {
+        if (url == null) {
+            return REDACTED;
+        }
+        try {
+            URI uri = URI.create(url);
+            if (uri.getScheme() == null || uri.getRawAuthority() == null) {
+                return REDACTED;
+            }
+            return redactParts(uri.getScheme(), uri.getRawAuthority(), uri.getRawPath());
+        } catch (IllegalArgumentException ignored) {
+            try {
+                URL parsed = new URL(url);
+                String authority = parsed.getAuthority();
+                String path = parsed.getPath();
+                if (authority == null
+                        || containsUnsafeLogCharacter(authority)
+                        || containsUnsafeLogCharacter(path)) {
+                    return REDACTED;
+                }
+                return redactParts(parsed.getProtocol(), authority, path);
+            } catch (MalformedURLException | IllegalArgumentException malformed) {
+                return REDACTED;
+            }
+        }
+    }
+
+    private static String redactParts(String scheme, String authority, String path) {
+        int userInfoEnd = authority.lastIndexOf('@');
+        if (userInfoEnd >= 0) {
+            authority = authority.substring(userInfoEnd + 1);
+        }
+        if (authority.isEmpty()) {
+            return REDACTED;
+        }
+        return scheme + "://" + authority + (path == null ? "" : path);

Review Comment:
   One redaction edge case remains: `https://user:supersecret/x.zip?token=TOPSECRET` fails port validation, but the error still contains `https://user:supersecret/x.zip`. Since there is no `@`, both redactors preserve the entire malformed authority. Could invalid-port authorities return `<redacted>` or omit the raw port, with coverage in both Java and Python?



##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -253,23 +257,118 @@ private static void copyJarEntries(URL jarUrl, String prefix, Path extractDir)
      * @throws IOException on connect / read failures or HTTP error responses.
      */
     public static Path downloadToTempFile(String url, int timeoutMs) throws IOException {
-        URL u = new URL(url);
-        HttpURLConnection conn = (HttpURLConnection) u.openConnection();
-        conn.setConnectTimeout(timeoutMs);
-        conn.setReadTimeout(timeoutMs);
-        conn.setRequestMethod("GET");
+        return downloadToTempFile(url, timeoutMs, false);
+    }
+
+    /**
+     * Download {@code url}, optionally permitting plain HTTP transport.
+     *
+     * @throws IOException on connect / read failures or HTTP error responses.
+     */
+    public static Path downloadToTempFile(String url, int timeoutMs, boolean allowInsecureHttp)
+            throws IOException {
+        URL u;
+        try {
+            u = new URL(url);
+        } catch (MalformedURLException ignored) {
+            throw new IOException("Invalid skill URL: " + SkillUrlUtils.redact(url));
+        }
+        String initialProtocol = requireValidDownloadUrl(u, allowInsecureHttp);
+        boolean followRedirects = HttpURLConnection.getFollowRedirects();
         Path tmpZip = Files.createTempFile(TEMP_DIR_PREFIX, ".zip");
-        try (InputStream in = conn.getInputStream()) {
-            Files.copy(in, tmpZip, StandardCopyOption.REPLACE_EXISTING);
+        HttpURLConnection conn = null;
+        try {
+            URL effectiveUrl = u;
+            int redirects = 0;
+            while (true) {
+                conn = (HttpURLConnection) effectiveUrl.openConnection();

Review Comment:
   Scoped IPv6 URLs accepted by the validator do not work with the Java downloader. On Temurin 11, `%25lo0` passes validation but `HttpURLConnection` treats the scope as `25lo0` and raises `UnknownHostException`; raw `%lo0` works with `HttpURLConnection` but is rejected by the validator. Could we either normalize the zone delimiter before downloading or reject scoped IPv6 URLs in both languages, with an end-to-end test?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3798478723


##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -254,14 +254,37 @@ private static void copyJarEntries(URL jarUrl, String prefix, Path extractDir)
      * @throws IOException on connect / read failures or HTTP error responses.
      */
     public static Path downloadToTempFile(String url, int timeoutMs) throws IOException {
+        return downloadToTempFile(url, timeoutMs, false);
+    }
+
+    /**
+     * Download {@code url}, optionally permitting plain HTTP transport.
+     *
+     * @throws IOException on connect / read failures or HTTP error responses.
+     */
+    public static Path downloadToTempFile(String url, int timeoutMs, boolean allowInsecureHttp)
+            throws IOException {
         URL u = new URL(url);
+        String initialProtocol = u.getProtocol();
+        if (!("https".equalsIgnoreCase(initialProtocol)
+                || (allowInsecureHttp && "http".equalsIgnoreCase(initialProtocol)))) {
+            throw new IOException("Skill URL uses a disallowed transport: " + url);
+        }
         HttpURLConnection conn = (HttpURLConnection) u.openConnection();
         conn.setConnectTimeout(timeoutMs);
         conn.setReadTimeout(timeoutMs);
         conn.setRequestMethod("GET");
         Path tmpZip = Files.createTempFile(TEMP_DIR_PREFIX, ".zip");
-        try (InputStream in = conn.getInputStream()) {
-            Files.copy(in, tmpZip, StandardCopyOption.REPLACE_EXISTING);
+        try {
+            int responseCode = conn.getResponseCode();
+            if (responseCode >= 300 && responseCode < 400) {

Review Comment:
   pinning remains the integrity control, but surfacing redirects is useful too. added URL logging in both runtimes and documented Java's cross-protocol behavior,



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#issuecomment-5422825567

   fixed conflicts, should be good to go


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3930676062


##########
api/src/test/java/org/apache/flink/agents/api/skills/SkillsResourceTest.java:
##########
@@ -48,6 +53,224 @@ void fromUrlEmitsUrlScheme() {
                 skills.getSources());
     }
 
+    @Test
+    void fromUrlAcceptsSharedValidHostSyntax() {
+        for (String url :
+                List.of(
+                        "https://localhost/x.zip",
+                        "https://127.0.0.1/x.zip",
+                        "https://[::1]/x.zip",
+                        "https://example.com./x.zip",
+                        "https://example.com:/x.zip",
+                        "https://example.com:65535/x.zip",
+                        "https://999/x.zip",
+                        "https://1bar/x.zip",
+                        "https://999./x.zip")) {
+            assertEquals(url, Skills.fromUrl(url).getSources().get(0).getParams().get("url"));
+        }
+    }
+
+    @Test
+    void fromUrlRejectsScopedIpv6() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> Skills.fromUrl("https://[fe80::1%25lo0]/x.zip"));
+        assertTrue(ex.getMessage().contains("must not include an IPv6 zone identifier"));
+        assertNull(ex.getCause());
+    }
+
+    @Test
+    void fromUrlWithSha256EmitsIntegrityParam() {
+        String digest = "A".repeat(64);
+        Skills skills = Skills.fromUrlWithSha256("https://example.com/x.zip", digest);
+        assertEquals(
+                List.of(
+                        new SkillSourceSpec(
+                                "url",
+                                Map.of("url", "https://example.com/x.zip", "sha256", digest))),
+                skills.getSources());
+    }
+
+    @Test
+    void fromUrlUnsafeRequiresExplicitParam() {
+        Skills skills = Skills.fromUrlUnsafe("http://example.com/x.zip");
+        assertEquals("true", skills.getSources().get(0).getParams().get("allow_insecure_http"));
+    }
+
+    @Test
+    void fromUrlUnsafeWithSha256EmitsBothParams() {
+        String digest = "a".repeat(64);
+        Skills skills = Skills.fromUrlUnsafeWithSha256("http://example.com/x.zip", digest);
+        assertEquals(
+                Map.of(
+                        "url",
+                        "http://example.com/x.zip",
+                        "sha256",
+                        digest,
+                        "allow_insecure_http",
+                        "true"),
+                skills.getSources().get(0).getParams());
+    }
+
+    @Test
+    void fromUrlRejectsPlainHttpByDefault() {
+        assertThrows(
+                IllegalArgumentException.class, () -> Skills.fromUrl("http://example.com/x.zip"));
+    }
+
+    @Test
+    void fromUrlWithSha256RejectsMalformedDigest() {
+        assertThrows(
+                IllegalArgumentException.class,
+                () -> Skills.fromUrlWithSha256("https://example.com/x.zip", "invalid"));
+    }
+
+    @Test
+    void fromUrlRejectsUnsupportedSchemeClearly() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () ->
+                                Skills.fromUrl(
+                                        "ftp://user:password@example.com/x.zip?token=secret#part"));
+        assertEquals(
+                "Only HTTP(S) skill URLs are supported: ftp://example.com/x.zip", ex.getMessage());
+        assertFalse(ex.getMessage().contains("password"));
+        assertFalse(ex.getMessage().contains("secret"));
+    }
+
+    @Test
+    void fromUrlRejectsMalformedUrl() {
+        IllegalArgumentException unparseable =
+                assertThrows(
+                        IllegalArgumentException.class, () -> Skills.fromUrl("https://[::1/x.zip"));
+        assertEquals("Invalid skill URL: <redacted>", unparseable.getMessage());
+
+        IllegalArgumentException unsafeForLogs =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> Skills.fromUrl("https://u:pw@example.com/a\nheader?token=SECRET"));
+        assertEquals("Invalid skill URL: <redacted>", unsafeForLogs.getMessage());
+        assertFalse(unsafeForLogs.getMessage().contains("pw"));
+        assertFalse(unsafeForLogs.getMessage().contains("SECRET"));
+
+        for (String url :
+                List.of("https://example.com/skills[1].zip", "https://[fe80::1%eth0]/x.zip")) {
+            IllegalArgumentException ex =
+                    assertThrows(IllegalArgumentException.class, () -> Skills.fromUrl(url), url);
+            assertEquals("Invalid skill URL: " + url, ex.getMessage());
+        }
+    }
+
+    @Test
+    void fromUrlRedactsMalformedUrlWithoutLosingSafeContext() {
+        for (String path : List.of("a b.zip", "%zz")) {
+            IllegalArgumentException ex =
+                    assertThrows(
+                            IllegalArgumentException.class,
+                            () ->
+                                    Skills.fromUrl(
+                                            "https://u:pw@example.com/"
+                                                    + path
+                                                    + "?token=SECRET#fragment"));
+            assertEquals("Invalid skill URL: https://example.com/" + path, ex.getMessage());
+            assertFalse(ex.getMessage().contains("pw"));
+            assertFalse(ex.getMessage().contains("SECRET"));
+        }
+    }
+
+    @Test
+    void fromUrlAcceptsBracketsOutsideRawPath() {
+        for (String url :
+                List.of(
+                        "https://example.com/skills%5B1%5D.zip",
+                        "https://example.com/x.zip?a[0]=1", "https://example.com/x.zip#f[1]")) {
+            assertEquals(url, Skills.fromUrl(url).getSources().get(0).getParams().get("url"));
+        }
+    }
+
+    @Test
+    void fromUrlRejectsInvalidHostAndPort() {
+        for (String url :
+                List.of(
+                        "https://example.com:bad/x.zip?token=top-secret",
+                        "https://user:supersecret/x.zip?token=TOPSECRET")) {

Review Comment:
   Added both cases to `fromUrlRejectsInvalidHostAndPort`, covering authorities with multiple colons and ports above 65535. The existing `<redacted>` assertion now catches either branch regressing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3911067878


##########
python/flink_agents/api/skills.py:
##########
@@ -57,13 +57,118 @@ def packaged_skills() -> Skills:
 
 from __future__ import annotations
 
+import re
+from ipaddress import AddressValueError, IPv6Address
 from typing import Dict, List, Tuple
+from urllib.parse import urlparse, urlsplit, urlunsplit
 
 from pydantic import BaseModel, ConfigDict, Field, field_validator
 from typing_extensions import override
 
 from flink_agents.api.resource import ResourceType, SerializableResource
 
+_INVALID_URI_CHARACTER = re.compile(r'[\x00-\x20\x7f<>"{}|\\^`]')
+_INVALID_PERCENT_ESCAPE = re.compile(r"%(?![0-9a-fA-F]{2})")
+_HOST_LABEL = re.compile(r"[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?")
+
+
+def redact_skill_url(url: str) -> str:
+    """Return a skill URL without user info, query parameters, or a fragment.
+
+    Internal contract shared with the runtime; not a stable public API.
+    """
+    try:
+        parts = urlsplit(url)
+        if not parts.scheme or not parts.netloc:
+            return "<redacted>"
+        netloc = parts.netloc.rsplit("@", 1)[-1]
+        if not netloc:
+            return "<redacted>"
+        return urlunsplit((parts.scheme, netloc, parts.path, "", ""))

Review Comment:
   nit: `redact_skill_url` still lets a raw control character through, while the new Java guard now blocks it.
   
   `https://u:pw@example.com/a\x1b[31mred?token=SECRET`:
   
   - Java: `Invalid skill URL: <redacted>`
   - Python: `Invalid skill URL: https://example.com/a\x1b[31mred`
   
   Small either way. Neither side leaks the password or the token, and CR/LF cannot get through on Python because `urlsplit` strips them first. So it is an ANSI escape in an exception message, not log line forging.
   
   The Java half is `containsUnsafeLogCharacter` (`SkillUrlUtils.java:133`), pinned by `SkillsResourceTest.java:144`. Worth mirroring here, or is the Java-only guard deliberate?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3930495872


##########
api/src/test/java/org/apache/flink/agents/api/skills/SkillsResourceTest.java:
##########
@@ -48,6 +53,224 @@ void fromUrlEmitsUrlScheme() {
                 skills.getSources());
     }
 
+    @Test
+    void fromUrlAcceptsSharedValidHostSyntax() {
+        for (String url :
+                List.of(
+                        "https://localhost/x.zip",
+                        "https://127.0.0.1/x.zip",
+                        "https://[::1]/x.zip",
+                        "https://example.com./x.zip",
+                        "https://example.com:/x.zip",
+                        "https://example.com:65535/x.zip",
+                        "https://999/x.zip",
+                        "https://1bar/x.zip",
+                        "https://999./x.zip")) {
+            assertEquals(url, Skills.fromUrl(url).getSources().get(0).getParams().get("url"));
+        }
+    }
+
+    @Test
+    void fromUrlRejectsScopedIpv6() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> Skills.fromUrl("https://[fe80::1%25lo0]/x.zip"));
+        assertTrue(ex.getMessage().contains("must not include an IPv6 zone identifier"));
+        assertNull(ex.getCause());
+    }
+
+    @Test
+    void fromUrlWithSha256EmitsIntegrityParam() {
+        String digest = "A".repeat(64);
+        Skills skills = Skills.fromUrlWithSha256("https://example.com/x.zip", digest);
+        assertEquals(
+                List.of(
+                        new SkillSourceSpec(
+                                "url",
+                                Map.of("url", "https://example.com/x.zip", "sha256", digest))),
+                skills.getSources());
+    }
+
+    @Test
+    void fromUrlUnsafeRequiresExplicitParam() {
+        Skills skills = Skills.fromUrlUnsafe("http://example.com/x.zip");
+        assertEquals("true", skills.getSources().get(0).getParams().get("allow_insecure_http"));
+    }
+
+    @Test
+    void fromUrlUnsafeWithSha256EmitsBothParams() {
+        String digest = "a".repeat(64);
+        Skills skills = Skills.fromUrlUnsafeWithSha256("http://example.com/x.zip", digest);
+        assertEquals(
+                Map.of(
+                        "url",
+                        "http://example.com/x.zip",
+                        "sha256",
+                        digest,
+                        "allow_insecure_http",
+                        "true"),
+                skills.getSources().get(0).getParams());
+    }
+
+    @Test
+    void fromUrlRejectsPlainHttpByDefault() {
+        assertThrows(
+                IllegalArgumentException.class, () -> Skills.fromUrl("http://example.com/x.zip"));
+    }
+
+    @Test
+    void fromUrlWithSha256RejectsMalformedDigest() {
+        assertThrows(
+                IllegalArgumentException.class,
+                () -> Skills.fromUrlWithSha256("https://example.com/x.zip", "invalid"));
+    }
+
+    @Test
+    void fromUrlRejectsUnsupportedSchemeClearly() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () ->
+                                Skills.fromUrl(
+                                        "ftp://user:password@example.com/x.zip?token=secret#part"));
+        assertEquals(
+                "Only HTTP(S) skill URLs are supported: ftp://example.com/x.zip", ex.getMessage());
+        assertFalse(ex.getMessage().contains("password"));
+        assertFalse(ex.getMessage().contains("secret"));
+    }
+
+    @Test
+    void fromUrlRejectsMalformedUrl() {
+        IllegalArgumentException unparseable =
+                assertThrows(
+                        IllegalArgumentException.class, () -> Skills.fromUrl("https://[::1/x.zip"));
+        assertEquals("Invalid skill URL: <redacted>", unparseable.getMessage());
+
+        IllegalArgumentException unsafeForLogs =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> Skills.fromUrl("https://u:pw@example.com/a\nheader?token=SECRET"));
+        assertEquals("Invalid skill URL: <redacted>", unsafeForLogs.getMessage());
+        assertFalse(unsafeForLogs.getMessage().contains("pw"));
+        assertFalse(unsafeForLogs.getMessage().contains("SECRET"));
+
+        for (String url :
+                List.of("https://example.com/skills[1].zip", "https://[fe80::1%eth0]/x.zip")) {
+            IllegalArgumentException ex =
+                    assertThrows(IllegalArgumentException.class, () -> Skills.fromUrl(url), url);
+            assertEquals("Invalid skill URL: " + url, ex.getMessage());
+        }
+    }
+
+    @Test
+    void fromUrlRedactsMalformedUrlWithoutLosingSafeContext() {
+        for (String path : List.of("a b.zip", "%zz")) {
+            IllegalArgumentException ex =
+                    assertThrows(
+                            IllegalArgumentException.class,
+                            () ->
+                                    Skills.fromUrl(
+                                            "https://u:pw@example.com/"
+                                                    + path
+                                                    + "?token=SECRET#fragment"));
+            assertEquals("Invalid skill URL: https://example.com/" + path, ex.getMessage());
+            assertFalse(ex.getMessage().contains("pw"));
+            assertFalse(ex.getMessage().contains("SECRET"));
+        }
+    }
+
+    @Test
+    void fromUrlAcceptsBracketsOutsideRawPath() {
+        for (String url :
+                List.of(
+                        "https://example.com/skills%5B1%5D.zip",
+                        "https://example.com/x.zip?a[0]=1", "https://example.com/x.zip#f[1]")) {
+            assertEquals(url, Skills.fromUrl(url).getSources().get(0).getParams().get("url"));
+        }
+    }
+
+    @Test
+    void fromUrlRejectsInvalidHostAndPort() {
+        for (String url :
+                List.of(
+                        "https://example.com:bad/x.zip?token=top-secret",
+                        "https://user:supersecret/x.zip?token=TOPSECRET")) {

Review Comment:
   The behaviour here is right, so this is only about test coverage.
   
   `hasInvalidPort` is new hand-written parsing with several ways to return true, and this list covers one of them. I flipped these two return values one at a time on a private copy and ran api + plan + runtime, 1387 tests. Each flip stays green and puts a credential-shaped authority back into a user-facing message:
   
   - `SkillUrlUtils.java:158`, more than one colon: `Skill URL must include a valid host and, when present, a valid port: https://user:sup:ersecret/x.zip`
   - `SkillUrlUtils.java:173`, port above 65535: `Skill URL port must be between 0 and 65535: https://user:999999/x.zip`
   
   Two more entries in this list catch both, since the `endsWith("<redacted>")` assertion already does the work:
   
   ```java
   List.of(
           "https://example.com:bad/x.zip?token=top-secret",
           "https://user:supersecret/x.zip?token=TOPSECRET",
           "https://user:sup:ersecret/x.zip?token=TOPSECRET",
           "https://user:999999/x.zip?token=TOPSECRET")) {
   ```
   
   Green on the real source, red under each flip when I tried it. Python looks fine here, because `_ = parts.port` is one statement and three tests already cover it.
   
   Worth adding, or are these covered somewhere I missed?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3917508201


##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -253,23 +257,118 @@ private static void copyJarEntries(URL jarUrl, String prefix, Path extractDir)
      * @throws IOException on connect / read failures or HTTP error responses.
      */
     public static Path downloadToTempFile(String url, int timeoutMs) throws IOException {
-        URL u = new URL(url);
-        HttpURLConnection conn = (HttpURLConnection) u.openConnection();
-        conn.setConnectTimeout(timeoutMs);
-        conn.setReadTimeout(timeoutMs);
-        conn.setRequestMethod("GET");
+        return downloadToTempFile(url, timeoutMs, false);
+    }
+
+    /**
+     * Download {@code url}, optionally permitting plain HTTP transport.
+     *
+     * @throws IOException on connect / read failures or HTTP error responses.
+     */
+    public static Path downloadToTempFile(String url, int timeoutMs, boolean allowInsecureHttp)
+            throws IOException {
+        URL u;
+        try {
+            u = new URL(url);
+        } catch (MalformedURLException ignored) {
+            throw new IOException("Invalid skill URL: " + SkillUrlUtils.redact(url));
+        }
+        String initialProtocol = requireValidDownloadUrl(u, allowInsecureHttp);
+        boolean followRedirects = HttpURLConnection.getFollowRedirects();
         Path tmpZip = Files.createTempFile(TEMP_DIR_PREFIX, ".zip");
-        try (InputStream in = conn.getInputStream()) {
-            Files.copy(in, tmpZip, StandardCopyOption.REPLACE_EXISTING);
+        HttpURLConnection conn = null;
+        try {
+            URL effectiveUrl = u;
+            int redirects = 0;
+            while (true) {
+                conn = (HttpURLConnection) effectiveUrl.openConnection();

Review Comment:
   Going to reject scoped IPv6 URLs consistently. 



##########
api/src/main/java/org/apache/flink/agents/api/skills/SkillUrlUtils.java:
##########
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.agents.api.skills;
+
+import org.apache.flink.annotation.Internal;
+
+import java.net.MalformedURLException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.util.Locale;
+import java.util.regex.Pattern;
+
+/**
+ * Shared validation and redaction helpers for URL-backed skill sources. Internal contract shared
+ * with the runtime module; not a stable public API.
+ */
+@Internal
+public final class SkillUrlUtils {
+
+    private static final String REDACTED = "<redacted>";
+    private static final Pattern INVALID_PERCENT_ESCAPE = Pattern.compile("%(?![0-9a-fA-F]{2})");
+
+    private SkillUrlUtils() {}
+
+    /**
+     * Validate {@code url} and return its lowercase {@code http} or {@code https} scheme.
+     *
+     * @throws IllegalArgumentException if the URL is invalid or violates the transport policy.
+     */
+    public static String validate(String url, boolean allowInsecureHttp) {
+        if (url == null) {
+            throw new IllegalArgumentException("skill URL must not be null");
+        }
+        if (INVALID_PERCENT_ESCAPE.matcher(url).find()) {
+            throw new IllegalArgumentException("Invalid skill URL: " + redact(url));
+        }
+        URI uri;
+        try {
+            uri = URI.create(url);
+        } catch (IllegalArgumentException ignored) {
+            throw new IllegalArgumentException("Invalid skill URL: " + redact(url));
+        }
+        String scheme = uri.getScheme();
+        scheme = scheme == null ? "" : scheme.toLowerCase(Locale.ROOT);
+        if (!(scheme.equals("http") || scheme.equals("https"))) {
+            throw new IllegalArgumentException(
+                    "Only HTTP(S) skill URLs are supported: " + redact(url));
+        }
+        try {
+            uri = uri.parseServerAuthority();
+        } catch (URISyntaxException ignored) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host and, when present, a valid port: "
+                            + redact(url));
+        }
+        if (uri.getRawUserInfo() != null) {
+            throw new IllegalArgumentException(
+                    "Skill URL must not include user info: " + redact(url));
+        }
+        if (uri.getHost() == null || uri.getHost().isEmpty()) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host: " + redact(url));
+        }
+        if (uri.getPort() > 65535) {
+            throw new IllegalArgumentException(
+                    "Skill URL port must be between 0 and 65535: " + redact(url));
+        }
+        if (scheme.equals("http") && !allowInsecureHttp) {
+            throw new IllegalArgumentException(
+                    "Plain HTTP skill URLs are disabled by default; use HTTPS or explicitly allow"
+                            + " insecure HTTP for this source: "
+                            + redact(url));
+        }
+        return scheme;
+    }
+
+    /** Return {@code url} without user info, query parameters, or a fragment. */
+    public static String redact(String url) {
+        if (url == null) {
+            return REDACTED;
+        }
+        try {
+            URI uri = URI.create(url);
+            if (uri.getScheme() == null || uri.getRawAuthority() == null) {
+                return REDACTED;
+            }
+            return redactParts(uri.getScheme(), uri.getRawAuthority(), uri.getRawPath());
+        } catch (IllegalArgumentException ignored) {
+            try {
+                URL parsed = new URL(url);
+                String authority = parsed.getAuthority();
+                String path = parsed.getPath();
+                if (authority == null
+                        || containsUnsafeLogCharacter(authority)
+                        || containsUnsafeLogCharacter(path)) {
+                    return REDACTED;
+                }
+                return redactParts(parsed.getProtocol(), authority, path);
+            } catch (MalformedURLException | IllegalArgumentException malformed) {
+                return REDACTED;
+            }
+        }
+    }
+
+    private static String redactParts(String scheme, String authority, String path) {
+        int userInfoEnd = authority.lastIndexOf('@');
+        if (userInfoEnd >= 0) {
+            authority = authority.substring(userInfoEnd + 1);
+        }
+        if (authority.isEmpty()) {
+            return REDACTED;
+        }
+        return scheme + "://" + authority + (path == null ? "" : path);

Review Comment:
   Invalid-port authorities now fail closed to `<redacted>` in both languages, including through `SkillManager`, with coverage for the reported credential-shaped case.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#issuecomment-5422710299

   thanks for looking this over, addressed the nit. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3781344787


##########
api/src/test/java/org/apache/flink/agents/api/skills/SkillsResourceTest.java:
##########
@@ -65,6 +122,16 @@ void roundTripsThroughJackson() throws Exception {
         assertEquals(original.getSources(), restored.getSources());
     }
 
+    @Test
+    void pinnedUrlRoundTripsThroughJackson() throws Exception {

Review Comment:
   nit: this round trip stops at `sha256`, and `test_skills.py::test_serialize_roundtrip` does the same, so `allow_insecure_http` isn't covered on either side.
   
   I compared what the two languages emit and they match exactly — both write the flag as the string `"true"` and omit it otherwise, and both read it back the same way — so there's no bug here today. The only gap is that nothing would catch it if they drifted. Raising it because #1003 mentions alignment "including plan serialization".
   
   Would adding the flag to this test and its Python counterpart be enough?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3840665754


##########
python/flink_agents/api/tests/test_skills.py:
##########
@@ -30,12 +32,53 @@ def test_from_local_dir_emits_local_scheme(self) -> None:
 
     def test_from_url_emits_url_scheme(self) -> None:
         s = Skills.from_url("https://example.com/x.zip")
+        assert s.sources == [
+            SkillSourceSpec(scheme="url", params={"url": "https://example.com/x.zip"})
+        ]
+
+    def test_from_url_with_sha256_emits_integrity_param(self) -> None:
+        digest = "A" * 64
+        s = Skills.from_url_with_sha256("https://example.com/x.zip", digest)
         assert s.sources == [
             SkillSourceSpec(
-                scheme="url", params={"url": "https://example.com/x.zip"}
+                scheme="url",
+                params={"url": "https://example.com/x.zip", "sha256": digest},
             )
         ]
 
+    def test_from_url_unsafe_requires_explicit_param(self) -> None:
+        s = Skills.from_url_unsafe("http://example.com/x.zip")
+        assert s.sources[0].params["allow_insecure_http"] == "true"
+
+    def test_from_url_unsafe_with_sha256_emits_both_params(self) -> None:
+        digest = "a" * 64
+        s = Skills.from_url_unsafe_with_sha256("http://example.com/x.zip", digest)
+        assert s.sources[0].params == {
+            "url": "http://example.com/x.zip",
+            "sha256": digest,
+            "allow_insecure_http": "true",
+        }
+
+    def test_from_url_rejects_plain_http_by_default(self) -> None:
+        with pytest.raises(ValueError, match="disabled by default"):
+            Skills.from_url("http://example.com/x.zip")
+
+    def test_from_url_with_sha256_rejects_malformed_digest(self) -> None:
+        with pytest.raises(ValueError, match="64 hexadecimal"):
+            Skills.from_url_with_sha256("https://example.com/x.zip", "invalid")
+
+    def test_from_url_rejects_unsupported_scheme_clearly(self) -> None:
+        with pytest.raises(ValueError, match=r"Only HTTP\(S\)"):
+            Skills.from_url("ftp://example.com/x.zip")
+
+    @pytest.mark.parametrize(
+        "url",
+        ["https://exa mple.com/x.zip", "https://example.com/%invalid"],

Review Comment:
   nit: both of these cases hit the two new regexes at `skills.py:189`, so the `except ValueError` just above at `skills.py:186-188` still has nothing exercising it. That branch is the only thing handling a URL `urlparse` itself refuses: `urlparse("https://[::1/x.zip")` raises `Invalid IPv6 URL` and neither regex matches that string, while `URI.create` rejects the same string as `Expected closing bracket for IPv6 address`.
   
   Adding it to this list would cover it. Nothing asserts `Invalid skill URL:` on the Java side either (`Skills.java:165-167`), if you want one per language like last time.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#issuecomment-5595078110

   Thanks for addressing the comments. LGTM.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 merged PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3891323377


##########
python/flink_agents/api/skills.py:
##########
@@ -57,13 +57,117 @@ def packaged_skills() -> Skills:
 
 from __future__ import annotations
 
+import re
+from ipaddress import AddressValueError, IPv6Address
 from typing import Dict, List, Tuple
+from urllib.parse import urlparse, urlsplit, urlunsplit
 
 from pydantic import BaseModel, ConfigDict, Field, field_validator
 from typing_extensions import override
 
 from flink_agents.api.resource import ResourceType, SerializableResource
 
+_INVALID_URI_CHARACTER = re.compile(r'[\x00-\x20\x7f<>"{}|\\^`]')
+_INVALID_PERCENT_ESCAPE = re.compile(r"%(?![0-9a-fA-F]{2})")
+_HOST_LABEL = re.compile(r"[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?")
+
+
+def redact_skill_url(url: str) -> str:
+    """Return a skill URL without user info, query parameters, or a fragment.
+
+    Internal contract shared with the runtime; not a stable public API.
+    """
+    try:
+        parts = urlsplit(url)
+        if not parts.scheme or not parts.netloc:
+            return "<redacted>"
+        netloc = parts.netloc.rsplit("@", 1)[-1]
+        if not netloc:
+            return "<redacted>"
+        return urlunsplit((parts.scheme, netloc, parts.path, "", ""))
+    except ValueError:
+        return "<redacted>"
+
+
+def validate_skill_url(url: str, *, allow_insecure_http: bool) -> str:
+    """Validate a skill URL using the contract shared with the Java API.
+
+    Internal contract shared with the runtime; not a stable public API.
+    """
+    if not isinstance(url, str):
+        msg = "skill URL must be a string"
+        raise TypeError(msg)
+    try:
+        parsed = urlparse(url)
+    except ValueError:
+        msg = f"Invalid skill URL: {redact_skill_url(url)}"
+        raise ValueError(msg) from None
+    if _INVALID_URI_CHARACTER.search(url) or _INVALID_PERCENT_ESCAPE.search(url):
+        msg = f"Invalid skill URL: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    # Java's URI rejects raw brackets in the path (but not in the query or
+    # fragment); encoded %5B/%5D and IPv6 authority brackets stay valid.
+    if any(c in f"{parsed.path};{parsed.params}" for c in "[]"):
+        msg = f"Invalid skill URL: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    scheme = parsed.scheme.lower()
+    if scheme not in {"http", "https"}:
+        msg = f"Only HTTP(S) skill URLs are supported: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    try:
+        hostname = parsed.hostname
+        _ = parsed.port
+    except ValueError:
+        msg = (
+            "Skill URL must include a valid host and, when present, a valid port: "
+            f"{redact_skill_url(url)}"
+        )
+        raise ValueError(msg) from None
+    if parsed.username is not None:
+        msg = f"Skill URL must not include user info: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    bracketed_host = parsed.netloc.rsplit("@", 1)[-1].startswith("[")
+    if (
+        not hostname
+        or (bracketed_host and ":" not in hostname)
+        or not _is_valid_hostname(hostname)
+    ):
+        msg = f"Skill URL must include a valid host: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    if scheme == "http" and not allow_insecure_http:
+        msg = (
+            "Plain HTTP skill URLs are disabled by default; use HTTPS or "
+            "explicitly allow insecure HTTP for this source: "
+            f"{redact_skill_url(url)}"
+        )
+        raise ValueError(msg)
+    return scheme
+
+
+def _is_valid_hostname(hostname: str) -> bool:
+    """Match the host syntax accepted by Java URI.parseServerAuthority()."""
+    if ":" in hostname:
+        try:
+            IPv6Address(hostname)
+        except AddressValueError:
+            return False
+        return True
+    if not hostname.isascii():

Review Comment:
   Two URLs where Python and Java disagree, in opposite directions. Measured on CPython 3.11.14, against a copy of `SkillUrlUtils` on temurin-11:
   
   | URL | Python | Java |
   |---|---|---|
   | `https://<U+212A KELVIN SIGN>.com/x.zip` | accepted | rejected (`must include a valid host and, when present, a valid port`) |
   | `https://[fe80::1%eth0]/x.zip` | rejected (`Invalid skill URL`) | accepted |
   
   Row 1: `urllib` lower-cases `parsed.hostname` at `:118`, before the `isascii()` check on this line. U+212A is the only non-ASCII character that lower-cases to an ASCII letter or digit, `k` (I walked U+0080 to U+10FFFF). So the check never sees a non-ASCII host, the log shows the Kelvin character, and `urllib` connects to `k.com`. It also gets past the rule `test_skills.py:131-137` sets. Adding `not parsed.netloc.isascii()` at `:130-134` catches it, with the skills suites (166 tests) still green.
   
   Row 2: `_INVALID_PERCENT_ESCAPE` (`:71`, used at `:105`) matches `%et`, so Python rejects the zone id before `_is_valid_hostname` runs, while Java's `URI.create` allows a raw `%` inside an IPv6 literal. Python is the stricter side, so nothing fails open, but the tests bake the split in: `test_skills.py:170` calls this URL malformed, and the Java twin at `SkillsResourceTest.java:134` leaves it out.
   
   `AGENTS.md` asks for Java and Python to line up. Which way would you want each of these to go?



##########
runtime/src/test/java/org/apache/flink/agents/runtime/skill/URLSkillRepositoryTest.java:
##########
@@ -97,30 +106,166 @@ void loadFromUrl(@TempDir Path tempDir) throws IOException {
         }
     }
 
+    @Test
+    void plainHttpRejectedByDefault() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> new URLSkillRepository("http://example.com/skills.zip"));
+        assertTrue(ex.getMessage().contains("disabled by default"));
+    }
+
+    @Test
+    void sha256MismatchRejectedBeforeExtraction(@TempDir Path tempDir) throws IOException {
+        Path zip = tempDir.resolve("skills.zip");
+        try (ZipOutputStream zos = new ZipOutputStream(Files.newOutputStream(zip))) {
+            zos.putNextEntry(new ZipEntry("../evil.txt"));
+            zos.write("pwn".getBytes(StandardCharsets.UTF_8));
+            zos.closeEntry();
+        }
+        HttpServer server = startZipServer(Files.readAllBytes(zip), 200);
+        try {
+            int port = server.getAddress().getPort();
+            String url = "http://127.0.0.1:" + port + "/skills.zip?token=top-secret#fragment";
+            IllegalArgumentException ex =
+                    assertThrows(
+                            IllegalArgumentException.class,
+                            () -> new URLSkillRepository(url, "0".repeat(64), true));
+            assertTrue(ex.getMessage().contains("SHA-256 mismatch"));
+            assertTrue(ex.getMessage().contains("http://127.0.0.1:" + port + "/skills.zip"));
+            assertFalse(ex.getMessage().contains("top-secret"));
+        } finally {
+            server.stop(0);
+        }
+    }
+
+    @Test
+    void invalidHostAndPortAreRejectedBeforeDownload() {
+        IllegalArgumentException malformedPort =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () ->
+                                new URLSkillRepository(
+                                        "https://example.com:bad/skills.zip?token=top-secret"));
+        assertNull(malformedPort.getCause());
+        assertTrue(malformedPort.getMessage().contains("https://example.com:bad/skills.zip"));
+        assertFalse(malformedPort.getMessage().contains("top-secret"));
+
+        for (String url :
+                List.of("https://:443/skills.zip", "https://example.com:65536/skills.zip")) {
+            assertThrows(IllegalArgumentException.class, () -> new URLSkillRepository(url), url);
+        }
+    }
+
+    @Test
+    void invalidHostnameSyntaxIsRejectedBeforeDownload() {
+        for (String url :
+                List.of(
+                        "https://exa_mple.com/skills.zip",
+                        "https://tést.com/skills.zip",
+                        "https://%65xample.com/skills.zip",
+                        "https://-example.com/skills.zip",
+                        "https://example-.com/skills.zip",
+                        "https://.example.com/skills.zip",
+                        "https://example..com/skills.zip",
+                        "https://a../skills.zip",
+                        "https://../skills.zip",
+                        "https://999.999.999.999/skills.zip",
+                        "https://127.1/skills.zip",
+                        "https://1.2.3/skills.zip",
+                        "https://foo.123/skills.zip",
+                        "https://foo.1bar/skills.zip",
+                        "https://1.2.3.4.5/skills.zip",
+                        "https://1.2.3./skills.zip",
+                        "https://1.2.3.4./skills.zip",
+                        "https://[v1.foo]/skills.zip")) {
+            assertThrows(IllegalArgumentException.class, () -> new URLSkillRepository(url), url);
+        }
+    }
+
+    @Test
+    void rawPercentEscapeIsRejectedByDownloader() {

Review Comment:
   The name says validation rejects this URL, but it does not. On temurin-11, `URI.create` (`SkillUrlUtils.java:47`) and `parseServerAuthority()` (`:58`) both accept it with `host=[fe80::1%eth0]`, and the user-info and port checks pass. The `IOException` you catch is an `UnknownHostException` from DNS. So this goes green because the network failed, and any unrelated network error would do the same.
   
   It also depends on the machine. macOS has no `eth0`, so the lookup fails right away. Where one exists, the network decides instead, against `REQUEST_TIMEOUT_MS = 90_000` (`URLSkillRepository.java:46`). I have not tried a Linux runner, so treat that half as a guess.
   
   Would asserting the message, on a URL the validator rejects by itself, be better here?



##########
api/src/test/java/org/apache/flink/agents/api/skills/SkillsResourceTest.java:
##########
@@ -48,6 +53,174 @@ void fromUrlEmitsUrlScheme() {
                 skills.getSources());
     }
 
+    @Test
+    void fromUrlAcceptsSharedValidHostSyntax() {
+        for (String url :
+                List.of(
+                        "https://localhost/x.zip",
+                        "https://127.0.0.1/x.zip",
+                        "https://[::1]/x.zip",
+                        "https://[fe80::1%25eth0]/x.zip",
+                        "https://example.com./x.zip",
+                        "https://example.com:/x.zip",
+                        "https://999/x.zip",
+                        "https://1bar/x.zip",
+                        "https://999./x.zip")) {

Review Comment:
   nit: the port bound at `SkillUrlUtils.java:72` is only half covered. I replayed this suite against the real class and changed the bound one line at a time. Deleting the check turns it red, and `> 65536` turns it red, but `>= 65535` stays green, because nothing here accepts port 65535. Adding `"https://example.com:65535/x.zip"` to this list would cover it. Worth one more entry?



##########
api/src/main/java/org/apache/flink/agents/api/skills/SkillUrlUtils.java:
##########
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.agents.api.skills;
+
+import org.apache.flink.annotation.Internal;
+
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Locale;
+
+/**
+ * Shared validation and redaction helpers for URL-backed skill sources. Internal contract shared
+ * with the runtime module; not a stable public API.
+ */
+@Internal
+public final class SkillUrlUtils {
+
+    private SkillUrlUtils() {}
+
+    /**
+     * Validate {@code url} and return its lowercase {@code http} or {@code https} scheme.
+     *
+     * @throws IllegalArgumentException if the URL is invalid or violates the transport policy.
+     */
+    public static String validate(String url, boolean allowInsecureHttp) {
+        if (url == null) {
+            throw new IllegalArgumentException("skill URL must not be null");
+        }
+        URI uri;
+        try {
+            uri = URI.create(url);
+        } catch (IllegalArgumentException ignored) {
+            throw new IllegalArgumentException("Invalid skill URL: " + redact(url));
+        }
+        String scheme = uri.getScheme();
+        scheme = scheme == null ? "" : scheme.toLowerCase(Locale.ROOT);
+        if (!(scheme.equals("http") || scheme.equals("https"))) {
+            throw new IllegalArgumentException(
+                    "Only HTTP(S) skill URLs are supported: " + redact(url));
+        }
+        try {
+            uri = uri.parseServerAuthority();
+        } catch (URISyntaxException ignored) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host and, when present, a valid port: "
+                            + redact(url));
+        }
+        if (uri.getRawUserInfo() != null) {
+            throw new IllegalArgumentException(
+                    "Skill URL must not include user info: " + redact(url));
+        }
+        if (uri.getHost() == null || uri.getHost().isEmpty()) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host: " + redact(url));
+        }
+        if (uri.getPort() > 65535) {
+            throw new IllegalArgumentException(
+                    "Skill URL port must be between 0 and 65535: " + redact(url));
+        }
+        if (scheme.equals("http") && !allowInsecureHttp) {
+            throw new IllegalArgumentException(
+                    "Plain HTTP skill URLs are disabled by default; use HTTPS or explicitly allow"
+                            + " insecure HTTP for this source: "
+                            + redact(url));
+        }
+        return scheme;
+    }
+
+    /** Return {@code url} without user info, query parameters, or a fragment. */
+    public static String redact(String url) {
+        if (url == null) {
+            return "<redacted>";
+        }
+        try {
+            URI uri = URI.create(url);
+            if (uri.getScheme() == null || uri.getRawAuthority() == null) {
+                return "<redacted>";
+            }
+            String authority = uri.getRawAuthority();
+            int userInfoEnd = authority.lastIndexOf('@');
+            if (userInfoEnd >= 0) {
+                authority = authority.substring(userInfoEnd + 1);
+            }
+            if (authority.isEmpty()) {
+                return "<redacted>";
+            }
+            return uri.getScheme() + "://" + authority + uri.getRawPath();
+        } catch (IllegalArgumentException ignored) {
+            return "<redacted>";

Review Comment:
   `redact` parses the URL again with `URI.create` at `:91`. That parse fails for exactly the URLs `validate` just rejected at `:47-50`, so the caller gets this constant instead of a URL. Over a 3455-URL fuzz run, 2902 of the rejections came out that way.
   
   | input | Java | Python |
   |---|---|---|
   | `https://u:pw@example.com/a b.zip?token=SECRET` | `Invalid skill URL: <redacted>` | `Invalid skill URL: https://example.com/a b.zip` |
   | `https://u:pw@example.com/%zz?token=SECRET` | `Invalid skill URL: <redacted>` | `Invalid skill URL: https://example.com/%zz` |
   
   Python does the same job with `urlsplit` and `urlunsplit` (`skills.py:80-89`): no raise on these inputs, user info and query still stripped, and still `<redacted>` when it cannot parse at all. Scanning both sides over that corpus for `SECRET`, `?`, `#` and `@` gave zero hits, so the extra detail leaks nothing.
   
   It bites in `fromUrl(String…)`, which validates inside a stream (`Skills.java:99-108`). Several URLs with one typo, and the message names none of them.
   
   I can read the constant as a deliberate fail-closed choice, and `SkillsResourceTest.java:138` locks it in. Is that the intent, or is the Python form worth matching here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3930991498


##########
api/src/test/java/org/apache/flink/agents/api/skills/SkillsResourceTest.java:
##########
@@ -48,6 +53,224 @@ void fromUrlEmitsUrlScheme() {
                 skills.getSources());
     }
 
+    @Test
+    void fromUrlAcceptsSharedValidHostSyntax() {
+        for (String url :
+                List.of(
+                        "https://localhost/x.zip",
+                        "https://127.0.0.1/x.zip",
+                        "https://[::1]/x.zip",
+                        "https://example.com./x.zip",
+                        "https://example.com:/x.zip",
+                        "https://example.com:65535/x.zip",
+                        "https://999/x.zip",
+                        "https://1bar/x.zip",
+                        "https://999./x.zip")) {
+            assertEquals(url, Skills.fromUrl(url).getSources().get(0).getParams().get("url"));
+        }
+    }
+
+    @Test
+    void fromUrlRejectsScopedIpv6() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> Skills.fromUrl("https://[fe80::1%25lo0]/x.zip"));
+        assertTrue(ex.getMessage().contains("must not include an IPv6 zone identifier"));
+        assertNull(ex.getCause());
+    }
+
+    @Test
+    void fromUrlWithSha256EmitsIntegrityParam() {
+        String digest = "A".repeat(64);
+        Skills skills = Skills.fromUrlWithSha256("https://example.com/x.zip", digest);
+        assertEquals(
+                List.of(
+                        new SkillSourceSpec(
+                                "url",
+                                Map.of("url", "https://example.com/x.zip", "sha256", digest))),
+                skills.getSources());
+    }
+
+    @Test
+    void fromUrlUnsafeRequiresExplicitParam() {
+        Skills skills = Skills.fromUrlUnsafe("http://example.com/x.zip");
+        assertEquals("true", skills.getSources().get(0).getParams().get("allow_insecure_http"));
+    }
+
+    @Test
+    void fromUrlUnsafeWithSha256EmitsBothParams() {
+        String digest = "a".repeat(64);
+        Skills skills = Skills.fromUrlUnsafeWithSha256("http://example.com/x.zip", digest);
+        assertEquals(
+                Map.of(
+                        "url",
+                        "http://example.com/x.zip",
+                        "sha256",
+                        digest,
+                        "allow_insecure_http",
+                        "true"),
+                skills.getSources().get(0).getParams());
+    }
+
+    @Test
+    void fromUrlRejectsPlainHttpByDefault() {
+        assertThrows(
+                IllegalArgumentException.class, () -> Skills.fromUrl("http://example.com/x.zip"));
+    }
+
+    @Test
+    void fromUrlWithSha256RejectsMalformedDigest() {
+        assertThrows(
+                IllegalArgumentException.class,
+                () -> Skills.fromUrlWithSha256("https://example.com/x.zip", "invalid"));
+    }
+
+    @Test
+    void fromUrlRejectsUnsupportedSchemeClearly() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () ->
+                                Skills.fromUrl(
+                                        "ftp://user:password@example.com/x.zip?token=secret#part"));
+        assertEquals(
+                "Only HTTP(S) skill URLs are supported: ftp://example.com/x.zip", ex.getMessage());
+        assertFalse(ex.getMessage().contains("password"));
+        assertFalse(ex.getMessage().contains("secret"));
+    }
+
+    @Test
+    void fromUrlRejectsMalformedUrl() {
+        IllegalArgumentException unparseable =
+                assertThrows(
+                        IllegalArgumentException.class, () -> Skills.fromUrl("https://[::1/x.zip"));
+        assertEquals("Invalid skill URL: <redacted>", unparseable.getMessage());
+
+        IllegalArgumentException unsafeForLogs =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> Skills.fromUrl("https://u:pw@example.com/a\nheader?token=SECRET"));
+        assertEquals("Invalid skill URL: <redacted>", unsafeForLogs.getMessage());
+        assertFalse(unsafeForLogs.getMessage().contains("pw"));
+        assertFalse(unsafeForLogs.getMessage().contains("SECRET"));
+
+        for (String url :
+                List.of("https://example.com/skills[1].zip", "https://[fe80::1%eth0]/x.zip")) {
+            IllegalArgumentException ex =
+                    assertThrows(IllegalArgumentException.class, () -> Skills.fromUrl(url), url);
+            assertEquals("Invalid skill URL: " + url, ex.getMessage());
+        }
+    }
+
+    @Test
+    void fromUrlRedactsMalformedUrlWithoutLosingSafeContext() {
+        for (String path : List.of("a b.zip", "%zz")) {
+            IllegalArgumentException ex =
+                    assertThrows(
+                            IllegalArgumentException.class,
+                            () ->
+                                    Skills.fromUrl(
+                                            "https://u:pw@example.com/"
+                                                    + path
+                                                    + "?token=SECRET#fragment"));
+            assertEquals("Invalid skill URL: https://example.com/" + path, ex.getMessage());
+            assertFalse(ex.getMessage().contains("pw"));
+            assertFalse(ex.getMessage().contains("SECRET"));
+        }
+    }
+
+    @Test
+    void fromUrlAcceptsBracketsOutsideRawPath() {
+        for (String url :
+                List.of(
+                        "https://example.com/skills%5B1%5D.zip",
+                        "https://example.com/x.zip?a[0]=1", "https://example.com/x.zip#f[1]")) {
+            assertEquals(url, Skills.fromUrl(url).getSources().get(0).getParams().get("url"));
+        }
+    }
+
+    @Test
+    void fromUrlRejectsInvalidHostAndPort() {
+        for (String url :
+                List.of(
+                        "https://example.com:bad/x.zip?token=top-secret",
+                        "https://user:supersecret/x.zip?token=TOPSECRET")) {

Review Comment:
   Thanks for addressing the comments. Nothing further from me.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3837507296


##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -254,14 +255,47 @@ private static void copyJarEntries(URL jarUrl, String prefix, Path extractDir)
      * @throws IOException on connect / read failures or HTTP error responses.
      */
     public static Path downloadToTempFile(String url, int timeoutMs) throws IOException {
+        return downloadToTempFile(url, timeoutMs, false);
+    }
+
+    /**
+     * Download {@code url}, optionally permitting plain HTTP transport.
+     *
+     * @throws IOException on connect / read failures or HTTP error responses.
+     */
+    public static Path downloadToTempFile(String url, int timeoutMs, boolean allowInsecureHttp)
+            throws IOException {
         URL u = new URL(url);
+        String initialProtocol = u.getProtocol();
+        if (!("https".equalsIgnoreCase(initialProtocol)
+                || (allowInsecureHttp && "http".equalsIgnoreCase(initialProtocol)))) {
+            throw new IOException("Skill URL uses a disallowed transport: " + url);
+        }
         HttpURLConnection conn = (HttpURLConnection) u.openConnection();
         conn.setConnectTimeout(timeoutMs);
         conn.setReadTimeout(timeoutMs);
         conn.setRequestMethod("GET");
+        // HttpURLConnection follows same-protocol redirects but leaves cross-protocol redirects
+        // unfollowed. Any future HTTP client must preserve that restriction.
+        conn.setInstanceFollowRedirects(true);

Review Comment:
   The comment is accurate, the JDK really does decline a scheme-changing redirect: `followRedirect()` compares the two protocols and bails before taking it.
   
   The call itself isn't quite a no-op though. `instanceFollowRedirects` is initialised from the static `followRedirects`, so if anything in the JVM has called `HttpURLConnection.setFollowRedirects(false)`, which some deployments do as a hardening step, this line overrides that and re-enables redirect following for skill downloads specifically. I checked on temurin-11: after `setFollowRedirects(false)` the instance default reads back `false`, and this line flips it to `true`. Without it the 3xx would surface and `:284` would fail closed with a clear message.
   
   Was pinning it to `true` deliberate, or is the comment doing the real work here?



##########
api/src/test/java/org/apache/flink/agents/api/yaml/YamlLoaderBuildAgentsTest.java:
##########
@@ -133,6 +133,34 @@ void skillsPerAgentAndShared() {
                         new SkillSourceSpec("local", Map.of("path", "./more")));
     }
 
+    @Test
+    void rejectsPlainHttpSkillUrlDuringLoading(@TempDir Path tmp) throws Exception {

Review Comment:
   nit: this covers the `urls:` list, and the new four-way test in `YamlLoaderBuildersTest` covers the `url_sources` shapes that succeed. The case #1003 is actually about, `url_sources: [{url: http://…}]` with `allow_insecure_http` absent, isn't asserted on either side, and the issue asks for "focused Java and Python tests for HTTP rejection or opt-in behavior".
   
   The positive cases do pin the factory choice indirectly, so this is coverage rather than a hole. Worth one line per language?



##########
python/flink_agents/api/skills.py:
##########
@@ -107,22 +109,96 @@ def from_local_dir(cls, *paths: str) -> Skills:
         a zip, its top-level entries are the skill subdirectories.
         """
         return cls(
-            sources=[
-                SkillSourceSpec(scheme="local", params={"path": p}) for p in paths
-            ]
+            sources=[SkillSourceSpec(scheme="local", params={"path": p}) for p in paths]
         )
 
     @classmethod
     def from_url(cls, *urls: str) -> Skills:
-        """Create a Skills resource from one or more http(s) URLs.
+        """Create a Skills resource from one or more HTTPS URLs.
 
         Each URL must point to a ``.zip`` whose top level is the baseDir
         (i.e. skill subdirectories sit at the top of the zip).
         """
+        for url in urls:
+            cls._require_url(url, allow_insecure_http=False)
         return cls(
             sources=[SkillSourceSpec(scheme="url", params={"url": u}) for u in urls]
         )
 
+    @classmethod
+    def from_url_with_sha256(cls, url: str, sha256: str) -> Skills:
+        """Create an HTTPS URL source pinned to a SHA-256 archive digest."""
+        cls._require_url(url, allow_insecure_http=False)
+        cls._require_sha256(sha256)
+        return cls(
+            sources=[
+                SkillSourceSpec(scheme="url", params={"url": url, "sha256": sha256})
+            ]
+        )
+
+    @classmethod
+    def from_url_unsafe(cls, *urls: str) -> Skills:
+        """Create URL sources that explicitly permit plain HTTP transport.
+
+        This compatibility escape hatch should be used only on trusted networks.
+        Prefer :meth:`from_url` with HTTPS.
+        """
+        for url in urls:
+            cls._require_url(url, allow_insecure_http=True)
+        return cls(
+            sources=[
+                SkillSourceSpec(
+                    scheme="url",
+                    params={"url": url, "allow_insecure_http": "true"},
+                )
+                for url in urls
+            ]
+        )
+
+    @classmethod
+    def from_url_unsafe_with_sha256(cls, url: str, sha256: str) -> Skills:
+        """Create a digest-pinned source that explicitly permits plain HTTP."""
+        cls._require_url(url, allow_insecure_http=True)
+        cls._require_sha256(sha256)
+        return cls(
+            sources=[
+                SkillSourceSpec(
+                    scheme="url",
+                    params={
+                        "url": url,
+                        "sha256": sha256,
+                        "allow_insecure_http": "true",
+                    },
+                )
+            ]
+        )
+
+    @staticmethod
+    def _require_url(url: str, *, allow_insecure_http: bool) -> None:
+        if not isinstance(url, str):
+            msg = "skill URL must be a string"
+            raise TypeError(msg)
+        parsed = urlparse(url)

Review Comment:
   `urlparse` doesn't reject a malformed URL, so `https://exa mple.com/x.zip` is accepted here and only fails later at download time, while `Skills.java:163-168` runs the same string through `URI.create` and rethrows it as `Invalid skill URL:` at call time.
   
   Before these commits neither YAML loader reached either validator. Now both do, so the split is exercised at load time on both runtimes rather than only through the code API.
   
   Is the later failure good enough here, or would you rather Python rejected it at the same point Java does?



##########
api/src/main/java/org/apache/flink/agents/api/skills/Skills.java:
##########
@@ -92,17 +94,94 @@ public static Skills fromLocalDir(String... paths) {
     }
 
     /**
-     * Create a {@link Skills} resource from one or more http(s) URLs.
+     * Create a {@link Skills} resource from one or more HTTPS URLs.
      *
      * <p>Each URL must point to a {@code .zip} whose top level is the baseDir.
      */
     public static Skills fromUrl(String... urls) {
         return new Skills(
                 Arrays.stream(urls)
-                        .map(u -> new SkillSourceSpec("url", Map.of("url", u)))
+                        .map(
+                                u -> {
+                                    requireUrl(u, false);
+                                    return new SkillSourceSpec("url", Map.of("url", u));
+                                })
                         .collect(Collectors.toList()));
     }
 
+    /**
+     * Create a {@link Skills} resource from an HTTPS URL pinned to a SHA-256 digest.
+     *
+     * <p>The digest is verified against the downloaded archive before extraction.
+     */
+    public static Skills fromUrlWithSha256(String url, String sha256) {
+        return urlSource(url, sha256, false);
+    }
+
+    /**
+     * Create a {@link Skills} resource that explicitly permits plain HTTP transport.
+     *
+     * <p>This compatibility escape hatch should be used only on trusted networks. Prefer {@link
+     * #fromUrl(String...)} with HTTPS.
+     */
+    public static Skills fromUrlUnsafe(String... urls) {
+        return new Skills(
+                Arrays.stream(urls)
+                        .map(
+                                u -> {
+                                    requireUrl(u, true);
+                                    return new SkillSourceSpec(
+                                            "url", Map.of("url", u, "allow_insecure_http", "true"));
+                                })
+                        .collect(Collectors.toList()));
+    }
+
+    /**
+     * Create a digest-pinned {@link Skills} resource that explicitly permits plain HTTP transport.
+     */
+    public static Skills fromUrlUnsafeWithSha256(String url, String sha256) {
+        return urlSource(url, sha256, true);
+    }
+
+    private static Skills urlSource(String url, String sha256, boolean allowInsecureHttp) {
+        requireUrl(url, allowInsecureHttp);
+        if (sha256 == null || !sha256.matches("[0-9a-fA-F]{64}")) {

Review Comment:
   nit: this accepts either case and `URLSkillRepository.java:90` lowercases before comparing, which matches what `skills.md:144` promises ("lowercase or uppercase SHA-256 digest"). Every digest literal in the suite is lowercase though (`"a".repeat(64)` and friends), so nothing walks the uppercase path.
   
   Would flipping one existing digest literal to uppercase cover it?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3798481236


##########
runtime/src/test/java/org/apache/flink/agents/runtime/skill/URLSkillRepositoryTest.java:
##########
@@ -97,19 +100,57 @@ void loadFromUrl(@TempDir Path tempDir) throws IOException {
         }
     }
 
+    @Test
+    void plainHttpRejectedByDefault() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> new URLSkillRepository("http://example.com/skills.zip"));
+        assertTrue(ex.getMessage().contains("disabled by default"));
+    }
+
+    @Test
+    void sha256MismatchRejectedBeforeExtraction(@TempDir Path tempDir) throws IOException {

Review Comment:
   no reason, that works well. updated tests so they now fail if extraction happens first.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3837978246


##########
api/src/main/java/org/apache/flink/agents/api/skills/Skills.java:
##########
@@ -92,17 +94,94 @@ public static Skills fromLocalDir(String... paths) {
     }
 
     /**
-     * Create a {@link Skills} resource from one or more http(s) URLs.
+     * Create a {@link Skills} resource from one or more HTTPS URLs.
      *
      * <p>Each URL must point to a {@code .zip} whose top level is the baseDir.
      */
     public static Skills fromUrl(String... urls) {
         return new Skills(
                 Arrays.stream(urls)
-                        .map(u -> new SkillSourceSpec("url", Map.of("url", u)))
+                        .map(
+                                u -> {
+                                    requireUrl(u, false);
+                                    return new SkillSourceSpec("url", Map.of("url", u));
+                                })
                         .collect(Collectors.toList()));
     }
 
+    /**
+     * Create a {@link Skills} resource from an HTTPS URL pinned to a SHA-256 digest.
+     *
+     * <p>The digest is verified against the downloaded archive before extraction.
+     */
+    public static Skills fromUrlWithSha256(String url, String sha256) {
+        return urlSource(url, sha256, false);
+    }
+
+    /**
+     * Create a {@link Skills} resource that explicitly permits plain HTTP transport.
+     *
+     * <p>This compatibility escape hatch should be used only on trusted networks. Prefer {@link
+     * #fromUrl(String...)} with HTTPS.
+     */
+    public static Skills fromUrlUnsafe(String... urls) {
+        return new Skills(
+                Arrays.stream(urls)
+                        .map(
+                                u -> {
+                                    requireUrl(u, true);
+                                    return new SkillSourceSpec(
+                                            "url", Map.of("url", u, "allow_insecure_http", "true"));
+                                })
+                        .collect(Collectors.toList()));
+    }
+
+    /**
+     * Create a digest-pinned {@link Skills} resource that explicitly permits plain HTTP transport.
+     */
+    public static Skills fromUrlUnsafeWithSha256(String url, String sha256) {
+        return urlSource(url, sha256, true);
+    }
+
+    private static Skills urlSource(String url, String sha256, boolean allowInsecureHttp) {
+        requireUrl(url, allowInsecureHttp);
+        if (sha256 == null || !sha256.matches("[0-9a-fA-F]{64}")) {

Review Comment:
   done!



##########
python/flink_agents/api/skills.py:
##########
@@ -107,22 +109,96 @@ def from_local_dir(cls, *paths: str) -> Skills:
         a zip, its top-level entries are the skill subdirectories.
         """
         return cls(
-            sources=[
-                SkillSourceSpec(scheme="local", params={"path": p}) for p in paths
-            ]
+            sources=[SkillSourceSpec(scheme="local", params={"path": p}) for p in paths]
         )
 
     @classmethod
     def from_url(cls, *urls: str) -> Skills:
-        """Create a Skills resource from one or more http(s) URLs.
+        """Create a Skills resource from one or more HTTPS URLs.
 
         Each URL must point to a ``.zip`` whose top level is the baseDir
         (i.e. skill subdirectories sit at the top of the zip).
         """
+        for url in urls:
+            cls._require_url(url, allow_insecure_http=False)
         return cls(
             sources=[SkillSourceSpec(scheme="url", params={"url": u}) for u in urls]
         )
 
+    @classmethod
+    def from_url_with_sha256(cls, url: str, sha256: str) -> Skills:
+        """Create an HTTPS URL source pinned to a SHA-256 archive digest."""
+        cls._require_url(url, allow_insecure_http=False)
+        cls._require_sha256(sha256)
+        return cls(
+            sources=[
+                SkillSourceSpec(scheme="url", params={"url": url, "sha256": sha256})
+            ]
+        )
+
+    @classmethod
+    def from_url_unsafe(cls, *urls: str) -> Skills:
+        """Create URL sources that explicitly permit plain HTTP transport.
+
+        This compatibility escape hatch should be used only on trusted networks.
+        Prefer :meth:`from_url` with HTTPS.
+        """
+        for url in urls:
+            cls._require_url(url, allow_insecure_http=True)
+        return cls(
+            sources=[
+                SkillSourceSpec(
+                    scheme="url",
+                    params={"url": url, "allow_insecure_http": "true"},
+                )
+                for url in urls
+            ]
+        )
+
+    @classmethod
+    def from_url_unsafe_with_sha256(cls, url: str, sha256: str) -> Skills:
+        """Create a digest-pinned source that explicitly permits plain HTTP."""
+        cls._require_url(url, allow_insecure_http=True)
+        cls._require_sha256(sha256)
+        return cls(
+            sources=[
+                SkillSourceSpec(
+                    scheme="url",
+                    params={
+                        "url": url,
+                        "sha256": sha256,
+                        "allow_insecure_http": "true",
+                    },
+                )
+            ]
+        )
+
+    @staticmethod
+    def _require_url(url: str, *, allow_insecure_http: bool) -> None:
+        if not isinstance(url, str):
+            msg = "skill URL must be a string"
+            raise TypeError(msg)
+        parsed = urlparse(url)

Review Comment:
   aligned Python with Java so malformed URLs fail immediately. also added coverage for whitespace and invalid percent escapes.



##########
api/src/test/java/org/apache/flink/agents/api/yaml/YamlLoaderBuildAgentsTest.java:
##########
@@ -133,6 +133,34 @@ void skillsPerAgentAndShared() {
                         new SkillSourceSpec("local", Map.of("path", "./more")));
     }
 
+    @Test
+    void rejectsPlainHttpSkillUrlDuringLoading(@TempDir Path tmp) throws Exception {

Review Comment:
   added focused Java and Python tests for plain HTTP in `url_sources` without the opt-in.



##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -254,14 +255,47 @@ private static void copyJarEntries(URL jarUrl, String prefix, Path extractDir)
      * @throws IOException on connect / read failures or HTTP error responses.
      */
     public static Path downloadToTempFile(String url, int timeoutMs) throws IOException {
+        return downloadToTempFile(url, timeoutMs, false);
+    }
+
+    /**
+     * Download {@code url}, optionally permitting plain HTTP transport.
+     *
+     * @throws IOException on connect / read failures or HTTP error responses.
+     */
+    public static Path downloadToTempFile(String url, int timeoutMs, boolean allowInsecureHttp)
+            throws IOException {
         URL u = new URL(url);
+        String initialProtocol = u.getProtocol();
+        if (!("https".equalsIgnoreCase(initialProtocol)
+                || (allowInsecureHttp && "http".equalsIgnoreCase(initialProtocol)))) {
+            throw new IOException("Skill URL uses a disallowed transport: " + url);
+        }
         HttpURLConnection conn = (HttpURLConnection) u.openConnection();
         conn.setConnectTimeout(timeoutMs);
         conn.setReadTimeout(timeoutMs);
         conn.setRequestMethod("GET");
+        // HttpURLConnection follows same-protocol redirects but leaves cross-protocol redirects
+        // unfollowed. Any future HTTP client must preserve that restriction.
+        conn.setInstanceFollowRedirects(true);

Review Comment:
   good catch. this wasn't intentional, removed the override and added a test to confirm. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3798483806


##########
docs/content/docs/development/yaml.md:
##########
@@ -208,13 +208,14 @@ agents:
             key: tenant_id
 ```
 
-**Skills** — bundles of agent skill assets loaded from one or more sources. At least one of `paths` / `urls` / `classpath` / `package` must be non-empty; multiple sources can coexist.
+**Skills** — bundles of agent skill assets loaded from one or more sources. At least one of `paths` / `urls` / `url_sources` / `classpath` / `package` must be non-empty; multiple sources can coexist.
 
 | Field | Required | Description |
 |-------|----------|-------------|
 | `name` | yes | Skills resource name. |
 | `paths` | one-of | `local` scheme: list of directories or `.zip` files. |
-| `urls` | one-of | `url` scheme: list of `http(s)` URLs pointing to `.zip` archives. |
+| `urls` | one-of | `url` scheme: list of HTTPS URLs pointing to `.zip` archives. |

Review Comment:
   probably both. the migration path belongs in the YAML docs, while the compatibility change belongs in release notes. I added the YAML migration example here.



##########
api/src/test/java/org/apache/flink/agents/api/skills/SkillsResourceTest.java:
##########
@@ -65,6 +122,16 @@ void roundTripsThroughJackson() throws Exception {
         assertEquals(original.getSources(), restored.getSources());
     }
 
+    @Test
+    void pinnedUrlRoundTripsThroughJackson() throws Exception {

Review Comment:
   agreed. extended both round-trip tests to cover `allow_insecure_http` and verify it remains the string `"true"` thru serialization and deserialization.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "awesomesauce06 (via GitHub)" <gi...@apache.org>.
awesomesauce06 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3798469005


##########
python/flink_agents/runtime/skill/repository/_materialize.py:
##########
@@ -159,8 +167,31 @@ def download_to_tempfile(url: str, timeout: int = 90) -> Path:
     tmp_path = Path(tmp_path_str)
     try:
         with urlopen(req, timeout=timeout) as resp, tmp_path.open("wb") as out:

Review Comment:
   agreed here - added handler that rejects protocol changes before the target request is sent, including redirects to HTTP or FTP. also replaced the stubbed test with a real `HTTPServer` test.



##########
python/flink_agents/runtime/skill/repository/_materialize.py:
##########
@@ -159,8 +167,31 @@ def download_to_tempfile(url: str, timeout: int = 90) -> Path:
     tmp_path = Path(tmp_path_str)
     try:
         with urlopen(req, timeout=timeout) as resp, tmp_path.open("wb") as out:

Review Comment:
   agreed here - added handler that rejects protocol changes before the target request is sent, including redirects to HTTP or FTP. also replaced the stubbed test with a real `HTTPServer` test.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3798476006


##########
python/flink_agents/runtime/skill/repository/_materialize.py:
##########
@@ -159,8 +167,31 @@ def download_to_tempfile(url: str, timeout: int = 90) -> Path:
     tmp_path = Path(tmp_path_str)
     try:
         with urlopen(req, timeout=timeout) as resp, tmp_path.open("wb") as out:

Review Comment:
   agreed here - added a handler that rejects protocol changes before the target request is sent, including redirects to HTTP or FTP. also replaced the stubbed test with a real `HTTPServer` test.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3861263507


##########
api/src/main/java/org/apache/flink/agents/api/skills/Skills.java:
##########
@@ -92,17 +94,94 @@ public static Skills fromLocalDir(String... paths) {
     }
 
     /**
-     * Create a {@link Skills} resource from one or more http(s) URLs.
+     * Create a {@link Skills} resource from one or more HTTPS URLs.
      *
      * <p>Each URL must point to a {@code .zip} whose top level is the baseDir.
      */
     public static Skills fromUrl(String... urls) {
         return new Skills(
                 Arrays.stream(urls)
-                        .map(u -> new SkillSourceSpec("url", Map.of("url", u)))
+                        .map(
+                                u -> {
+                                    requireUrl(u, false);
+                                    return new SkillSourceSpec("url", Map.of("url", u));
+                                })
                         .collect(Collectors.toList()));
     }
 
+    /**
+     * Create a {@link Skills} resource from an HTTPS URL pinned to a SHA-256 digest.
+     *
+     * <p>The digest is verified against the downloaded archive before extraction.
+     */
+    public static Skills fromUrlWithSha256(String url, String sha256) {
+        return urlSource(url, sha256, false);
+    }
+
+    /**
+     * Create a {@link Skills} resource that explicitly permits plain HTTP transport.
+     *
+     * <p>This compatibility escape hatch should be used only on trusted networks. Prefer {@link
+     * #fromUrl(String...)} with HTTPS.
+     */
+    public static Skills fromUrlUnsafe(String... urls) {
+        return new Skills(
+                Arrays.stream(urls)
+                        .map(
+                                u -> {
+                                    requireUrl(u, true);
+                                    return new SkillSourceSpec(
+                                            "url", Map.of("url", u, "allow_insecure_http", "true"));
+                                })
+                        .collect(Collectors.toList()));
+    }
+
+    /**
+     * Create a digest-pinned {@link Skills} resource that explicitly permits plain HTTP transport.
+     */
+    public static Skills fromUrlUnsafeWithSha256(String url, String sha256) {
+        return urlSource(url, sha256, true);
+    }
+
+    private static Skills urlSource(String url, String sha256, boolean allowInsecureHttp) {
+        requireUrl(url, allowInsecureHttp);
+        if (sha256 == null || !sha256.matches("[0-9a-fA-F]{64}")) {
+            throw new IllegalArgumentException(
+                    "sha256 must contain exactly 64 hexadecimal characters");
+        }
+        Map<String, String> params =
+                allowInsecureHttp
+                        ? Map.of("url", url, "sha256", sha256, "allow_insecure_http", "true")
+                        : Map.of("url", url, "sha256", sha256);
+        return new Skills(List.of(new SkillSourceSpec("url", params)));
+    }
+
+    private static void requireUrl(String url, boolean allowInsecureHttp) {
+        if (url == null) {
+            throw new IllegalArgumentException("skill URL must not be null");
+        }
+        URI uri;
+        try {
+            uri = URI.create(url);
+        } catch (IllegalArgumentException e) {
+            throw new IllegalArgumentException("Invalid skill URL: " + url, e);
+        }
+        String scheme = uri.getScheme();
+        scheme = scheme == null ? "" : scheme.toLowerCase(Locale.ROOT);
+        if (!(scheme.equals("http") || scheme.equals("https"))) {
+            throw new IllegalArgumentException("Only HTTP(S) skill URLs are supported: " + url);
+        }
+        if (scheme.equals("http") && !allowInsecureHttp) {
+            throw new IllegalArgumentException(
+                    "Plain HTTP skill URLs are disabled by default; use HTTPS or explicitly allow"
+                            + " insecure HTTP for this source: "
+                            + url);
+        }
+        if (uri.getRawAuthority() == null || uri.getRawAuthority().isEmpty()) {

Review Comment:
   `getRawAuthority()` only checks that the authority string is non-empty, so URLs like `https://:443/x.zip` and `https://example.com:bad/x.zip` are accepted. Python's `parsed.netloc` check has the same issue. Could we validate the actual hostname and port in both implementations?



##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/URLSkillRepository.java:
##########
@@ -18,43 +18,119 @@
 
 package org.apache.flink.agents.runtime.skill.repository;
 
+import javax.annotation.Nullable;
+
 import java.io.IOException;
+import java.io.InputStream;
+import java.net.URI;
+import java.net.URISyntaxException;
 import java.nio.file.Files;
 import java.nio.file.Path;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.util.Locale;
+import java.util.regex.Pattern;
 
 /**
- * Skill repository backed by an http(s) URL pointing to a zip.
+ * Skill repository backed by an HTTPS URL pointing to a zip.
  *
  * <p>The zip is downloaded to a temp file and extracted into a process-local temp directory. The
  * downloaded zip itself is removed once extraction completes; the extracted directory is released
  * via {@link #close()} (cascaded through {@code SkillManager} → {@code ResourceContextImpl} →
  * {@code ResourceCache} on operator close). A JVM shutdown hook acts as fallback cleanup if {@code
- * close()} is never called.
+ * close()} is never called. Plain HTTP is rejected unless the caller explicitly opts in, and an
+ * optional SHA-256 digest is verified before extraction.
  */
 public final class URLSkillRepository extends AbstractMaterializedSkillRepository {
 
     private static final int REQUEST_TIMEOUT_MS = 90_000;
+    private static final Pattern SHA256_PATTERN = Pattern.compile("[0-9a-fA-F]{64}");
 
     private final String url;
 
     public URLSkillRepository(String url) throws IOException {
-        super(materialize(url));
+        this(url, null, false);
+    }
+
+    public URLSkillRepository(String url, @Nullable String sha256, boolean allowInsecureHttp)
+            throws IOException {
+        super(materialize(url, sha256, allowInsecureHttp));
         this.url = url;
     }
 
     public String getUrl() {
         return url;
     }
 
-    private static SkillMaterializer.Materialized materialize(String url) throws IOException {
-        if (!(url.startsWith("http://") || url.startsWith("https://"))) {
-            throw new IllegalArgumentException("Only http(s) URLs are supported: " + url);
+    private static SkillMaterializer.Materialized materialize(
+            String url, @Nullable String sha256, boolean allowInsecureHttp) throws IOException {
+        URI uri;
+        if (url == null) {
+            throw new IllegalArgumentException("skill URL must not be null");
         }
-        Path tmpZip = SkillMaterializer.downloadToTempFile(url, REQUEST_TIMEOUT_MS);
         try {
+            uri = new URI(url);
+        } catch (URISyntaxException e) {
+            throw new IllegalArgumentException("Invalid skill URL: " + url, e);
+        }
+        String scheme = uri.getScheme();
+        scheme = scheme == null ? "" : scheme.toLowerCase(Locale.ROOT);
+        if (!(scheme.equals("http") || scheme.equals("https"))) {
+            throw new IllegalArgumentException("Only HTTP(S) URLs are supported: " + url);
+        }
+        if (scheme.equals("http") && !allowInsecureHttp) {
+            throw new IllegalArgumentException(
+                    "Plain HTTP skill URLs are disabled by default; use HTTPS or explicitly allow"
+                            + " insecure HTTP for this source: "
+                            + url);
+        }
+        if (uri.getRawAuthority() == null || uri.getRawAuthority().isEmpty()) {
+            throw new IllegalArgumentException("Skill URL must include a host: " + url);
+        }
+        String normalizedSha256 = sha256 == null ? null : sha256.toLowerCase(Locale.ROOT);
+        if (normalizedSha256 != null && !SHA256_PATTERN.matcher(normalizedSha256).matches()) {
+            throw new IllegalArgumentException(
+                    "sha256 must contain exactly 64 hexadecimal characters");
+        }
+        Path tmpZip =
+                SkillMaterializer.downloadToTempFile(url, REQUEST_TIMEOUT_MS, allowInsecureHttp);
+        try {
+            if (normalizedSha256 != null) {
+                String actual = sha256(tmpZip);
+                if (!actual.equals(normalizedSha256)) {
+                    throw new IllegalArgumentException(
+                            "SHA-256 mismatch for skill archive "
+                                    + url

Review Comment:
   This error includes the full configured URL, so signed URLs may expose query credentials through logs or exception reports. Python has the same behavior, and both `SkillManager` implementations also include the raw `spec.params`. Could we sanitize or omit URLs in these error paths?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "rob-9 (via GitHub)" <gi...@apache.org>.
rob-9 commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3892221472


##########
api/src/test/java/org/apache/flink/agents/api/skills/SkillsResourceTest.java:
##########
@@ -48,6 +53,174 @@ void fromUrlEmitsUrlScheme() {
                 skills.getSources());
     }
 
+    @Test
+    void fromUrlAcceptsSharedValidHostSyntax() {
+        for (String url :
+                List.of(
+                        "https://localhost/x.zip",
+                        "https://127.0.0.1/x.zip",
+                        "https://[::1]/x.zip",
+                        "https://[fe80::1%25eth0]/x.zip",
+                        "https://example.com./x.zip",
+                        "https://example.com:/x.zip",
+                        "https://999/x.zip",
+                        "https://1bar/x.zip",
+                        "https://999./x.zip")) {

Review Comment:
   added `65535` as a valid port boundary test in both Java and Python.



##########
api/src/main/java/org/apache/flink/agents/api/skills/SkillUrlUtils.java:
##########
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.agents.api.skills;
+
+import org.apache.flink.annotation.Internal;
+
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Locale;
+
+/**
+ * Shared validation and redaction helpers for URL-backed skill sources. Internal contract shared
+ * with the runtime module; not a stable public API.
+ */
+@Internal
+public final class SkillUrlUtils {
+
+    private SkillUrlUtils() {}
+
+    /**
+     * Validate {@code url} and return its lowercase {@code http} or {@code https} scheme.
+     *
+     * @throws IllegalArgumentException if the URL is invalid or violates the transport policy.
+     */
+    public static String validate(String url, boolean allowInsecureHttp) {
+        if (url == null) {
+            throw new IllegalArgumentException("skill URL must not be null");
+        }
+        URI uri;
+        try {
+            uri = URI.create(url);
+        } catch (IllegalArgumentException ignored) {
+            throw new IllegalArgumentException("Invalid skill URL: " + redact(url));
+        }
+        String scheme = uri.getScheme();
+        scheme = scheme == null ? "" : scheme.toLowerCase(Locale.ROOT);
+        if (!(scheme.equals("http") || scheme.equals("https"))) {
+            throw new IllegalArgumentException(
+                    "Only HTTP(S) skill URLs are supported: " + redact(url));
+        }
+        try {
+            uri = uri.parseServerAuthority();
+        } catch (URISyntaxException ignored) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host and, when present, a valid port: "
+                            + redact(url));
+        }
+        if (uri.getRawUserInfo() != null) {
+            throw new IllegalArgumentException(
+                    "Skill URL must not include user info: " + redact(url));
+        }
+        if (uri.getHost() == null || uri.getHost().isEmpty()) {
+            throw new IllegalArgumentException(
+                    "Skill URL must include a valid host: " + redact(url));
+        }
+        if (uri.getPort() > 65535) {
+            throw new IllegalArgumentException(
+                    "Skill URL port must be between 0 and 65535: " + redact(url));
+        }
+        if (scheme.equals("http") && !allowInsecureHttp) {
+            throw new IllegalArgumentException(
+                    "Plain HTTP skill URLs are disabled by default; use HTTPS or explicitly allow"
+                            + " insecure HTTP for this source: "
+                            + redact(url));
+        }
+        return scheme;
+    }
+
+    /** Return {@code url} without user info, query parameters, or a fragment. */
+    public static String redact(String url) {
+        if (url == null) {
+            return "<redacted>";
+        }
+        try {
+            URI uri = URI.create(url);
+            if (uri.getScheme() == null || uri.getRawAuthority() == null) {
+                return "<redacted>";
+            }
+            String authority = uri.getRawAuthority();
+            int userInfoEnd = authority.lastIndexOf('@');
+            if (userInfoEnd >= 0) {
+                authority = authority.substring(userInfoEnd + 1);
+            }
+            if (authority.isEmpty()) {
+                return "<redacted>";
+            }
+            return uri.getScheme() + "://" + authority + uri.getRawPath();
+        } catch (IllegalArgumentException ignored) {
+            return "<redacted>";

Review Comment:
   Matched Python. Preserves safe host/path context while stripping secrets and retaining `<redacted>` for unsafe inputs.



##########
python/flink_agents/api/skills.py:
##########
@@ -57,13 +57,117 @@ def packaged_skills() -> Skills:
 
 from __future__ import annotations
 
+import re
+from ipaddress import AddressValueError, IPv6Address
 from typing import Dict, List, Tuple
+from urllib.parse import urlparse, urlsplit, urlunsplit
 
 from pydantic import BaseModel, ConfigDict, Field, field_validator
 from typing_extensions import override
 
 from flink_agents.api.resource import ResourceType, SerializableResource
 
+_INVALID_URI_CHARACTER = re.compile(r'[\x00-\x20\x7f<>"{}|\\^`]')
+_INVALID_PERCENT_ESCAPE = re.compile(r"%(?![0-9a-fA-F]{2})")
+_HOST_LABEL = re.compile(r"[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?")
+
+
+def redact_skill_url(url: str) -> str:
+    """Return a skill URL without user info, query parameters, or a fragment.
+
+    Internal contract shared with the runtime; not a stable public API.
+    """
+    try:
+        parts = urlsplit(url)
+        if not parts.scheme or not parts.netloc:
+            return "<redacted>"
+        netloc = parts.netloc.rsplit("@", 1)[-1]
+        if not netloc:
+            return "<redacted>"
+        return urlunsplit((parts.scheme, netloc, parts.path, "", ""))
+    except ValueError:
+        return "<redacted>"
+
+
+def validate_skill_url(url: str, *, allow_insecure_http: bool) -> str:
+    """Validate a skill URL using the contract shared with the Java API.
+
+    Internal contract shared with the runtime; not a stable public API.
+    """
+    if not isinstance(url, str):
+        msg = "skill URL must be a string"
+        raise TypeError(msg)
+    try:
+        parsed = urlparse(url)
+    except ValueError:
+        msg = f"Invalid skill URL: {redact_skill_url(url)}"
+        raise ValueError(msg) from None
+    if _INVALID_URI_CHARACTER.search(url) or _INVALID_PERCENT_ESCAPE.search(url):
+        msg = f"Invalid skill URL: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    # Java's URI rejects raw brackets in the path (but not in the query or
+    # fragment); encoded %5B/%5D and IPv6 authority brackets stay valid.
+    if any(c in f"{parsed.path};{parsed.params}" for c in "[]"):
+        msg = f"Invalid skill URL: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    scheme = parsed.scheme.lower()
+    if scheme not in {"http", "https"}:
+        msg = f"Only HTTP(S) skill URLs are supported: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    try:
+        hostname = parsed.hostname
+        _ = parsed.port
+    except ValueError:
+        msg = (
+            "Skill URL must include a valid host and, when present, a valid port: "
+            f"{redact_skill_url(url)}"
+        )
+        raise ValueError(msg) from None
+    if parsed.username is not None:
+        msg = f"Skill URL must not include user info: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    bracketed_host = parsed.netloc.rsplit("@", 1)[-1].startswith("[")
+    if (
+        not hostname
+        or (bracketed_host and ":" not in hostname)
+        or not _is_valid_hostname(hostname)
+    ):
+        msg = f"Skill URL must include a valid host: {redact_skill_url(url)}"
+        raise ValueError(msg)
+    if scheme == "http" and not allow_insecure_http:
+        msg = (
+            "Plain HTTP skill URLs are disabled by default; use HTTPS or "
+            "explicitly allow insecure HTTP for this source: "
+            f"{redact_skill_url(url)}"
+        )
+        raise ValueError(msg)
+    return scheme
+
+
+def _is_valid_hostname(hostname: str) -> bool:
+    """Match the host syntax accepted by Java URI.parseServerAuthority()."""
+    if ":" in hostname:
+        try:
+            IPv6Address(hostname)
+        except AddressValueError:
+            return False
+        return True
+    if not hostname.isascii():

Review Comment:
   Aligned both languages to reject raw Unicode authorities and raw `%` IPv6 zones while accepting encoded `%25` zones.



##########
runtime/src/test/java/org/apache/flink/agents/runtime/skill/URLSkillRepositoryTest.java:
##########
@@ -97,30 +106,166 @@ void loadFromUrl(@TempDir Path tempDir) throws IOException {
         }
     }
 
+    @Test
+    void plainHttpRejectedByDefault() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> new URLSkillRepository("http://example.com/skills.zip"));
+        assertTrue(ex.getMessage().contains("disabled by default"));
+    }
+
+    @Test
+    void sha256MismatchRejectedBeforeExtraction(@TempDir Path tempDir) throws IOException {
+        Path zip = tempDir.resolve("skills.zip");
+        try (ZipOutputStream zos = new ZipOutputStream(Files.newOutputStream(zip))) {
+            zos.putNextEntry(new ZipEntry("../evil.txt"));
+            zos.write("pwn".getBytes(StandardCharsets.UTF_8));
+            zos.closeEntry();
+        }
+        HttpServer server = startZipServer(Files.readAllBytes(zip), 200);
+        try {
+            int port = server.getAddress().getPort();
+            String url = "http://127.0.0.1:" + port + "/skills.zip?token=top-secret#fragment";
+            IllegalArgumentException ex =
+                    assertThrows(
+                            IllegalArgumentException.class,
+                            () -> new URLSkillRepository(url, "0".repeat(64), true));
+            assertTrue(ex.getMessage().contains("SHA-256 mismatch"));
+            assertTrue(ex.getMessage().contains("http://127.0.0.1:" + port + "/skills.zip"));
+            assertFalse(ex.getMessage().contains("top-secret"));
+        } finally {
+            server.stop(0);
+        }
+    }
+
+    @Test
+    void invalidHostAndPortAreRejectedBeforeDownload() {
+        IllegalArgumentException malformedPort =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () ->
+                                new URLSkillRepository(
+                                        "https://example.com:bad/skills.zip?token=top-secret"));
+        assertNull(malformedPort.getCause());
+        assertTrue(malformedPort.getMessage().contains("https://example.com:bad/skills.zip"));
+        assertFalse(malformedPort.getMessage().contains("top-secret"));
+
+        for (String url :
+                List.of("https://:443/skills.zip", "https://example.com:65536/skills.zip")) {
+            assertThrows(IllegalArgumentException.class, () -> new URLSkillRepository(url), url);
+        }
+    }
+
+    @Test
+    void invalidHostnameSyntaxIsRejectedBeforeDownload() {
+        for (String url :
+                List.of(
+                        "https://exa_mple.com/skills.zip",
+                        "https://tést.com/skills.zip",
+                        "https://%65xample.com/skills.zip",
+                        "https://-example.com/skills.zip",
+                        "https://example-.com/skills.zip",
+                        "https://.example.com/skills.zip",
+                        "https://example..com/skills.zip",
+                        "https://a../skills.zip",
+                        "https://../skills.zip",
+                        "https://999.999.999.999/skills.zip",
+                        "https://127.1/skills.zip",
+                        "https://1.2.3/skills.zip",
+                        "https://foo.123/skills.zip",
+                        "https://foo.1bar/skills.zip",
+                        "https://1.2.3.4.5/skills.zip",
+                        "https://1.2.3./skills.zip",
+                        "https://1.2.3.4./skills.zip",
+                        "https://[v1.foo]/skills.zip")) {
+            assertThrows(IllegalArgumentException.class, () -> new URLSkillRepository(url), url);
+        }
+    }
+
+    @Test
+    void rawPercentEscapeIsRejectedByDownloader() {

Review Comment:
   Fixed by rejecting this URL in the shared Java validator before networking and asserting its exact exception message.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [api][runtime][java][python] Require HTTPS and add digest pinning for URL skill sources [flink-agents]

Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #1005:
URL: https://github.com/apache/flink-agents/pull/1005#discussion_r3781343774


##########
runtime/src/main/java/org/apache/flink/agents/runtime/skill/repository/SkillMaterializer.java:
##########
@@ -254,14 +254,37 @@ private static void copyJarEntries(URL jarUrl, String prefix, Path extractDir)
      * @throws IOException on connect / read failures or HTTP error responses.
      */
     public static Path downloadToTempFile(String url, int timeoutMs) throws IOException {
+        return downloadToTempFile(url, timeoutMs, false);
+    }
+
+    /**
+     * Download {@code url}, optionally permitting plain HTTP transport.
+     *
+     * @throws IOException on connect / read failures or HTTP error responses.
+     */
+    public static Path downloadToTempFile(String url, int timeoutMs, boolean allowInsecureHttp)
+            throws IOException {
         URL u = new URL(url);
+        String initialProtocol = u.getProtocol();
+        if (!("https".equalsIgnoreCase(initialProtocol)
+                || (allowInsecureHttp && "http".equalsIgnoreCase(initialProtocol)))) {
+            throw new IOException("Skill URL uses a disallowed transport: " + url);
+        }
         HttpURLConnection conn = (HttpURLConnection) u.openConnection();
         conn.setConnectTimeout(timeoutMs);
         conn.setReadTimeout(timeoutMs);
         conn.setRequestMethod("GET");
         Path tmpZip = Files.createTempFile(TEMP_DIR_PREFIX, ".zip");
-        try (InputStream in = conn.getInputStream()) {
-            Files.copy(in, tmpZip, StandardCopyOption.REPLACE_EXISTING);
+        try {
+            int responseCode = conn.getResponseCode();
+            if (responseCode >= 300 && responseCode < 400) {

Review Comment:
   This catches a redirect that changes protocol, but a same-protocol redirect is followed silently and the final URL is never checked. Java only looks at the protocol of the configured URL (`:267-272`), never at `conn.getURL()`; Python compares just the scheme of `resp.geturl()` (`_materialize.py:189-196`), not the host.
   
   So for an unpinned HTTPS source, an open redirect at the configured host can pull the archive from somewhere else, and nothing records it — `SkillOrigin` (`SkillSourceRegistry.java:56`) still reports the URL that was configured. Since the archive becomes agent instructions and possibly scripts, where the bytes actually came from seems worth surfacing.
   
   Pinning a `sha256` does catch this. Is that the intended answer, or would logging the final URL when it differs be worth adding?
   
   One related note: the protection against a scheme change here is real but implicit, coming from the JDK declining cross-protocol redirects rather than from this code. A later move to `java.net.http.HttpClient` would reverse it, since its default policy follows those. Might be worth a comment naming that.



##########
python/flink_agents/runtime/skill/repository/_materialize.py:
##########
@@ -159,8 +167,31 @@ def download_to_tempfile(url: str, timeout: int = 90) -> Path:
     tmp_path = Path(tmp_path_str)
     try:
         with urlopen(req, timeout=timeout) as resp, tmp_path.open("wb") as out:

Review Comment:
   `urlopen` here uses the default opener, and its redirect handler allows `http`, `https` and `ftp` targets. So when an HTTPS source redirects to `http://…`, Python performs the plaintext GET and only rejects it afterwards at `:191-193`. An `ftp://` location gets dialled too, since `FTPHandler` is installed by default.
   
   No archive bytes are written, so the content is never trusted. But the request itself, URL and headers included, does travel over the downgraded connection. Java never gets that far — `HttpURLConnection` refuses the cross-protocol redirect, so the second request is never made.
   
   The test covering this (`test_materialize.py:164`) swaps `urlopen` for a stub that reports a different `geturl()`, so it checks the string comparison rather than urllib's redirect handling. This case would slip past it.
   
   What do you think about declining the redirect in a custom opener, so the request is never issued? A real `HTTPServer` returning a 302 to an `http://` location, the way `SkillMaterializerTest.java:153` does on the Java side, would cover it end to end.



##########
docs/content/docs/development/yaml.md:
##########
@@ -208,13 +208,14 @@ agents:
             key: tenant_id
 ```
 
-**Skills** — bundles of agent skill assets loaded from one or more sources. At least one of `paths` / `urls` / `classpath` / `package` must be non-empty; multiple sources can coexist.
+**Skills** — bundles of agent skill assets loaded from one or more sources. At least one of `paths` / `urls` / `url_sources` / `classpath` / `package` must be non-empty; multiple sources can coexist.
 
 | Field | Required | Description |
 |-------|----------|-------------|
 | `name` | yes | Skills resource name. |
 | `paths` | one-of | `local` scheme: list of directories or `.zip` files. |
-| `urls` | one-of | `url` scheme: list of `http(s)` URLs pointing to `.zip` archives. |
+| `urls` | one-of | `url` scheme: list of HTTPS URLs pointing to `.zip` archives. |

Review Comment:
   `urls: [http://…]` worked before this PR and now fails, and neither doc says what to switch to. The answer also differs by surface: in code there's `from_url_unsafe` / `fromUrlUnsafe` (`skills.md:145`), but `urls` has no opt-in at all — the entry has to move into `url_sources` with `allow_insecure_http: true`. That's a different shape rather than a flag, so it's the harder one to guess from the error alone.
   
   Something like this after the row, if it helps: "`urls` entries must be HTTPS. To keep an existing plain-HTTP source working, move it to `url_sources` with `allow_insecure_http: true`."
   
   Does that feel like doc material, or more of a release-note thing?



##########
runtime/src/test/java/org/apache/flink/agents/runtime/skill/URLSkillRepositoryTest.java:
##########
@@ -97,19 +100,57 @@ void loadFromUrl(@TempDir Path tempDir) throws IOException {
         }
     }
 
+    @Test
+    void plainHttpRejectedByDefault() {
+        IllegalArgumentException ex =
+                assertThrows(
+                        IllegalArgumentException.class,
+                        () -> new URLSkillRepository("http://example.com/skills.zip"));
+        assertTrue(ex.getMessage().contains("disabled by default"));
+    }
+
+    @Test
+    void sha256MismatchRejectedBeforeExtraction(@TempDir Path tempDir) throws IOException {

Review Comment:
   The name says `BeforeExtraction`, but the body only checks the exception and its message. If the digest check moved to after `extractZipSafely`, the archive would extract, the same `IllegalArgumentException` with the same `"SHA-256 mismatch"` text would still be thrown, and this test would stay green. Python's `test_sha256_mismatch_rejected` (`test_url_repository.py:103`) has the same gap, just without the name promising more.
   
   Checking before extraction is the property #1003 actually asks for, so a test that turns red when it changes seems worth having. One option: serve an archive that extraction itself would reject — the zip-slip fixture at `SkillMaterializerTest.java:72` — and assert the failure is the digest error rather than `"Unsafe zip entry"`. Flip the order and you would get the zip-slip error instead.
   
   Any reason that wouldn't work here?



##########
api/src/main/java/org/apache/flink/agents/api/yaml/YamlLoader.java:
##########
@@ -173,8 +174,19 @@ public static Skills buildSkills(SkillsSpec spec) {
         for (String p : spec.getPaths()) {
             sources.add(new SkillSourceSpec("local", Map.of("path", p)));
         }
-        for (String u : spec.getUrls()) {
-            sources.add(new SkillSourceSpec("url", Map.of("url", u)));
+        for (String url : spec.getUrls()) {
+            sources.add(new SkillSourceSpec("url", Map.of("url", url)));

Review Comment:
   This builds the source without running the URL check, and `loader.py:208-210` does the same. So `urls: [http://x/s.zip]` parses cleanly and only fails later on the TaskManager, while `Skills.fromUrl("http://…")` fails right away. A malformed `sha256` under `url_sources` behaves the same way.
   
   Both languages do this identically, so it isn't a parity issue. It's that the same mistake surfaces at very different moments depending on how you configure it, and the YAML user finds out last. Nothing on either side tests that a YAML `http://` entry is rejected, either.
   
   Was that deliberate, keeping the loaders out of transport policy, or did it just work out that way?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org