[PR] [oss] Add fs.oss.cname.enabled so non-Aliyun endpoints sign the bucket [paimon]

["PDGGK (via GitHub)" <gi...@apache.org>]

PDGGK opened a new pull request, #9238:
URL: https://github.com/apache/paimon/pull/9238

   ### Purpose
   
   Closes #8852.
   
   `OSSFileIO` gives no way to switch off the OSS SDK's CNAME heuristic, and on an OSS-compatible endpoint that is not an Aliyun domain that heuristic drops the bucket out of the host being signed. Every request then fails with `SignatureDoesNotMatch`.
   
   The path, in `aliyun-sdk-oss` 3.17.4 — the version this module pins:
   
   ```java
   // OSSUtils.buildCanonicalHost
   boolean isCname = false;
   if (clientConfig.isSupportCname()) {
       isCname = cnameExcludeFilter(host, clientConfig.getCnameExcludeList());
   }
   if (bucket != null && !isCname && !clientConfig.isSLDEnabled()) {
       cannonicalHost.append(bucket).append(".").append(host);
   } else {
       cannonicalHost.append(host);          // <- bucket never reaches the signed host
   }
   ```
   
   `cnameExcludeFilter` returns `false` only when the host ends with an entry of the exclude list, which defaults to `aliyuncs.com, aliyun-inc.com, aliyun.com`. Any other host — a private-cloud or Apsara Stack deployment, a MinIO-style gateway — therefore takes the `isCname = true` branch. `supportCname` defaults to `true` in `ClientConfiguration:90`, and `hadoop-aliyun` 3.3.4 builds a bare `new ClientConfiguration()` (`AliyunOSSFileSystemStore:99`) with no reference to cname anywhere in that artifact, so there is currently nothing a user can set to opt out.
   
   ### What changes
   
   One new key, `fs.oss.cname.enabled`, handled the same way `fs.oss.sld.enabled` already is:
   
   ```java
   if (!hadoopOptions.getBoolean(OSS_CNAME_ENABLED, true)) {
       disableCname(fs);
   }
   ```
   
   `disableCname` reaches the client's `ClientConfiguration` through the existing `getOssClient(fs)` +
   `ReflectionUtils.getPrivateFieldValue(ossClient, "serviceClient")` pair that `enableSecondLevelDomain` uses, and calls `setSupportCname(false)`.
   
   Setting it after `fs.initialize(...)` is enough because the flag is read per request, not captured at construction: `OSSRequestMessageBuilder:170` fetches `innerClient.getClientConfiguration()` while building each request and hands it to `determineFinalEndpoint` on line 177. That is the same object — `ServiceClient.getClientConfiguration()` returns the instance it was constructed with, and `OSSClient.getClientConfiguration()` (`:426`) is a straight delegation to it. The neighbouring `isSLDEnabled()` is consumed one line later, on 178, so the existing `fs.oss.sld.enabled` path already depends on exactly this.
   
   ### Blast radius
   
   Unset means unchanged. The default is `true`, matching the SDK's own default, so a catalog that does not set the key builds precisely the client it built before, and a public-cloud endpoint is unaffected in either position of the flag — its host is on the exclude list, so the heuristic never fires. The second test below pins that.
   
   ### Tests
   
   Two, both asserting the host the signer actually computes rather than that a setter was called — `signedHost` calls `OSSUtils.determineFinalEndpoint` with the client's own configuration, which is the call `OSSRequestMessageBuilder` makes for every request.
   
   | endpoint | before | after `setSupportCname(false)` |
   |---|---|---|
   | `oss-cn-x.inter.env99.example.com` | `oss-cn-x.inter.env99.example.com` | `my-bucket.oss-cn-x.inter.env99.example.com` |
   | `oss-cn-hangzhou.aliyuncs.com` | `my-bucket.oss-cn-hangzhou.aliyuncs.com` | `my-bucket.oss-cn-hangzhou.aliyuncs.com` |
   
   The first row is the bug — a bucket that never reaches the host, so the client signs one string and the server another. The second is the blast-radius check.
   
   Making `setSupportCname` a no-op fails exactly one of the 17 tests in the class, and it is the first row:
   
   ```
   [ERROR] OSSFileIOTest.testDisableCnameRestoresTheBucketInTheSignedHost:563
   expected: "my-bucket.oss-cn-x.inter.env99.example.com"
    but was: "oss-cn-x.inter.env99.example.com"
   [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0
   ```
   
   The public-cloud test stays green under that mutation, which is what makes it a blast-radius check rather than a second copy of the first.
   
   **One thing worth knowing, because it caught me out while writing the test.** My first version built the client with `new OSSClientBuilder().build(endpoint, "ak", "sk")` and the first assertion failed — the bucket was *already* in the host. `ClientBuilderConfiguration extends ClientConfiguration` and sets `supportCname = false` in its constructor, so the SDK's own builder opts out of this heuristic by default. It is specifically the bare `new ClientConfiguration()` that `hadoop-aliyun` uses which leaves it on. The test therefore constructs the client the way `AliyunOSSFileSystemStore` does; a test written against `OSSClientBuilder` would pass without the fix and prove nothing.
   
   I have no private-cloud OSS deployment to point at, so what is verified here is the host string the SDK builds for signing, not an end-to-end request against such an endpoint. That is the step the reporter identified as the failure, and it is the step this change moves.
   
   ### API and Format
   
   No change to any existing key, signature or on-disk format. One added configuration key, inert unless set to `false`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [oss] Add fs.oss.cname.enabled so non-Aliyun endpoints sign the bucket [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #9238:
URL: https://github.com/apache/paimon/pull/9238


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org