You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/06/12 05:32:53 UTC
[PR] Azure: Reduce allocations in ADLSLocation URI parsing [iceberg]
wombatu-kun opened a new pull request, #16788:
URL: https://github.com/apache/iceberg/pull/16788
ADLSLocation is constructed once per FileIO operation (newInputFile, newOutputFile, deleteFile, listPrefix, deletePrefix all call new ADLSLocation(path)), so its constructor is on the metadata path of every scan and commit.
The constructor split the authority with String.split("@", -1) and then extracted the storage account with host.split("\\.", -1)[0]. The second split is the wasteful part: to read the account it materializes every dotted segment of the host (account.dfs.core.windows.net produces 5 strings plus the backing array) and then discards all but the first. This change parses the same fields with indexOf/substring, dropping the intermediate ArrayList, array, and unused substrings. The regex match is kept for scheme validation, so all parsing and error behavior is unchanged.
Behavior is identical for every valid abfs/abfss/wasb/wasbs URI (zero or one @, host with or without dots); only pathological multi-@ authorities, which are already invalid, would differ. The storage-account extraction now guards the no-dot case (indexOf returns -1) explicitly, which a new test pins.
Local JMH results (AverageTime, 5x1s warmup, 5x1s measurement, 1 fork, gc profiler; gc.alloc.rate.norm is deterministic):
| URI shape | ns/op before | ns/op after | B/op before | B/op after |
| --- | --- | --- | --- | --- |
| container@account.dfs.core.windows.net/... | 441 | 254 | 1072 | 656 |
| account.dfs.core.windows.net/... | 381 | 240 | 864 | 504 |
That is roughly -40% allocation and about 1.6x faster per parse. The benchmark was run locally and is not included (the azure module has no JMH source set).
Tests: TestADLSLocation already covers the abfs/abfss/wasb/wasbs, with/without container, no-path and host-extraction cases; added testHostWithoutDot to pin the no-dot storage-account branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Azure: Reduce allocations in ADLSLocation URI parsing [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16788:
URL: https://github.com/apache/iceberg/pull/16788#issuecomment-5610715636
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Azure: Reduce allocations in ADLSLocation URI parsing [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16788:
URL: https://github.com/apache/iceberg/pull/16788#issuecomment-5234611401
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Azure: Reduce allocations in ADLSLocation URI parsing [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16788:
URL: https://github.com/apache/iceberg/pull/16788#issuecomment-6103789150
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Azure: Reduce allocations in ADLSLocation URI parsing [iceberg]
Posted by "wombatu-kun (via GitHub) via issues" <is...@iceberg.apache.org>.
wombatu-kun closed pull request #16788: Azure: Reduce allocations in ADLSLocation URI parsing
URL: https://github.com/apache/iceberg/pull/16788
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org