You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2023/03/02 05:07:05 UTC
[GitHub] [beam] dependabot[bot] opened a new pull request, #25689: Bump github.com/tetratelabs/wazero from 1.0.0-pre.9 to 1.0.0-rc.1 in /sdks
dependabot[bot] opened a new pull request, #25689:
URL: https://github.com/apache/beam/pull/25689
Bumps [github.com/tetratelabs/wazero](https://github.com/tetratelabs/wazero) from 1.0.0-pre.9 to 1.0.0-rc.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/tetratelabs/wazero/releases">github.com/tetratelabs/wazero's releases</a>.</em></p>
<blockquote>
<p>wazero v1.0.0-rc.1 starts our journey towards 1.0, with no public API changes. We will have at least one more release candidate between now and 1.0, in a week or two. The next candidate will include CLI binaries, which will allow our non-Go community to use wazero without installing Go, first.</p>
<p>wazero 1.0 will happen March 24th, during the wazero release party at <a href="https://wasmio.tech/">wasmio</a> hosted by <a href="https://tetrate.io/">Tetrate</a>. Follow <a href="https://evacchi.github.io/">Edoardo</a> for updates on on the party, who's doing great job organizing including lightning talks by end users.</p>
<p>Many of you have requested a community page on our website. We added <a href="https://wazero.io/community/">this</a> including a list of <a href="https://wazero.io/community/users/">users</a> who explicitly opted in. The user list is opt-in, and generally higher signal than the "Used By" list on the <a href="https://github.com/tetratelabs/wazero">GitHub site</a>, as the latter includes transient dependencies.</p>
<p>The biggest internal code change in v1.0.0-rc.1 let to wazero passing the entire wasi-testsuite on linux, darwin (macOS) and windows, with zero exceptions. To do so took tens of days of effort from both <a href="https://github.com/codefromthecrypt"><code>@codefromthecrypt</code></a> and <a href="https://github.com/mathetake"><code>@mathetake</code></a>, and described at the bottom for the curious. By passing every test defined by WebAssembly, as well stdlib tests in TinyGo and Zig, it is easier for end users to feel confident wazero is a great choice for stability. This is worth the couple weeks of pain.</p>
<p>We intentionally didn't do any other large changes, but there were several people to thank for minor changes. Some of the below took many days of effort each!</p>
<ul>
<li><a href="https://github.com/achille-roussel"><code>@achille-roussel</code></a> for refactoring an internal type with generics so we can use it for open directories.</li>
<li><a href="https://github.com/ckaznocha"><code>@ckaznocha</code></a> for fixing a concurrency issue on context cancellation.</li>
<li><a href="https://github.com/mathetake"><code>@mathetake</code></a> for a mountain of optimizations to reduce compilation overhead</li>
<li><a href="https://github.com/dmvolod"><code>@dmvolod</code></a> for removing a redundant wasm decoding validation.</li>
<li><a href="https://github.com/evacchi"><code>@evacchi</code></a> for adding <code>-timeout duration</code> to the CLI which stops runaway processes.</li>
<li><a href="https://github.com/evacchi"><code>@evacchi</code></a> for fixing a corner case around max memory limit</li>
<li><a href="https://github.com/mathetake"><code>@mathetake</code></a> for allowing the CLI to be built with an external version (for packaging)</li>
</ul>
<p>In closing, thanks very much for sticking with us this last year and a half almost, leading to 1.0. Our fantastic ecosystem is the reason you have a zero dependency runtime, something you can embed without thinking about work or version clashes with other tools. We've done great work together to bring Go forward in the WebAssembly ecosystem, and people are noticing!</p>
<h3>Notes on wasi-testsuite compliance</h3>
<p>Most people don't need to read this part, so only do if you are interested in low level details, or bored!</p>
<p><a href="https://github.com/WebAssembly/wasi-testsuite">wasi-testsuite</a> compliance means passing tests that verify the expected behaviors of <a href="https://github.com/WebAssembly/WASI/blob/snapshot-01/phases/snapshot/docs.md">wasi_snapshot_preview1</a>, as decided by the <a href="https://github.com/WebAssembly/WASI/blob/main/Charter.md">WASI W3C subgroup</a>. It does so by running tests compiled in different languages, currently AssemblyScript, C and Rust.</p>
<p>As described at the beginning of the release notes, wazero v1.0.0-rc.1 passes all tests as doing so is least confusing to the ecosystem. This means passing things we don't advise or agree with. Our former release left out a couple tests due to performance overhead, but we pass them now despite it. If you care about this, read on!</p>
<p>The last pre-release skipped a test on dot and dot-dot entries in directory listings. <a href="https://github.com/golang/go/blob/master/src/os/dir_unix.go#L107-L110">Go throws away these entries</a> before we can read them. To resurrect them costs tens of microseconds. The cost is fixed overhead around inodes, but this same topic has a large performance footprint when multiplied in directory listings.</p>
<p>Recently, a test was added to require inode data (file serial numbers) inside all directory listings. For example, if your directory includes "wazero.txt", a non-zero inode must be returned</p>
<p>inode data isn't typically used except comparing file identity, and even then requires the device to do so properly. For example, Go has a <a href="https://pkg.go.dev/os#SameFile">SameFile</a> function which lazily gets this data, as it is well.. expensive. wasi-libc <a href="https://github-redirect.dependabot.com/WebAssembly/wasi-libc/pull/345">made a change</a> recently to fetch this data regardless of it it is used or not. Not all compilers use wasi-libc, for example Zig has their own directory logic. However, at least C and Rust do, so mitigating this problem became an issue for us, not just for the spectest, but the underlying logic in wasi-libc. Specifically, compilers that use wasi-libc 17+ will perform a guest side stat fan-out when the ino returned is zero. Doing a guest-side fan out is much more expensive than host side.</p>
<p>This problem is roughly analogous to <a href="https://pkg.go.dev/os#File.ReadDir">ReadDir</a> vs <a href="https://pkg.go.dev/os#File.Readdir">Readdir</a> problem in Go. The lower-d version says "Most clients are better served by the more efficient ReadDir method", ultimately due to an internal stat fanout in worst case. Unfortunately, the performance impact is worse than upper vs lower d readdir in go. In windows, the inode information isn't in the <code>FileInfo.Sys</code> data, so there's an additional N syscalls to get it from somewhere else.</p>
<p>System performance isn't consistent, but compliance with WASI on this is at least in tens of microseconds additional overhead on directory listings, and it is also linear wrt directory size. Not all users will list directories at after configuration time, neither will all find this overhead intolerable even if they did.</p>
<p>In case you are curious, we did discuss these topics at length with the WASI maintainers. The end of it is that this performance overhead is something they feel as a least bad option of choices before them. That said, progress was made, as they changed the next specification, <a href="https://github.com/WebAssembly/wasi-filesystem">wasi-filesystem</a> to be performant by default, both not requiring dot directories or eager inodes. We're grateful to have been listened to.</p>
<p>If your performance after initialization time is dominated by this, you may of course file an issue to request a non-strict wasi setting. That said, we'd prefer to not make that setting. We have near term plans, likely this summer, to make the whole filesystem behaviour pluggable. In other words, avoiding this cost will be something you can choose to do on your own later, even for the current WASI specs.</p>
<p>If you'd like to discuss more on this or anything else, jump on <a href="https://gophers.slack.com/">gophers slack</a> <code>#wazero</code> channel. Note: You may need an <a href="https://invite.slack.golangbridge.org/">invite</a> to join gophers. If you like what we are doing, please <a href="https://github.com/tetratelabs/wazero/stargazers">star</a> our repo as folks appreciate it.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/tetratelabs/wazero/commit/e77f24fe311332681fff10f34c35f8af5fda8ca4"><code>e77f24f</code></a> sysfs: drops os.File special casing for fs.FS to pass wasi-testsuite (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1174">#1174</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/4fd3d9d6e2ea2626c41470a56b01033269ec26d7"><code>4fd3d9d</code></a> Adds scale.sh to users.md (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1179">#1179</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/5ffc3f61fd5a831188d1388ad46cd10c254c6f17"><code>5ffc3f6</code></a> ci: artifacts creation check for non windows platforms (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1178">#1178</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/5598e491f74c372ccb51d80451a930a19029800d"><code>5598e49</code></a> Add timeout support in <code>wazero run</code> cli (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1173">#1173</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/9c07b2793df58e321822106e39025b5ba07f0742"><code>9c07b27</code></a> Allows to set the version of CLI (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1176">#1176</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/599e01b65a9bb009e4a8a436d9a026665b9f6b22"><code>599e01b</code></a> fuzz: update README instructions (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1175">#1175</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/3d5b6d609a67b7542d6ce2aca4c6dce09a463de4"><code>3d5b6d6</code></a> implements lstat and fixes inode stat on windows go 1.20 (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1168">#1168</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/d955cd7a13799bb9c8c7a6d8db09a188fa882ebf"><code>d955cd7</code></a> windows: fixes unlinking symlink to dir, rmdir on opened empty dir. (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1172">#1172</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/75aa6b2a6efa86b20e9b4eecd2889c83b714b1ca"><code>75aa6b2</code></a> examples: updates to the latest SDKs (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1169">#1169</a>)</li>
<li><a href="https://github.com/tetratelabs/wazero/commit/5eab1a7307a332d637993e07deb57b203d613866"><code>5eab1a7</code></a> compiler: pass runtimeValueLocationStack by values to reduce allocations (<a href="https://github-redirect.dependabot.com/tetratelabs/wazero/issues/1170">#1170</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/tetratelabs/wazero/compare/v1.0.0-pre.9...v1.0.0-rc.1">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #25689: Bump github.com/tetratelabs/wazero from 1.0.0-pre.9 to 1.0.0-rc.1 in /sdks
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #25689:
URL: https://github.com/apache/beam/pull/25689#issuecomment-1456287844
Run Go PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse merged pull request #25689: Bump github.com/tetratelabs/wazero from 1.0.0-pre.9 to 1.0.0-rc.1 in /sdks
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse merged PR #25689:
URL: https://github.com/apache/beam/pull/25689
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25689: Bump github.com/tetratelabs/wazero from 1.0.0-pre.9 to 1.0.0-rc.1 in /sdks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25689:
URL: https://github.com/apache/beam/pull/25689#issuecomment-1451325716
Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @riteshghorse for label go.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #25689: Bump github.com/tetratelabs/wazero from 1.0.0-pre.9 to 1.0.0-rc.1 in /sdks
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #25689:
URL: https://github.com/apache/beam/pull/25689#issuecomment-1451313014
# [Codecov](https://codecov.io/gh/apache/beam/pull/25689?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#25689](https://codecov.io/gh/apache/beam/pull/25689?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4e66743) into [master](https://codecov.io/gh/apache/beam/commit/e1491a637db0389eed5089c63aabc0dd9a97b249?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e1491a6) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #25689 +/- ##
==========================================
- Coverage 72.81% 72.81% -0.01%
==========================================
Files 775 775
Lines 102928 102928
==========================================
- Hits 74950 74949 -1
Misses 26525 26525
- Partials 1453 1454 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| go | `53.96% <ø> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/25689?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...o/pkg/beam/runners/prism/internal/worker/worker.go](https://codecov.io/gh/apache/beam/pull/25689?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL3ByaXNtL2ludGVybmFsL3dvcmtlci93b3JrZXIuZ28=) | `63.42% <0.00%> (-1.17%)` | :arrow_down: |
| [.../beam/runners/prism/internal/jobservices/server.go](https://codecov.io/gh/apache/beam/pull/25689?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9nby9wa2cvYmVhbS9ydW5uZXJzL3ByaXNtL2ludGVybmFsL2pvYnNlcnZpY2VzL3NlcnZlci5nbw==) | `76.00% <0.00%> (+8.00%)` | :arrow_up: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org