You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/26 12:02:36 UTC
[GitHub] [beam] Deleplace opened a new issue, #22447: [Bug]: TestParDo timing out on OSX (arm64)
Deleplace opened a new issue, #22447:
URL: https://github.com/apache/beam/issues/22447
### What happened?
Running the full go test suite in `sdk` with `go test ./...` fails on my mac.
Here is a minimal repro:
```
% cd sdks/go/pkg/beam/core/runtime/exec
% go test -v -run TestParDo
=== RUN TestParDo
panic: test timed out after 10m0s
goroutine 13 [running]:
testing.(*M).startAlarm.func1()
/usr/local/go/src/testing/testing.go:2029 +0x8c
created by time.goFunc
/usr/local/go/src/time/sleep.go:176 +0x3c
goroutine 1 [chan receive, 9 minutes]:
testing.(*T).Run(0x140002e81a0, {0x1025ef4b3?, 0x25907902ec08c?}, 0x10284ffa8)
/usr/local/go/src/testing/testing.go:1487 +0x314
testing.runTests.func1(0x0?)
/usr/local/go/src/testing/testing.go:1839 +0x74
testing.tRunner(0x140002e81a0, 0x140001afcb8)
/usr/local/go/src/testing/testing.go:1439 +0x110
testing.runTests(0x140001d4500?, {0x102c4cbc0, 0x2f, 0x2f}, {0x9800000000000000?, 0x1020e2ff8?, 0x102c57880?})
/usr/local/go/src/testing/testing.go:1837 +0x3f0
testing.(*M).Run(0x140001d4500)
/usr/local/go/src/testing/testing.go:1719 +0x500
main.main()
_testmain.go:173 +0x1d0
goroutine 6 [runnable]:
reflect.(*rtype).NumOut(0x10274aee0?)
/usr/local/go/src/reflect/type.go:1040 +0xb8
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.sumFn(0x8?, 0x0?, {0x1400003c408?, 0x1?, 0x14000192628?}, 0x140002e6d50, 0x140002e6e10, 0x140002e6c30)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/pardo_test.go:41 +0x88
reflect.Value.call({0x102788ac0?, 0x1028500a8?, 0x90?}, {0x1025eaaec, 0x4}, {0x140000e0ab0, 0x6, 0x10226c224?})
/usr/local/go/src/reflect/value.go:556 +0x5e4
reflect.Value.Call({0x102788ac0?, 0x1028500a8?, 0x102fe0101?}, {0x140000e0ab0, 0x6, 0x6})
/usr/local/go/src/reflect/value.go:339 +0x98
github.com/apache/beam/sdks/v2/go/pkg/beam/core/util/reflectx.(*reflectFunc).Call(0x1400000ed80, {0x14000095620?, 0x140000b4dc8?, 0x102593e78?})
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/util/reflectx/call.go:87 +0x6c
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*invoker).initCall.func46({0x0, 0x0, 0x0, 0x0, 0x0}, {0x102c3f4c0, 0x1, 0x1}, 0x140002e6c30?)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/fn_arity.go:307 +0x68
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*invoker).Invoke(0x1400019c900, {0x10285b540?, 0x140002ea440?}, {0x0, 0x0, 0x0, 0x0, 0x0}, {0x102c3f4c0, 0x1, ...}, ...)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/fn.go:236 +0x8b0
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).invokeProcessFn(0x140001b60f0, {0x10285b540, 0x140002ea440}, {0x0, 0x0, 0x0, 0x0, 0x0}, {0x102c3f4c0, 0x1, ...}, ...)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:357 +0x10c
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).processSingleWindow(0x140001b60f0, 0x140001932a8)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:177 +0x74
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).processMainInput(0x140002ea440?, 0x10285b540?)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:155 +0x5c
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*ParDo).ProcessElement(0x140001b60f0, {0x0?, 0x0?}, 0x140000e0a20, {0x0, 0x0, 0x0})
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/pardo.go:140 +0x114
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*FixedRoot).Process(0x140002e6ae0, {0x10285b460, 0x1400003a080})
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/unit_test.go:142 +0xfc
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.callNoPanic({0x10285b460?, 0x1400003a080?}, 0x0?)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/util.go:62 +0x64
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.(*Plan).Execute(0x140001d3420, {0x10285b460, 0x1400003a080}, {0x1025ea4a6, 0x1}, {{0x0?, 0x0?}, {0x0?, 0x0?}})
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/plan.go:129 +0x2e4
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec.TestParDo(0x140002e8340)
/Users/deleplace/git/apache/beam/sdks/go/pkg/beam/core/runtime/exec/pardo_test.go:88 +0x14d4
testing.tRunner(0x140002e8340, 0x10284ffa8)
/usr/local/go/src/testing/testing.go:1439 +0x110
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1486 +0x300
exit status 2
FAIL github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/exec 600.239s
```
### Issue Priority
Priority: 1
### Issue Component
Component: sdk-go
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Deleplace commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
Deleplace commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1195706445
I confirm that `go test -v -run "^TestParDo$"` produces the same problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1204005143
Huh. That's surprising. Extremely good to know, I've asked a user who reported a similar issue to update and re-run their pipeline. This may be something we can report as a known compatibility issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Deleplace commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
Deleplace commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1203997635
Plot twist: yesterday I upgraded from macOS Monterey 12.4 to 12.5, and now I'm unable to reproduce the problem with Go 1.18 and Go 1.19.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Deleplace commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
Deleplace commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1195393983
I'm not experiencing this issue in Linux (my other machine).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1195573846
Yes I have reproduced on the M1 Mac and poked around a bit, it's specifically the `TestParDo` unit test and looks to be an issue with getting values out of the iterators/reiterators. That's where the test stops, looks like it's the same as #21998 where the iterator isn't returning false after the last element and is creating an infinite loop as a result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1204139489
For what it's worth I am on 12.5 and still get the time-out. So we're in some strange territory now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Deleplace commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
Deleplace commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1195397664
`go version go1.18 darwin/arm64`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #22447: [Bug]: TestParDo timing out on OSX (arm64)
Posted by GitBox <gi...@apache.org>.
damccorm commented on issue #22447:
URL: https://github.com/apache/beam/issues/22447#issuecomment-1195564187
Hm, I'm not able to repro on my mac (not an M1). @jrmccluskey I think you have an M1, right? Any chance you can repro? In general, we've had a few potentially arm64 specific issues pop up (#21998, #21667).
@Deleplace could you try to narrow things down even a little more? A couple things to try:
1) `go test -v -run "^TestParDo$"` to narrow down to just TestParDo. If that doesn't repro, that means the issue is actually in `TestParDo_WindowObservation`. It would be worth knowing if `go test -v -run TestParDo_WindowObservation` repros as well.
2) Using a minimal case from (1) could you try [commenting out calls in the test](https://github.com/apache/beam/blob/5141ad8790a57e2fa62af607f32736e3eed399e3/sdks/go/pkg/beam/core/runtime/exec/pardo_test.go#L88) to determine which call is actually hanging?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org