You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/22 16:59:50 UTC
[GitHub] [beam] Haupc opened a new issue, #21998: [Bug]: Loop last element when write to file using textio
Haupc opened a new issue, #21998:
URL: https://github.com/apache/beam/issues/21998
### What happened?
I wrote a test that writing to text file from a PCollection<string> using go SDK.
Expected:
- Write each element of PCollection only one time to file and finish the task
Actual result:
- Each element of PCollection write to file only one time EXCEPT last element.
- Last element was keep writing infinitely to file and file keeps grow bigger and bigger.
Code:
```go
package main
import (
"context"
"flag"
"fmt"
"github.com/apache/beam/sdks/v2/go/pkg/beam"
_ "github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/local"
"github.com/apache/beam/sdks/v2/go/pkg/beam/io/textio"
"github.com/apache/beam/sdks/v2/go/pkg/beam/runners/direct"
)
func main() {
flag.Parse()
beam.Init()
// In order to start creating the pipeline for execution, a Pipeline object is needed.
p := beam.NewPipeline()
s := p.Root()
firstPCol := beam.Create(s, "first", "mid1", "mid2", "mid3", "last")
textio.Write(s, "/Users/haupc/project/abeam/output.txt", firstPCol)
if _, err := direct.Execute(context.Background(), p); err != nil {
fmt.Printf("Pipeline failed: %v", err)
}
}
```
### Issue Priority
Priority: 0
### Issue Component
Component: io-common
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1165814763
We already did.
On Fri, Jun 24, 2022, 10:06 AM Alexey Romanenko ***@***.***>
wrote:
> Is it really P0 issue? Can we down it to P1?
> For ref: https://beam.apache.org/contribute/issue-priorities/
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/issues/21998#issuecomment-1165766449>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADKDOFPQQL7LYHGLZZVSPELVQXTIPANCNFSM5ZQ7B4ZQ>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Haupc commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
Haupc commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1165136355
> Otherwise, the bug is almost certainly in the direct runner.
>
> I'd guess something in the iterator is not returning false after the last element.
It's true, same thing happened when I collect element from <K,V> PCollection. iterator is not returning false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1165830926
@lostluck sorry for noice, I didn't notice that in the beginning
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
pabloem commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1164754982
@lostluck @damccorm could y'all take a look? : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1164776263
@Haupc could you provide the following information to help us narrow this down?
SDK version
Go version (`go version`)
OS and Architecture (`go env GOOS` and `go env GOARCH`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aromanenko-dev commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
aromanenko-dev commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1165766449
Is it really P0 issue? Can we down it to P1?
For ref: https://beam.apache.org/contribute/issue-priorities/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1164780472
Otherwise, the bug is almost certainly in the direct runner.
I'd guess something in the iterator is not returning false after the last element.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Haupc commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
Haupc commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1165135608
> @Haupc could you provide the following information to help us narrow this down?
>
> SDK version Go version (`go version`) OS and Architecture (`go env GOOS` and `go env GOARCH`)
go version: `go version go1.18.1 darwin/arm64`
go env GOOS: `darwin`
go env GOARCH: `arm64`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on issue #21998: [Bug]: Loop last element when write to file using textio in Go SDK
Posted by GitBox <gi...@apache.org>.
jrmccluskey commented on issue #21998:
URL: https://github.com/apache/beam/issues/21998#issuecomment-1204004235
The user who reported #22447 recently updated their MacOS version to Monterey 12.5 and no longer has this issue. If you are able to, could you update your OS and re-run your pipeline?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org