You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/05/24 23:09:16 UTC
[GitHub] [beam] brucearctor opened a new pull request #11806: Flatten Kata for Go
brucearctor opened a new pull request #11806:
URL: https://github.com/apache/beam/pull/11806
Adding a Flatten Kata for Go SDK.
@henryken
@damondouglas
@rionmonster
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damondouglas commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
damondouglas commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-634999073
@henryken Related to [Comment from PR 11803](https://github.com/apache/beam/pull/11803#issuecomment-634696399), would you mind to let me know when this is ready to update Stepik along with PR #11803?
Thank you @brucearctor for doing this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck merged pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
lostluck merged pull request #11806:
URL: https://github.com/apache/beam/pull/11806
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
lostluck commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-634864321
Retest This Please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] henryken commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
henryken commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-635811119
Yes it is correct @damondouglas.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] henryken commented on a change in pull request #11806: Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11806:
URL: https://github.com/apache/beam/pull/11806#discussion_r429796786
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/task-info.yaml
##########
@@ -0,0 +1,16 @@
+type: edu
Review comment:
Missing Apache license header
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/cmd/main.go
##########
@@ -0,0 +1,28 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import "github.com/apache/beam/sdks/go/pkg/beam"
+
+func ApplyTransform(s beam.Scope, input beam.PCollection) beam.PCollection {
Review comment:
Please provide main() function to run the pipeline normally.
You can refer to other main.go files for reference.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] brucearctor commented on a change in pull request #11806: Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
brucearctor commented on a change in pull request #11806:
URL: https://github.com/apache/beam/pull/11806#discussion_r430072602
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/cmd/main.go
##########
@@ -0,0 +1,28 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import "github.com/apache/beam/sdks/go/pkg/beam"
+
+func ApplyTransform(s beam.Scope, input beam.PCollection) beam.PCollection {
Review comment:
Ya, saw that after reading your email! Figuring out the squash.
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/cmd/main.go
##########
@@ -0,0 +1,28 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import "github.com/apache/beam/sdks/go/pkg/beam"
+
+func ApplyTransform(s beam.Scope, input beam.PCollection) beam.PCollection {
Review comment:
@henryken -- I think I got it (can't believe I overlooked that) :-) let me know how it looks, thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] henryken commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
henryken commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-635102477
This looks good now. Thanks @brucearctor!
@lostluck, please help to merge this PR.
@damondouglas, once this is merged, you can merge this to your PR and then we can upload to Stepik.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
lostluck commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-635568870
@damondouglas That sounds correct to me as well, in order to avoid colliding stepik updates.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] brucearctor commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
brucearctor commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-634795059
@henryken --> I think got the comment syntax (# instead of //) addressed
Otherwise, looks good to go from comments (unless I accidentally closed any; hope I didn't)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] henryken commented on a change in pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11806:
URL: https://github.com/apache/beam/pull/11806#discussion_r431156976
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/cmd/main.go
##########
@@ -0,0 +1,28 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import "github.com/apache/beam/sdks/go/pkg/beam"
+
+func ApplyTransform(s beam.Scope, input beam.PCollection) beam.PCollection {
Review comment:
Looks good now. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] henryken commented on a change in pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11806:
URL: https://github.com/apache/beam/pull/11806#discussion_r431148739
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/task-info.yaml
##########
@@ -0,0 +1,31 @@
+ // Licensed to the Apache Software Foundation (ASF) under one or more
Review comment:
Incorrect comment syntax for yaml file.
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/task.md
##########
@@ -0,0 +1,38 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+
+Flatten
+-------
+
+Flatten is a Beam transform for PCollection objects that store the same data type. Flatten merges
+multiple PCollection objects into a single logical PCollection.
+
+**Kata:** Implement a
Review comment:
The use of "implement" here means to implement a solution. The same description is used across many task descriptions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damondouglas commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
damondouglas commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-635563478
@henryken Just confirming these steps:
1. @lostluck merges this PR #11806 to master
1. @damondouglas merges new changes from PR #11806 to PR #11803
1. @damondouglas uploads to [Stepik](https://stepik.org/course/70387)
1. @lostluck merges PR #11803 to master
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on pull request #11806: [BEAM-9679] Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
lostluck commented on pull request #11806:
URL: https://github.com/apache/beam/pull/11806#issuecomment-634864461
(jenkin's trigger)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on a change in pull request #11806: Flatten Kata for Go
Posted by GitBox <gi...@apache.org>.
lostluck commented on a change in pull request #11806:
URL: https://github.com/apache/beam/pull/11806#discussion_r430502515
##########
File path: learning/katas/go/Core Transforms/Flatten/Flatten/task.md
##########
@@ -0,0 +1,38 @@
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one
+ ~ or more contributor license agreements. See the NOTICE file
+ ~ distributed with this work for additional information
+ ~ regarding copyright ownership. The ASF licenses this file
+ ~ to you under the Apache License, Version 2.0 (the
+ ~ "License"); you may not use this file except in compliance
+ ~ with the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+
+Flatten
+-------
+
+Flatten is a Beam transform for PCollection objects that store the same data type. Flatten merges
+multiple PCollection objects into a single logical PCollection.
+
+**Kata:** Implement a
Review comment:
Grammatical Nit: Flatten is a primitive transform in the beam model (it can't be efficiently implemented in terms of the other primitives), so one never "implements" a Flatten. One can use a flatten, but I fear saying "implements" might cause learners to head down the wrong path. (If that's the pedagogical intent here, then no worries).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org