You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/04/08 03:03:53 UTC
[GitHub] [beam] damondouglas opened a new pull request #11340: [BEAM-9678]
Create Go SDK introduction kata
damondouglas opened a new pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340
This pull request closes [BEAM-9678](https://issues.apache.org/jira/browse/BEAM-9678) by adding an Introduction Beam Kata for learning the Go SDK. It patterns after the existing Java and Python Beam Katas.
I would like to request @lostluck to review this PR.
Currently, this PR submits:
# Introduction
- Setup
- Hello Beam
- Hello Test
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [x] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] boyuanzz commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
boyuanzz commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-614877141
There are 2 go.sum files without apache license header, which breaks RAT check. Could you please take a look at it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407073077
##########
File path: learning/katas/go/course-info.yaml
##########
@@ -0,0 +1,26 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+title: Beam Katas - Go
+language: English
+summary: "This course provides a series of katas to get familiar with Apache Beam.\
+ \ \n\nApache Beam website – https://beam.apache.org/"
+programming_language: Go
Review comment:
It might be due to the early release of the plugin for Go. I think we can skip that setting for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407072000
##########
File path: learning/katas/go/README.md
##########
@@ -0,0 +1,29 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+### How to Setup
+Please follow the below steps in order to setup the project properly:
+* Using Goland Education (or Goland with EduTools plugin), select "+ New Project"
+* Click "Yes" to create project from existing sources when prompted
+* Wait for indexing to finish
+* Open the "Project" tool window, and select the "Course" view
+* Your project is ready
+
+For further instructions on how the Goland Education works, you can refer
Review comment:
Typo "GoLand"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas edited a comment on issue #11340: [BEAM-9678]
Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas edited a comment on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616002810
There is also no license information in any of the `*-remote-info.yaml` files either but that didn't break the RAT check? Should we add them there too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407074268
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/test/task_test.go
##########
@@ -0,0 +1,46 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package test
+
+import (
+ "io/ioutil"
+ "task"
+ "testing"
+)
+
+func TestTask(t *testing.T) {
Review comment:
I propose to avoid testing using output file to simplify (refer to task.go review comment).
```go
func TestTask(t *testing.T) {
ctx := context.Background()
p, s := beam.NewPipelineWithRoot()
helloBeam := main.HelloBeam(s)
passert.Equals(s, helloBeam, "Hello Beam")
if err := beamx.Run(ctx, p); err != nil {
log.Exitf(ctx, "Failed to execute job: %v", err)
}
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407072546
##########
File path: learning/katas/go/Introduction/Setup/lesson-info.yaml
##########
@@ -0,0 +1,22 @@
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+content:
+ - Setup
Review comment:
I think we should not require a separate "Setup" task.
How about merging this "Setup" task with the "Hello Beam" task and thus we have 3 answer placeholders in the "Hello Beam" task, i.e. pipeline setup, beam.Create, and pipeline run?
I understand the rationale for including the pipeline setup and running as part of the kata.
My concern last time for not including the pipeline setup was that it introduces slight difficulty and learning curve for a new student to finish the first kata, i.e. having to do multiple things at once.
And thus, I skip the pipeline setup since it's already revealed as part of the skeleton code.
If you think we should include the pipeline setup as part of the kata, then we should adopt it for the Java and Python version as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616137818
As noted in a previous reply, i already submitted a change to ignore go.sum
files. (It's ok, there were many replies).
https://github.com/apache/beam/blob/7869455ff38ce4791c5531022ffb75e7f007e06e/build.gradle#L77
On Sun, Apr 19, 2020, 12:15 AM Henry Suryawirawan <no...@github.com>
wrote:
> I'd suggest we do the same for the go.sum files.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/11340#issuecomment-616063862>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADKDOFNHMT7NHGQDZEWH6G3RNKQJ7ANCNFSM4MDS5PKQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r409279710
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam Test/go.mod
##########
@@ -0,0 +1,25 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+module hello_beam_test
+
+go 1.13
+
+require (
Review comment:
My IDE suggests that there are some missing indirect modules, i.e. x/lint, x/tools, go/tools.
Are those required to be committed?
```go
require (
github.com/apache/beam v2.19.0+incompatible
github.com/golang/protobuf v1.3.5 // indirect
github.com/google/go-cmp v0.4.0 // indirect
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3 // indirect
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135 // indirect
google.golang.org/grpc v1.28.1 // indirect
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc // indirect
)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on issue #11340: [BEAM-9678] Create
Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616002810
There is also no license not in any of the `*-remote-info.yaml` files either but that didn't break the RAT check? Should we add them there too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407073913
##########
File path: learning/katas/README.md
##########
@@ -23,4 +23,4 @@ that can help you to learn Apache Beam concepts and programming model hands-on.
Built based on [JetBrains Educational Products](https://www.jetbrains.com/education/), Beam Katas
objective is to provide a series of structured hands-on learning experiences for learners
to understand about Apache Beam and its SDKs by solving exercises with gradually increasing
-complexity. Beam Katas are available for both Java and Python SDKs.
+complexity. Beam Katas are available for Java, Python, and Go (In Development) SDKs.
Review comment:
I think we can just omit the "In Development"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616002822
I think i tried previously and they wouldn't stick on regeneration so I've
already submitted a change to ignore the go.sum files in the license
check. Note that I did ask for licenses to be put on the files earlier, so
i assume you just ran into the same problem i did.
They're important information in that it's checksums for the versions
indicated in the go.mod file, but they're ultimately automatically derived
from files with licenses, so this should be fine.
On Sat, Apr 18, 2020, 6:30 PM Damon Douglas <no...@github.com>
wrote:
> Thank you for letting me know. 😨 @lostluck <https://github.com/lostluck>
> Should we put the license information in the go.sum files? And yes that
> README on how to keep the go katas up to date will be helpful. I created
> BEAM-9782 <https://issues.apache.org/jira/browse/BEAM-9782> subtask so
> its not forgotten.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/11340#issuecomment-616002270>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADKDOFORKN56JIQNUUPYIHLRNJH4DANCNFSM4MDS5PKQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck merged pull request #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck merged pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407383884
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam Test/task.md
##########
@@ -0,0 +1,43 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Testing in Apache Beam
+
+Now that you have a simple pipeline up and running, let's talk about testing in Apache Beam using the Golang SDK.
+
+Quoting the Apache Beam documentation:
Review comment:
I think we can remove this line?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616003209
Yes. Already being ignored.
https://github.com/apache/beam/blob/7869455ff38ce4791c5531022ffb75e7f007e06e/build.gradle#L105
I suspect it's more trouble than it's worth to force users who are
refreshing auto generated files to manually reinstate the license which
might interfere with the automatic users of those files.
Community over code.
On Sat, Apr 18, 2020, 6:35 PM Robert Burke <ro...@frantil.com> wrote:
> I'd need to check but I suspect they're probably getting ignored already?
> Yaml is another finicky format.
>
> On Sat, Apr 18, 2020, 6:34 PM Damon Douglas <no...@github.com>
> wrote:
>
>> There is also no license not in any of the *-remote-info.yaml files
>> either but that didn't break the RAT check? Should we add them there too?
>>
>> —
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/apache/beam/pull/11340#issuecomment-616002810>, or
>> unsubscribe
>> <https://github.com/notifications/unsubscribe-auth/ADKDOFNB6WFEY3J6RLNWRLTRNJIJFANCNFSM4MDS5PKQ>
>> .
>>
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407077227
##########
File path: learning/katas/go/Introduction/Hello Test/Hello Test/test/task_test.go
##########
@@ -0,0 +1,37 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package test
+
+import (
+ "github.com/apache/beam/sdks/go/pkg/beam"
+ "github.com/apache/beam/sdks/go/pkg/beam/testing/passert"
+ "github.com/apache/beam/sdks/go/pkg/beam/testing/ptest"
+ "testing"
+)
+
+func TestBeam(t *testing.T) {
+ p, s := beam.NewPipelineWithRoot()
+
+ want := []interface{}{1, 2, 3}
Review comment:
Can we inline this in the passert.Equals instead?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-611611765
@henryken That would be valuable! I'm familiar with Go, and Beam, but not katas. Please chime in!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on issue #11340: [BEAM-9678] Create
Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-612567176
Thank you @henryken. I've changed the PR based on your helpful feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616063862
I'd suggest we do the same for the go.sum files.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r406584247
##########
File path: learning/katas/go/Introduction/task2/test/task_test.go
##########
@@ -0,0 +1,44 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package test
+
+import (
+ "io/ioutil"
+ "task"
+ "testing"
+)
+
+func TestTask(t *testing.T) {
+ err := task.Task()
+ if err != nil {
+ t.Error(err)
+ }
+ data, err := ioutil.ReadFile(task.OutputFile)
+ if err != nil {
+ t.Error(err)
+ }
+
+ want := "Hello Beam\n"
+ got := string(data)
+ if want != got {
+ t.Errorf("want: %s got: %s", want, got)
+ }
+
+ err = ioutil.WriteFile(task.OutputFile, []byte{}, 0644)
Review comment:
Done :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-614808287
Thank you very much! I look forward to the next lessons.
This can wait for a next PR, but having a small README on how to keep the go side of the katas up to date (eg commands to run, like go mod tidy or pointers to instructions for how to upload them to stepik) would be valuable for ongoing maintenance and improvement as the SDK has more features added (like Trigger support).
I know that when the SDK becomes module compatible and no-longer experimental, we'll definitely need to update these files :) .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r406327575
##########
File path: learning/katas/go/Introduction/task1/task-info.yaml
##########
@@ -0,0 +1,39 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+type: edu
+custom_name: Setup
+files:
+- name: test/task_test.go
+ visible: false
+- name: task.go
+ visible: true
+ placeholders:
+ - offset: 961
+ length: 26
+ placeholder_text: 'TODO: create a new pipeline'
+ - offset: 1012
+ length: 34
+ placeholder_text: 'TODO: execute the pipeline'
+- name: go.mod
+ visible: false
+- name: go.sum
+ visible: true
Review comment:
Why have the go.sum's visible?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407383208
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/pkg/task/task.go
##########
@@ -0,0 +1,24 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import (
+ "github.com/apache/beam/sdks/go/pkg/beam"
+)
+
+func HelloBeam(s beam.Scope) beam.PCollection {
Review comment:
I might have confused you on this.
Can we have this in one file with the main() function?
If we can't do that due to the function renaming to "main()", let's revert back to the previous naming so that we can have it in just one file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r409285045
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/pkg/task/task.go
##########
@@ -0,0 +1,24 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import (
+ "github.com/apache/beam/sdks/go/pkg/beam"
+)
+
+func HelloBeam(s beam.Scope) beam.PCollection {
Review comment:
Thank you, Henry. I test on both GoLand and IntelliJ in case individuals are coming from either. Thank you for embracing my suggestion :).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407072825
##########
File path: learning/katas/go/Introduction/section-info.yaml
##########
@@ -0,0 +1,23 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+content:
Review comment:
Instead of having the lessons in the section-info, I prefer to have it all under one lesson.
Proposed structure
```yaml
Introduction (content)
- Hello Beam (lesson)
- Hello Beam (task)
- Hello Beam Test (task)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r406791391
##########
File path: learning/katas/go/course-info.yaml
##########
@@ -0,0 +1,26 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+title: Beam Katas - Go
+language: English
+summary: "This course provides a series of katas to get familiar with Apache Beam.\
+ \ \n\nApache Beam website – https://beam.apache.org/"
+programming_language: Go
Review comment:
FYI there is an optional `programming_language_version` that I had tried to set with `1.13` and IntelliJ Edu kept throwing an error "programming language version not found"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407383208
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/pkg/task/task.go
##########
@@ -0,0 +1,24 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import (
+ "github.com/apache/beam/sdks/go/pkg/beam"
+)
+
+func HelloBeam(s beam.Scope) beam.PCollection {
Review comment:
I might have confused you on this. Apology for that.
Can we have this in one file with the main() function?
If we can't do that due to the function renaming to "main()", let's revert back to the previous naming so that we can have it in just one file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on issue #11340: [BEAM-9678] Create
Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-611857902
@lostluck @henryken I've made the recommended changes. Thank you for reviewing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616063770
Yeah the *-remote-info.yaml files are auto generated and they are overwritten after the course has been uploaded.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616002932
I'd need to check but I suspect they're probably getting ignored already?
Yaml is another finicky format.
On Sat, Apr 18, 2020, 6:34 PM Damon Douglas <no...@github.com>
wrote:
> There is also no license not in any of the *-remote-info.yaml files
> either but that didn't break the RAT check? Should we add them there too?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/beam/pull/11340#issuecomment-616002810>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADKDOFNB6WFEY3J6RLNWRLTRNJIJFANCNFSM4MDS5PKQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r406583732
##########
File path: learning/katas/go/Introduction/lesson-info.yaml
##########
@@ -0,0 +1,23 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+content:
Review comment:
Thank you for reviewing. I reorganized the Introduction section to pattern after the structure in the Java katas.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407071755
##########
File path: learning/katas/go/README.md
##########
@@ -0,0 +1,29 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+### How to Setup
+Please follow the below steps in order to setup the project properly:
+* Using Goland Education (or Goland with EduTools plugin), select "+ New Project"
Review comment:
There is no GoLand Edu yet. Need to use GoLand with EduTools plugin.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on issue #11340: [BEAM-9678] Create
Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-616002270
Thank you for letting me know. 😨 @lostluck Should we put the license information in the `go.sum` files? And yes that README on how to keep the go katas up to date will be helpful. I created [BEAM-9782](https://issues.apache.org/jira/browse/BEAM-9782) subtask so its not forgotten.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407074183
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/task.go
##########
@@ -0,0 +1,36 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import (
+ "context"
+ "github.com/apache/beam/sdks/go/pkg/beam"
+ "github.com/apache/beam/sdks/go/pkg/beam/x/beamx"
+)
+
+func Task() (err error) {
Review comment:
I don't prefer having to run the test using output file.
I propose the following solution here and in the task.go (and the equivalent test code in the task_test.go).
We can have the LogElements function as part of a utility module.
The reason we need the LogElements is so that the learner is able to run the program and observe the output.
Also I think it's better to use `func main()`.
```go
func main() {
ctx := context.Background()
p, s := beam.NewPipelineWithRoot()
helloBeam := HelloBeam(s)
LogElements(s, ctx, helloBeam)
if err := beamx.Run(ctx, p); err != nil {
log.Exitf(ctx, "Failed to execute job: %v", err)
}
}
func HelloBeam(s beam.Scope) beam.PCollection {
return beam.Create(s, "Hello Beam")
}
func LogElements(s beam.Scope, ctx context.Context, helloBeam beam.PCollection) beam.PCollection {
return beam.ParDo(s, func(w string) string {
log.Infof(ctx, "%s", w)
return w
}, helloBeam)
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-611327100
I can help to review this as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on issue #11340: [BEAM-9678] Create
Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-614425499
@henryken I think everything is ok and if you could help uploading this to Stepik that would be really helpful. I'll email you about hopefully meeting with you to plan out the rest of the katas.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r406302136
##########
File path: learning/katas/go/Introduction/task2/test/task_test.go
##########
@@ -0,0 +1,44 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package test
+
+import (
+ "io/ioutil"
+ "task"
+ "testing"
+)
+
+func TestTask(t *testing.T) {
+ err := task.Task()
+ if err != nil {
+ t.Error(err)
+ }
+ data, err := ioutil.ReadFile(task.OutputFile)
+ if err != nil {
+ t.Error(err)
+ }
+
+ want := "Hello Beam\n"
+ got := string(data)
+ if want != got {
+ t.Errorf("want: %s got: %s", want, got)
+ }
+
+ err = ioutil.WriteFile(task.OutputFile, []byte{}, 0644)
Review comment:
Is this intended to clear/cleanup the OutputFile? Consider a comment to make it's purpose unambiguous.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r409292176
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam Test/go.mod
##########
@@ -0,0 +1,25 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+module hello_beam_test
+
+go 1.13
+
+require (
Review comment:
I ran `go mod tidy` in both Hello Beam and Hello Beam Test as well as `go mod verify`. I also removed my entire contents of `$HOME/pkg` and `$HOME/src` and subsequently tested on both IntelliJ and GoLand. I was able to generate a course preview for both IntelliJ and GoLand and verify tasks. As I understand, the go.mod is auto-generated as we code in go. In IntelliJ/GoLand it does it for you automatically. In vim with coc, when we run `:GoImports` (I think) it also does this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407385381
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam Test/task.md
##########
@@ -0,0 +1,43 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Testing in Apache Beam
+
+Now that you have a simple pipeline up and running, let's talk about testing in Apache Beam using the Golang SDK.
+
+Quoting the Apache Beam documentation:
+
+> Testing your pipeline is a particularly important step in developing an effective data processing solution. The indirect nature of the Beam model, in which your user code constructs a pipeline graph to be executed remotely, can make debugging-failed runs a non-trivial task. Often it is faster and simpler to perform local unit testing on your pipeline code than to debug a pipeline’s remote execution.
+
+You can read more about testing on the Apache Beam website:
+https://beam.apache.org/documentation/pipelines/test-your-pipeline/.
+
+**Kata:** Your second kata is to develop tests for your pipeline.
Review comment:
How about this?
```
**Kata:** Develop test case assertion that checks whether the PCollection elements are equal to the given input from task.HelloBeam.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r407076727
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/task-info.yaml
##########
@@ -0,0 +1,37 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+type: edu
+files:
+- name: test/task_test.go
+ visible: false
+- name: test/output.txt
+ visible: false
+- name: task.go
+ visible: true
+ placeholders:
+ - offset: 1143
+ length: 28
+ placeholder_text: 'TODO: inject your pipeline with "Hello Beam".'
Review comment:
In Java & Python katas, we only use "TODO()".
I find that the kata description and hint(s) provide good enough instruction.
Do you think this will be more useful for the learner?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r409279364
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/pkg/task/task.go
##########
@@ -0,0 +1,24 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import (
+ "github.com/apache/beam/sdks/go/pkg/beam"
+)
+
+func HelloBeam(s beam.Scope) beam.PCollection {
Review comment:
Noted that Damon. I'm not that familiar with Go. Just did some reading and I guess this is good for now. Thanks for clarifying.
Also FYI, you may want to consider using GoLand instead of IntelliJ for developing this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r406584290
##########
File path: learning/katas/go/Introduction/task1/task-info.yaml
##########
@@ -0,0 +1,39 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+type: edu
+custom_name: Setup
+files:
+- name: test/task_test.go
+ visible: false
+- name: task.go
+ visible: true
+ placeholders:
+ - offset: 961
+ length: 26
+ placeholder_text: 'TODO: create a new pipeline'
+ - offset: 1012
+ length: 34
+ placeholder_text: 'TODO: execute the pipeline'
+- name: go.mod
+ visible: false
+- name: go.sum
+ visible: true
Review comment:
Fixed :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] lostluck commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
lostluck commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r405722761
##########
File path: learning/katas/go/Introduction/lesson-info.yaml
##########
@@ -0,0 +1,23 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+content:
Review comment:
It looks like in Introduction, there should be a section-info.yaml, and that lesson-info files goes into each lesson individually.
See the Java structure:
https://github.com/apache/beam/tree/db996fe1ff9de0307c312e8e81f73526e3c26323/learning/katas/java/Introduction
Probably just means adding a new section-info.yaml (renaming this one) and having lesson infos for the sub tasks, and so on. In short, this PR is just missing the "section" level.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-614452390
Awesome! @lostluck, please help to merge this PR. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] damondouglas commented on a change in pull request #11340:
[BEAM-9678] Create Go SDK introduction kata
Posted by GitBox <gi...@apache.org>.
damondouglas commented on a change in pull request #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#discussion_r408584291
##########
File path: learning/katas/go/Introduction/Hello Beam/Hello Beam/pkg/task/task.go
##########
@@ -0,0 +1,24 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package task
+
+import (
+ "github.com/apache/beam/sdks/go/pkg/beam"
+)
+
+func HelloBeam(s beam.Scope) beam.PCollection {
Review comment:
May we consider keeping this? When I consolidated everything into the same file, I received the error `flag redefined: runner`. IntelliJ Edu constrains us to having tests in the test folder in order to achieve lesson task validation, so we can't do any task validation with tests in any other folder. Also, while there doesn't seem to be a consensus around go project structure in the community, this repo seems to follows the `./{cmd,pkg}/*` structure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [beam] henryken commented on issue #11340: [BEAM-9678] Create Go
SDK introduction kata
Posted by GitBox <gi...@apache.org>.
henryken commented on issue #11340: [BEAM-9678] Create Go SDK introduction kata
URL: https://github.com/apache/beam/pull/11340#issuecomment-614414805
If everything is okay, I can help to upload this course to Stepik.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services