You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/12/06 12:18:49 UTC
[GitHub] [beam] daria-malkova opened a new pull request #16138: [BEAM-13343][Playground] Support go unit tests
daria-malkova opened a new pull request #16138:
URL: https://github.com/apache/beam/pull/16138
[BEAM-13343] Add cmd for running go unit tests
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
`ValidatesRunner` compliance status (on master branch)
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon?subject=Portable+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/badge/icon?subject=Portable">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon?subject=Structured+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon?subject=ValCont">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
</tbody>
</table>
Examples testing status on various runners
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/badge/icon?subject=V1+Java11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
Post-Commit SDK/Transform Integration Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>Go</th>
<th>Java</th>
<th>Python</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon?subject=3.6">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon?subject=3.7">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon?subject=3.8">
</a>
</td>
</tr>
</tbody>
</table>
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>---</th>
<th>Java</th>
<th>Python</th>
<th>Go</th>
<th>Website</th>
<th>Whitespace</th>
<th>Typescript</th>
</tr>
</thead>
<tbody>
<tr>
<td>Non-portable</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Tests">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon?subject=Lint">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon?subject=Docker">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon?subject=Docs">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Portable</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AydarZaynutdinov commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
AydarZaynutdinov commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r764652644
##########
File path: playground/backend/cmd/server/server.go
##########
@@ -107,8 +108,13 @@ func setupCache(ctx context.Context, appEnv environment.ApplicationEnvs) (cache.
}
func main() {
+ os.Setenv("PROTOCOL_TYPE", "TCP")
Review comment:
It seems that we do not need this one.
##########
File path: playground/backend/internal/code_processing/code_processing.go
##########
@@ -111,23 +111,30 @@ func Process(ctx context.Context, cacheService cache.Cache, lc *fs_tool.LifeCycl
switch sdkEnv.ApacheBeamSdk {
case pb.Sdk_SDK_JAVA, pb.Sdk_SDK_GO:
- // Compile
- logger.Infof("%s: Compile() ...\n", pipelineId)
- compileCmd := executor.Compile(ctxWithTimeout)
- var compileError bytes.Buffer
- var compileOutput bytes.Buffer
- runCmdWithOutput(compileCmd, &compileOutput, &compileError, successChannel, errorChannel)
-
- ok, err = processStep(ctxWithTimeout, pipelineId, cacheService, cancelChannel, successChannel)
- if err != nil {
- return
- }
- if !ok {
- _ = processCompileError(ctxWithTimeout, errorChannel, compileError.Bytes(), pipelineId, cacheService)
- return
- }
- if err := processCompileSuccess(ctxWithTimeout, compileOutput.Bytes(), pipelineId, cacheService); err != nil {
- return
+ isUnitTest, ok := validationResults.Load(validators.UnitTestValidatorName)
+ if !isUnitTest.(bool) {
+ // Compile
+ logger.Infof("%s: Compile() ...\n", pipelineId)
+ compileCmd := executor.Compile(ctxWithTimeout)
+ var compileError bytes.Buffer
+ var compileOutput bytes.Buffer
+ runCmdWithOutput(compileCmd, &compileOutput, &compileError, successChannel, errorChannel)
+
+ ok, err = processStep(ctxWithTimeout, pipelineId, cacheService, cancelChannel, successChannel)
+ if err != nil {
+ return
+ }
+ if !ok {
+ _ = processCompileError(ctxWithTimeout, errorChannel, compileError.Bytes(), pipelineId, cacheService)
+ return
+ }
+ if err := processCompileSuccess(ctxWithTimeout, compileOutput.Bytes(), pipelineId, cacheService); err != nil {
+ return
+ }
+ } else {
+ if err := processCompileSuccess(ctxWithTimeout, []byte(""), pipelineId, cacheService); err != nil {
+ return
+ }
}
case pb.Sdk_SDK_PYTHON:
if err := processCompileSuccess(ctxWithTimeout, []byte(""), pipelineId, cacheService); err != nil {
Review comment:
Could we combine this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] daria-malkova commented on pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
daria-malkova commented on pull request #16138:
URL: https://github.com/apache/beam/pull/16138#issuecomment-986724742
R: @ilya-kozyrev @KhaninArtur @AydarZaynutdinov
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r766822306
##########
File path: playground/backend/internal/code_processing/code_processing.go
##########
@@ -88,14 +88,20 @@ func Process(ctx context.Context, cacheService cache.Cache, lc *fs_tool.LifeCycl
_ = processError(ctxWithTimeout, errorChannel, pipelineId, cacheService, "Validate", pb.Status_STATUS_VALIDATION_ERROR)
return
}
+ // Check if unit test
+ isUnitTest := false
+ valResult, ok := validationResults.Load(validators.UnitTestValidatorName)
Review comment:
```suggestion
validateIsUnitTest, ok := validationResults.Load(validators.UnitTestValidatorName)
```
##########
File path: playground/backend/internal/validators/validator_test.go
##########
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package validators
+
+import (
+ "fmt"
+ "os"
+ "testing"
+)
+
+func TestMain(m *testing.M) {
Review comment:
Can you add documentation to this file to explain that it defines some common methods for validators? (I was confused looking for a test within it, but there aren't any tests in it, right?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] daria-malkova commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
daria-malkova commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r765526011
##########
File path: playground/backend/cmd/server/server.go
##########
@@ -107,8 +108,13 @@ func setupCache(ctx context.Context, appEnv environment.ApplicationEnvs) (cache.
}
func main() {
+ os.Setenv("PROTOCOL_TYPE", "TCP")
Review comment:
Sorry it was draft version, needs it for testing
##########
File path: playground/backend/cmd/server/server.go
##########
@@ -107,8 +108,13 @@ func setupCache(ctx context.Context, appEnv environment.ApplicationEnvs) (cache.
}
func main() {
+ os.Setenv("PROTOCOL_TYPE", "TCP")
err := runServer()
if err != nil {
panic(err)
}
}
+
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam_test\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam\"\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/testing/
passert\"\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/testing/ptest\"\n\t\"github.com/golang/protobuf/proto\"\n)\n\ntype wc struct {\n\tK string\n\tV int\n}\n\nfunc TestCreate(t *testing.T) {\n\ttests := []struct {\n\t\tvalues []interface{}\n\t}{\n\t\t{[]interface{}{1, 2, 3}},\n\t\t{[]interface{}{\"1\", \"2\", \"3\"}},\n\t\t{[]interface{}{float32(0.1), float32(0.2), float32(0.3)}},\n\t\t{[]interface{}{float64(0.1), float64(0.2), float64(0.3)}},\n\t\t{[]interface{}{uint(1), uint(2), uint(3)}},\n\t\t{[]interface{}{false, true, true, false, true}},\n\t\t{[]interface{}{wc{\"a\", 23}, wc{\"b\", 42}, wc{\"c\", 5}}},\n\t\t{[]interface{}{&testProto{}, &testProto{stringValue(\"test\")}}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.Create(s, test.values...)\n\t\tpassert.Equals(s, c, test.values...)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.Create(%v) failed: %v\", test.values, err)\n\t\t}\n\t
}\n}\n\nfunc TestCreateList(t *testing.T) {\n\ttests := []struct {\n\t\tvalues interface{}\n\t}{\n\t\t{[]int{1, 2, 3}},\n\t\t{[]string{\"1\", \"2\", \"3\"}},\n\t\t{[]float32{float32(0.1), float32(0.2), float32(0.3)}},\n\t\t{[]float64{float64(0.1), float64(0.2), float64(0.3)}},\n\t\t{[]uint{uint(1), uint(2), uint(3)}},\n\t\t{[]bool{false, true, true, false, true}},\n\t\t{[]wc{wc{\"a\", 23}, wc{\"b\", 42}, wc{\"c\", 5}}},\n\t\t{[]*testProto{&testProto{}, &testProto{stringValue(\"test\")}}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.CreateList(s, test.values)\n\n\t\tvar values []interface{}\n\t\tv := reflect.ValueOf(test.values)\n\t\tfor i := 0; i < v.Len(); i++ {\n\t\t\tvalues = append(values, v.Index(i).Interface())\n\t\t}\n\t\tpassert.Equals(s, c, values...)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.CreateList(%v) failed: %v\", test.values, err)\n\t\t}\n\t}\n}\n\nfunc TestCreateEmptyLi
st(t *testing.T) {\n\ttests := []struct {\n\t\tvalues interface{}\n\t}{\n\t\t{[]int{}},\n\t\t{[]string{}},\n\t\t{[]float32{}},\n\t\t{[]float64{}},\n\t\t{[]uint{}},\n\t\t{[]bool{}},\n\t\t{[]wc{}},\n\t\t{[]*testProto{}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.CreateList(s, test.values)\n\n\t\tpassert.Empty(s, c)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.CreateList(%v) failed: %v\", test.values, err)\n\t\t}\n\t}\n}\n\ntype testProto struct {\n\t// OneOfField is an interface-typed field and cannot be JSON-marshaled, but\n\t// should be specially handled by Beam as a field of a proto.Message.\n\tOneOfField _isOneOfField\n}\n\ntype stringValue string\n\nfunc (stringValue) isOneOfField() {}\n\ntype _isOneOfField interface {\n\tisOneOfField()\n}\n\nfunc (t *testProto) Reset() { *t = testProto{} }\nfunc (t *testProto) String() string { return fmt.Sprintf(\"one_of_field: %#v\", t.OneO
fField) }\nfunc (t *testProto) ProtoMessage() {}\n\nfunc (t *testProto) Marshal() ([]byte, error) {\n\tif t.OneOfField == nil {\n\t\treturn nil, nil\n\t}\n\treturn []byte(t.OneOfField.(stringValue)), nil\n}\nfunc (t *testProto) Unmarshal(b []byte) error {\n\tif len(b) == 0 {\n\t\treturn nil\n\t}\n\tt.OneOfField = stringValue(b)\n\treturn nil\n}\n\n// Ensure testProto is detected as a proto.Message and can be (un)marshalled by\n// the proto library.\nvar (\n\t_ proto.Message = &testProto{}\n\t_ proto.Marshaler = &testProto{}\n\t_ proto.Unmarshaler = &testProto{}\n)\n"
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema\"\n)\n\nfunc TestJSONCo
der(t *testing.T) {\n\tv := \"teststring\"\n\ttests := []interface{}{\n\t\t43,\n\t\t12431235,\n\t\t-2,\n\t\t0,\n\t\t1,\n\t\ttrue,\n\t\t\"a string\",\n\t\tmap[int64]string{1: \"one\", 11: \"oneone\", 21: \"twoone\", 1211: \"onetwooneone\"},\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t}\n\n\tfor _, test := range tests {\n\t\tvar results []string\n\t\tfor i := 0; i < 10; i++ {\n\t\t\tdata, err := jsonEnc(test)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t}\n\t\t\tresults = append(results, string(data))\n\t\t}\n\t\tfor i, data := range results {\n\t\t\tif data != results[0] {\n\t\t\t\tt.Errorf(\"coder not deterministic: data[%d]: %v != %v \", i, data, results[0])\n\t\t\t}\n\t\t}\n\n\t\tdecoded, err := jsonDec(reflect.TypeOf(test), []byte(results[0]))\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to decode: %v\", err)\n\t\t}\n\n\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\tt.Errorf(\"Corrupt coding: %v,
want %v\", decoded, test)\n\t\t}\n\t}\n}\n\nfunc TestSchemaCoder(t *testing.T) {\n\tv := \"teststring\"\n\ttests := []interface{}{\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t\t&struct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t\tstruct {\n\t\t\tA map[string]int\n\t\t\tB []int\n\t\t}{map[string]int{\"three\": 3}, []int{4}},\n\t\tstruct {\n\t\t\tA map[string]int\n\t\t\tB [1]int\n\t\t}{map[string]int{\"three\": 3}, [...]int{4}},\n\t}\n\n\tfor _, test := range tests {\n\t\trt := reflect.TypeOf(test)\n\t\tt.Run(fmt.Sprintf(\"%v\", rt), func(t *testing.T) {\n\t\t\tvar results []string\n\t\t\tfor i := 0; i < 10; i++ {\n\t\t\t\tdata, err := schemaEnc(rt, test)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t\t}\n\t\t\t\tresults = append(results, string(data))\n\t\t\t}\n\t\t\tfor i, data := range results {\n\t\t\t\tif data != results[0] {\n\t\t\t\t\tt.Errorf(\"coder not determinis
tic: data[%d]: %v != %v \", i, data, results[0])\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdecoded, err := schemaDec(rt, []byte(results[0]))\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to decode: %v\", err)\n\t\t\t}\n\n\t\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\t\tt.Errorf(\"Corrupt coding: %v, want %v\", decoded, test)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCoders(t *testing.T) {\n\tptrString := \"test *string\"\n\ttype regTestType struct {\n\t\tA [4]int\n\t}\n\tschema.RegisterType(reflect.TypeOf((*regTestType)(nil)))\n\ttests := []interface{}{\n\t\t43,\n\t\t12431235,\n\t\t-2,\n\t\t0,\n\t\t1,\n\t\ttrue,\n\t\t\"a string\",\n\t\tmap[int64]string{1: \"one\", 11: \"oneone\", 21: \"twoone\", 1211: \"onetwooneone\"}, // (not supported by custom type registration)\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &ptrString, false},\n\t\t[...]int64{1, 2, 3, 4, 5}, // array (not supported by custom type registration)\n\t\t[]int64{1, 2, 3, 4, 5}, // slice\n\t\tstruct {\n
\t\t\tA []int\n\t\t\tB [3]int\n\t\t}{A: []int{1, 2, 3}, B: [...]int{4, 5, 6}},\n\t\t[...]struct{ A int }{{1}, {2}, {3}, {4}, {5}},\n\t\t[]struct{ B int }{{1}, {2}, {3}, {4}, {5}},\n\t\tregTestType{[4]int{4, 2, 4, 2}},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(fmt.Sprintf(\"%T\", test), func(t *testing.T) {\n\t\t\tvar results []string\n\t\t\trt := reflect.TypeOf(test)\n\t\t\tenc := NewElementEncoder(rt)\n\t\t\tfor i := 0; i < 10; i++ {\n\t\t\t\tvar buf bytes.Buffer\n\t\t\t\tif err := enc.Encode(test, &buf); err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t\t}\n\t\t\t\tresults = append(results, string(buf.Bytes()))\n\t\t\t}\n\t\t\tfor i, d := range results {\n\t\t\t\tif d != results[0] {\n\t\t\t\t\tt.Errorf(\"coder not deterministic: encoding %d not the same as the first encoding: %v != %v \", i, d, results[0])\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdec := NewElementDecoder(rt)\n\t\t\tbuf := bytes.NewBuffer([]byte(results[0]))\n\t\t\tdecoded, err := dec.De
code(buf)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to decode: %q, %v\", results[0], err)\n\t\t\t}\n\n\t\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\t\tt.Errorf(\"Corrupt coding: %v, want %v\", decoded, test)\n\t\t\t}\n\t\t})\n\t}\n}\n"
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam_test\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam\"\n)\n\n// foolFn is a no-op CombineFn.\ntype foolFn struct {\n\tOutp
utType beam.EncodedType\n}\n\ntype foolAccum struct{}\n\nfunc (f *foolFn) CreateAccumulator() *foolAccum {\n\treturn &foolAccum{}\n}\n\nfunc (f *foolFn) AddInput(a *foolAccum, v beam.U) *foolAccum {\n\treturn a\n}\n\nfunc (f *foolFn) MergeAccumulators(a *foolAccum, b *foolAccum) *foolAccum {\n\treturn a\n}\n\nfunc (f *foolFn) ExtractOutput(a *foolAccum) beam.V {\n\treturn reflect.New(f.OutputType.T).Elem().Interface()\n}\n\nfunc TestCombineWithTypeDefinition(t *testing.T) {\n\t_, s := beam.NewPipelineWithRoot()\n\tin := beam.Create(s, 1, 2, 3)\n\tstrType := reflect.TypeOf(\"\")\n\tcombineFn := &foolFn{OutputType: beam.EncodedType{T: strType}}\n\toutput := beam.Combine(s, combineFn, in, beam.TypeDefinition{Var: beam.VType, T: strType})\n\tif output.Type().Type() != strType {\n\t\tt.Errorf(\"expect combine output type to be %v, got %v\", strType, output.Type().Type())\n\t}\n}\n"
Review comment:
Sorry it was draft version, needs it for testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] daria-malkova commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
daria-malkova commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r765819792
##########
File path: playground/backend/internal/code_processing/code_processing.go
##########
@@ -111,23 +111,30 @@ func Process(ctx context.Context, cacheService cache.Cache, lc *fs_tool.LifeCycl
switch sdkEnv.ApacheBeamSdk {
case pb.Sdk_SDK_JAVA, pb.Sdk_SDK_GO:
- // Compile
- logger.Infof("%s: Compile() ...\n", pipelineId)
- compileCmd := executor.Compile(ctxWithTimeout)
- var compileError bytes.Buffer
- var compileOutput bytes.Buffer
- runCmdWithOutput(compileCmd, &compileOutput, &compileError, successChannel, errorChannel)
-
- ok, err = processStep(ctxWithTimeout, pipelineId, cacheService, cancelChannel, successChannel)
- if err != nil {
- return
- }
- if !ok {
- _ = processCompileError(ctxWithTimeout, errorChannel, compileError.Bytes(), pipelineId, cacheService)
- return
- }
- if err := processCompileSuccess(ctxWithTimeout, compileOutput.Bytes(), pipelineId, cacheService); err != nil {
- return
+ isUnitTest, ok := validationResults.Load(validators.UnitTestValidatorName)
+ if !isUnitTest.(bool) {
+ // Compile
+ logger.Infof("%s: Compile() ...\n", pipelineId)
+ compileCmd := executor.Compile(ctxWithTimeout)
+ var compileError bytes.Buffer
+ var compileOutput bytes.Buffer
+ runCmdWithOutput(compileCmd, &compileOutput, &compileError, successChannel, errorChannel)
+
+ ok, err = processStep(ctxWithTimeout, pipelineId, cacheService, cancelChannel, successChannel)
+ if err != nil {
+ return
+ }
+ if !ok {
+ _ = processCompileError(ctxWithTimeout, errorChannel, compileError.Bytes(), pipelineId, cacheService)
+ return
+ }
+ if err := processCompileSuccess(ctxWithTimeout, compileOutput.Bytes(), pipelineId, cacheService); err != nil {
+ return
+ }
+ } else {
+ if err := processCompileSuccess(ctxWithTimeout, []byte(""), pipelineId, cacheService); err != nil {
+ return
+ }
}
case pb.Sdk_SDK_PYTHON:
if err := processCompileSuccess(ctxWithTimeout, []byte(""), pipelineId, cacheService); err != nil {
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] daria-malkova edited a comment on pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
daria-malkova edited a comment on pull request #16138:
URL: https://github.com/apache/beam/pull/16138#issuecomment-986724742
R: @ilya-kozyrev @KhaninArtur @AydarZaynutdinov @pavel-avilov
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] daria-malkova commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
daria-malkova commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r765526226
##########
File path: playground/backend/cmd/server/server.go
##########
@@ -107,8 +108,13 @@ func setupCache(ctx context.Context, appEnv environment.ApplicationEnvs) (cache.
}
func main() {
+ os.Setenv("PROTOCOL_TYPE", "TCP")
err := runServer()
if err != nil {
panic(err)
}
}
+
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam_test\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam\"\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/testing/
passert\"\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/testing/ptest\"\n\t\"github.com/golang/protobuf/proto\"\n)\n\ntype wc struct {\n\tK string\n\tV int\n}\n\nfunc TestCreate(t *testing.T) {\n\ttests := []struct {\n\t\tvalues []interface{}\n\t}{\n\t\t{[]interface{}{1, 2, 3}},\n\t\t{[]interface{}{\"1\", \"2\", \"3\"}},\n\t\t{[]interface{}{float32(0.1), float32(0.2), float32(0.3)}},\n\t\t{[]interface{}{float64(0.1), float64(0.2), float64(0.3)}},\n\t\t{[]interface{}{uint(1), uint(2), uint(3)}},\n\t\t{[]interface{}{false, true, true, false, true}},\n\t\t{[]interface{}{wc{\"a\", 23}, wc{\"b\", 42}, wc{\"c\", 5}}},\n\t\t{[]interface{}{&testProto{}, &testProto{stringValue(\"test\")}}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.Create(s, test.values...)\n\t\tpassert.Equals(s, c, test.values...)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.Create(%v) failed: %v\", test.values, err)\n\t\t}\n\t
}\n}\n\nfunc TestCreateList(t *testing.T) {\n\ttests := []struct {\n\t\tvalues interface{}\n\t}{\n\t\t{[]int{1, 2, 3}},\n\t\t{[]string{\"1\", \"2\", \"3\"}},\n\t\t{[]float32{float32(0.1), float32(0.2), float32(0.3)}},\n\t\t{[]float64{float64(0.1), float64(0.2), float64(0.3)}},\n\t\t{[]uint{uint(1), uint(2), uint(3)}},\n\t\t{[]bool{false, true, true, false, true}},\n\t\t{[]wc{wc{\"a\", 23}, wc{\"b\", 42}, wc{\"c\", 5}}},\n\t\t{[]*testProto{&testProto{}, &testProto{stringValue(\"test\")}}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.CreateList(s, test.values)\n\n\t\tvar values []interface{}\n\t\tv := reflect.ValueOf(test.values)\n\t\tfor i := 0; i < v.Len(); i++ {\n\t\t\tvalues = append(values, v.Index(i).Interface())\n\t\t}\n\t\tpassert.Equals(s, c, values...)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.CreateList(%v) failed: %v\", test.values, err)\n\t\t}\n\t}\n}\n\nfunc TestCreateEmptyLi
st(t *testing.T) {\n\ttests := []struct {\n\t\tvalues interface{}\n\t}{\n\t\t{[]int{}},\n\t\t{[]string{}},\n\t\t{[]float32{}},\n\t\t{[]float64{}},\n\t\t{[]uint{}},\n\t\t{[]bool{}},\n\t\t{[]wc{}},\n\t\t{[]*testProto{}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.CreateList(s, test.values)\n\n\t\tpassert.Empty(s, c)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.CreateList(%v) failed: %v\", test.values, err)\n\t\t}\n\t}\n}\n\ntype testProto struct {\n\t// OneOfField is an interface-typed field and cannot be JSON-marshaled, but\n\t// should be specially handled by Beam as a field of a proto.Message.\n\tOneOfField _isOneOfField\n}\n\ntype stringValue string\n\nfunc (stringValue) isOneOfField() {}\n\ntype _isOneOfField interface {\n\tisOneOfField()\n}\n\nfunc (t *testProto) Reset() { *t = testProto{} }\nfunc (t *testProto) String() string { return fmt.Sprintf(\"one_of_field: %#v\", t.OneO
fField) }\nfunc (t *testProto) ProtoMessage() {}\n\nfunc (t *testProto) Marshal() ([]byte, error) {\n\tif t.OneOfField == nil {\n\t\treturn nil, nil\n\t}\n\treturn []byte(t.OneOfField.(stringValue)), nil\n}\nfunc (t *testProto) Unmarshal(b []byte) error {\n\tif len(b) == 0 {\n\t\treturn nil\n\t}\n\tt.OneOfField = stringValue(b)\n\treturn nil\n}\n\n// Ensure testProto is detected as a proto.Message and can be (un)marshalled by\n// the proto library.\nvar (\n\t_ proto.Message = &testProto{}\n\t_ proto.Marshaler = &testProto{}\n\t_ proto.Unmarshaler = &testProto{}\n)\n"
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema\"\n)\n\nfunc TestJSONCo
der(t *testing.T) {\n\tv := \"teststring\"\n\ttests := []interface{}{\n\t\t43,\n\t\t12431235,\n\t\t-2,\n\t\t0,\n\t\t1,\n\t\ttrue,\n\t\t\"a string\",\n\t\tmap[int64]string{1: \"one\", 11: \"oneone\", 21: \"twoone\", 1211: \"onetwooneone\"},\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t}\n\n\tfor _, test := range tests {\n\t\tvar results []string\n\t\tfor i := 0; i < 10; i++ {\n\t\t\tdata, err := jsonEnc(test)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t}\n\t\t\tresults = append(results, string(data))\n\t\t}\n\t\tfor i, data := range results {\n\t\t\tif data != results[0] {\n\t\t\t\tt.Errorf(\"coder not deterministic: data[%d]: %v != %v \", i, data, results[0])\n\t\t\t}\n\t\t}\n\n\t\tdecoded, err := jsonDec(reflect.TypeOf(test), []byte(results[0]))\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to decode: %v\", err)\n\t\t}\n\n\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\tt.Errorf(\"Corrupt coding: %v,
want %v\", decoded, test)\n\t\t}\n\t}\n}\n\nfunc TestSchemaCoder(t *testing.T) {\n\tv := \"teststring\"\n\ttests := []interface{}{\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t\t&struct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t\tstruct {\n\t\t\tA map[string]int\n\t\t\tB []int\n\t\t}{map[string]int{\"three\": 3}, []int{4}},\n\t\tstruct {\n\t\t\tA map[string]int\n\t\t\tB [1]int\n\t\t}{map[string]int{\"three\": 3}, [...]int{4}},\n\t}\n\n\tfor _, test := range tests {\n\t\trt := reflect.TypeOf(test)\n\t\tt.Run(fmt.Sprintf(\"%v\", rt), func(t *testing.T) {\n\t\t\tvar results []string\n\t\t\tfor i := 0; i < 10; i++ {\n\t\t\t\tdata, err := schemaEnc(rt, test)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t\t}\n\t\t\t\tresults = append(results, string(data))\n\t\t\t}\n\t\t\tfor i, data := range results {\n\t\t\t\tif data != results[0] {\n\t\t\t\t\tt.Errorf(\"coder not determinis
tic: data[%d]: %v != %v \", i, data, results[0])\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdecoded, err := schemaDec(rt, []byte(results[0]))\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to decode: %v\", err)\n\t\t\t}\n\n\t\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\t\tt.Errorf(\"Corrupt coding: %v, want %v\", decoded, test)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCoders(t *testing.T) {\n\tptrString := \"test *string\"\n\ttype regTestType struct {\n\t\tA [4]int\n\t}\n\tschema.RegisterType(reflect.TypeOf((*regTestType)(nil)))\n\ttests := []interface{}{\n\t\t43,\n\t\t12431235,\n\t\t-2,\n\t\t0,\n\t\t1,\n\t\ttrue,\n\t\t\"a string\",\n\t\tmap[int64]string{1: \"one\", 11: \"oneone\", 21: \"twoone\", 1211: \"onetwooneone\"}, // (not supported by custom type registration)\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &ptrString, false},\n\t\t[...]int64{1, 2, 3, 4, 5}, // array (not supported by custom type registration)\n\t\t[]int64{1, 2, 3, 4, 5}, // slice\n\t\tstruct {\n
\t\t\tA []int\n\t\t\tB [3]int\n\t\t}{A: []int{1, 2, 3}, B: [...]int{4, 5, 6}},\n\t\t[...]struct{ A int }{{1}, {2}, {3}, {4}, {5}},\n\t\t[]struct{ B int }{{1}, {2}, {3}, {4}, {5}},\n\t\tregTestType{[4]int{4, 2, 4, 2}},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(fmt.Sprintf(\"%T\", test), func(t *testing.T) {\n\t\t\tvar results []string\n\t\t\trt := reflect.TypeOf(test)\n\t\t\tenc := NewElementEncoder(rt)\n\t\t\tfor i := 0; i < 10; i++ {\n\t\t\t\tvar buf bytes.Buffer\n\t\t\t\tif err := enc.Encode(test, &buf); err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t\t}\n\t\t\t\tresults = append(results, string(buf.Bytes()))\n\t\t\t}\n\t\t\tfor i, d := range results {\n\t\t\t\tif d != results[0] {\n\t\t\t\t\tt.Errorf(\"coder not deterministic: encoding %d not the same as the first encoding: %v != %v \", i, d, results[0])\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdec := NewElementDecoder(rt)\n\t\t\tbuf := bytes.NewBuffer([]byte(results[0]))\n\t\t\tdecoded, err := dec.De
code(buf)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to decode: %q, %v\", results[0], err)\n\t\t\t}\n\n\t\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\t\tt.Errorf(\"Corrupt coding: %v, want %v\", decoded, test)\n\t\t\t}\n\t\t})\n\t}\n}\n"
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam_test\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam\"\n)\n\n// foolFn is a no-op CombineFn.\ntype foolFn struct {\n\tOutp
utType beam.EncodedType\n}\n\ntype foolAccum struct{}\n\nfunc (f *foolFn) CreateAccumulator() *foolAccum {\n\treturn &foolAccum{}\n}\n\nfunc (f *foolFn) AddInput(a *foolAccum, v beam.U) *foolAccum {\n\treturn a\n}\n\nfunc (f *foolFn) MergeAccumulators(a *foolAccum, b *foolAccum) *foolAccum {\n\treturn a\n}\n\nfunc (f *foolFn) ExtractOutput(a *foolAccum) beam.V {\n\treturn reflect.New(f.OutputType.T).Elem().Interface()\n}\n\nfunc TestCombineWithTypeDefinition(t *testing.T) {\n\t_, s := beam.NewPipelineWithRoot()\n\tin := beam.Create(s, 1, 2, 3)\n\tstrType := reflect.TypeOf(\"\")\n\tcombineFn := &foolFn{OutputType: beam.EncodedType{T: strType}}\n\toutput := beam.Combine(s, combineFn, in, beam.TypeDefinition{Var: beam.VType, T: strType})\n\tif output.Type().Type() != strType {\n\t\tt.Errorf(\"expect combine output type to be %v, got %v\", strType, output.Type().Type())\n\t}\n}\n"
Review comment:
Sorry it was draft version, needed it for testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AydarZaynutdinov commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
AydarZaynutdinov commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r764652499
##########
File path: playground/backend/cmd/server/server.go
##########
@@ -107,8 +108,13 @@ func setupCache(ctx context.Context, appEnv environment.ApplicationEnvs) (cache.
}
func main() {
+ os.Setenv("PROTOCOL_TYPE", "TCP")
err := runServer()
if err != nil {
panic(err)
}
}
+
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam_test\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam\"\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/testing/
passert\"\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/testing/ptest\"\n\t\"github.com/golang/protobuf/proto\"\n)\n\ntype wc struct {\n\tK string\n\tV int\n}\n\nfunc TestCreate(t *testing.T) {\n\ttests := []struct {\n\t\tvalues []interface{}\n\t}{\n\t\t{[]interface{}{1, 2, 3}},\n\t\t{[]interface{}{\"1\", \"2\", \"3\"}},\n\t\t{[]interface{}{float32(0.1), float32(0.2), float32(0.3)}},\n\t\t{[]interface{}{float64(0.1), float64(0.2), float64(0.3)}},\n\t\t{[]interface{}{uint(1), uint(2), uint(3)}},\n\t\t{[]interface{}{false, true, true, false, true}},\n\t\t{[]interface{}{wc{\"a\", 23}, wc{\"b\", 42}, wc{\"c\", 5}}},\n\t\t{[]interface{}{&testProto{}, &testProto{stringValue(\"test\")}}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.Create(s, test.values...)\n\t\tpassert.Equals(s, c, test.values...)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.Create(%v) failed: %v\", test.values, err)\n\t\t}\n\t
}\n}\n\nfunc TestCreateList(t *testing.T) {\n\ttests := []struct {\n\t\tvalues interface{}\n\t}{\n\t\t{[]int{1, 2, 3}},\n\t\t{[]string{\"1\", \"2\", \"3\"}},\n\t\t{[]float32{float32(0.1), float32(0.2), float32(0.3)}},\n\t\t{[]float64{float64(0.1), float64(0.2), float64(0.3)}},\n\t\t{[]uint{uint(1), uint(2), uint(3)}},\n\t\t{[]bool{false, true, true, false, true}},\n\t\t{[]wc{wc{\"a\", 23}, wc{\"b\", 42}, wc{\"c\", 5}}},\n\t\t{[]*testProto{&testProto{}, &testProto{stringValue(\"test\")}}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.CreateList(s, test.values)\n\n\t\tvar values []interface{}\n\t\tv := reflect.ValueOf(test.values)\n\t\tfor i := 0; i < v.Len(); i++ {\n\t\t\tvalues = append(values, v.Index(i).Interface())\n\t\t}\n\t\tpassert.Equals(s, c, values...)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.CreateList(%v) failed: %v\", test.values, err)\n\t\t}\n\t}\n}\n\nfunc TestCreateEmptyLi
st(t *testing.T) {\n\ttests := []struct {\n\t\tvalues interface{}\n\t}{\n\t\t{[]int{}},\n\t\t{[]string{}},\n\t\t{[]float32{}},\n\t\t{[]float64{}},\n\t\t{[]uint{}},\n\t\t{[]bool{}},\n\t\t{[]wc{}},\n\t\t{[]*testProto{}}, // Test for BEAM-4401\n\t}\n\n\tfor _, test := range tests {\n\t\tp, s := beam.NewPipelineWithRoot()\n\t\tc := beam.CreateList(s, test.values)\n\n\t\tpassert.Empty(s, c)\n\n\t\tif err := ptest.Run(p); err != nil {\n\t\t\tt.Errorf(\"beam.CreateList(%v) failed: %v\", test.values, err)\n\t\t}\n\t}\n}\n\ntype testProto struct {\n\t// OneOfField is an interface-typed field and cannot be JSON-marshaled, but\n\t// should be specially handled by Beam as a field of a proto.Message.\n\tOneOfField _isOneOfField\n}\n\ntype stringValue string\n\nfunc (stringValue) isOneOfField() {}\n\ntype _isOneOfField interface {\n\tisOneOfField()\n}\n\nfunc (t *testProto) Reset() { *t = testProto{} }\nfunc (t *testProto) String() string { return fmt.Sprintf(\"one_of_field: %#v\", t.OneO
fField) }\nfunc (t *testProto) ProtoMessage() {}\n\nfunc (t *testProto) Marshal() ([]byte, error) {\n\tif t.OneOfField == nil {\n\t\treturn nil, nil\n\t}\n\treturn []byte(t.OneOfField.(stringValue)), nil\n}\nfunc (t *testProto) Unmarshal(b []byte) error {\n\tif len(b) == 0 {\n\t\treturn nil\n\t}\n\tt.OneOfField = stringValue(b)\n\treturn nil\n}\n\n// Ensure testProto is detected as a proto.Message and can be (un)marshalled by\n// the proto library.\nvar (\n\t_ proto.Message = &testProto{}\n\t_ proto.Marshaler = &testProto{}\n\t_ proto.Unmarshaler = &testProto{}\n)\n"
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema\"\n)\n\nfunc TestJSONCo
der(t *testing.T) {\n\tv := \"teststring\"\n\ttests := []interface{}{\n\t\t43,\n\t\t12431235,\n\t\t-2,\n\t\t0,\n\t\t1,\n\t\ttrue,\n\t\t\"a string\",\n\t\tmap[int64]string{1: \"one\", 11: \"oneone\", 21: \"twoone\", 1211: \"onetwooneone\"},\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t}\n\n\tfor _, test := range tests {\n\t\tvar results []string\n\t\tfor i := 0; i < 10; i++ {\n\t\t\tdata, err := jsonEnc(test)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t}\n\t\t\tresults = append(results, string(data))\n\t\t}\n\t\tfor i, data := range results {\n\t\t\tif data != results[0] {\n\t\t\t\tt.Errorf(\"coder not deterministic: data[%d]: %v != %v \", i, data, results[0])\n\t\t\t}\n\t\t}\n\n\t\tdecoded, err := jsonDec(reflect.TypeOf(test), []byte(results[0]))\n\t\tif err != nil {\n\t\t\tt.Fatalf(\"Failed to decode: %v\", err)\n\t\t}\n\n\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\tt.Errorf(\"Corrupt coding: %v,
want %v\", decoded, test)\n\t\t}\n\t}\n}\n\nfunc TestSchemaCoder(t *testing.T) {\n\tv := \"teststring\"\n\ttests := []interface{}{\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t\t&struct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &v, false},\n\t\tstruct {\n\t\t\tA map[string]int\n\t\t\tB []int\n\t\t}{map[string]int{\"three\": 3}, []int{4}},\n\t\tstruct {\n\t\t\tA map[string]int\n\t\t\tB [1]int\n\t\t}{map[string]int{\"three\": 3}, [...]int{4}},\n\t}\n\n\tfor _, test := range tests {\n\t\trt := reflect.TypeOf(test)\n\t\tt.Run(fmt.Sprintf(\"%v\", rt), func(t *testing.T) {\n\t\t\tvar results []string\n\t\t\tfor i := 0; i < 10; i++ {\n\t\t\t\tdata, err := schemaEnc(rt, test)\n\t\t\t\tif err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t\t}\n\t\t\t\tresults = append(results, string(data))\n\t\t\t}\n\t\t\tfor i, data := range results {\n\t\t\t\tif data != results[0] {\n\t\t\t\t\tt.Errorf(\"coder not determinis
tic: data[%d]: %v != %v \", i, data, results[0])\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdecoded, err := schemaDec(rt, []byte(results[0]))\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to decode: %v\", err)\n\t\t\t}\n\n\t\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\t\tt.Errorf(\"Corrupt coding: %v, want %v\", decoded, test)\n\t\t\t}\n\t\t})\n\t}\n}\n\nfunc TestCoders(t *testing.T) {\n\tptrString := \"test *string\"\n\ttype regTestType struct {\n\t\tA [4]int\n\t}\n\tschema.RegisterType(reflect.TypeOf((*regTestType)(nil)))\n\ttests := []interface{}{\n\t\t43,\n\t\t12431235,\n\t\t-2,\n\t\t0,\n\t\t1,\n\t\ttrue,\n\t\t\"a string\",\n\t\tmap[int64]string{1: \"one\", 11: \"oneone\", 21: \"twoone\", 1211: \"onetwooneone\"}, // (not supported by custom type registration)\n\t\tstruct {\n\t\t\tA int\n\t\t\tB *string\n\t\t\tC bool\n\t\t}{4, &ptrString, false},\n\t\t[...]int64{1, 2, 3, 4, 5}, // array (not supported by custom type registration)\n\t\t[]int64{1, 2, 3, 4, 5}, // slice\n\t\tstruct {\n
\t\t\tA []int\n\t\t\tB [3]int\n\t\t}{A: []int{1, 2, 3}, B: [...]int{4, 5, 6}},\n\t\t[...]struct{ A int }{{1}, {2}, {3}, {4}, {5}},\n\t\t[]struct{ B int }{{1}, {2}, {3}, {4}, {5}},\n\t\tregTestType{[4]int{4, 2, 4, 2}},\n\t}\n\n\tfor _, test := range tests {\n\t\tt.Run(fmt.Sprintf(\"%T\", test), func(t *testing.T) {\n\t\t\tvar results []string\n\t\t\trt := reflect.TypeOf(test)\n\t\t\tenc := NewElementEncoder(rt)\n\t\t\tfor i := 0; i < 10; i++ {\n\t\t\t\tvar buf bytes.Buffer\n\t\t\t\tif err := enc.Encode(test, &buf); err != nil {\n\t\t\t\t\tt.Fatalf(\"Failed to encode %v: %v\", test, err)\n\t\t\t\t}\n\t\t\t\tresults = append(results, string(buf.Bytes()))\n\t\t\t}\n\t\t\tfor i, d := range results {\n\t\t\t\tif d != results[0] {\n\t\t\t\t\tt.Errorf(\"coder not deterministic: encoding %d not the same as the first encoding: %v != %v \", i, d, results[0])\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tdec := NewElementDecoder(rt)\n\t\t\tbuf := bytes.NewBuffer([]byte(results[0]))\n\t\t\tdecoded, err := dec.De
code(buf)\n\t\t\tif err != nil {\n\t\t\t\tt.Fatalf(\"Failed to decode: %q, %v\", results[0], err)\n\t\t\t}\n\n\t\t\tif !reflect.DeepEqual(decoded, test) {\n\t\t\t\tt.Errorf(\"Corrupt coding: %v, want %v\", decoded, test)\n\t\t\t}\n\t\t})\n\t}\n}\n"
+// "// Licensed to the Apache Software Foundation (ASF) under one or more\n// contributor license agreements. See the NOTICE file distributed with\n// this work for additional information regarding copyright ownership.\n// The ASF licenses this file to You under the Apache License, Version 2.0\n// (the \"License\"); you may not use this file except in compliance with\n// the License. You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage beam_test\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n\n\t\"github.com/apache/beam/sdks/v2/go/pkg/beam\"\n)\n\n// foolFn is a no-op CombineFn.\ntype foolFn struct {\n\tOutp
utType beam.EncodedType\n}\n\ntype foolAccum struct{}\n\nfunc (f *foolFn) CreateAccumulator() *foolAccum {\n\treturn &foolAccum{}\n}\n\nfunc (f *foolFn) AddInput(a *foolAccum, v beam.U) *foolAccum {\n\treturn a\n}\n\nfunc (f *foolFn) MergeAccumulators(a *foolAccum, b *foolAccum) *foolAccum {\n\treturn a\n}\n\nfunc (f *foolFn) ExtractOutput(a *foolAccum) beam.V {\n\treturn reflect.New(f.OutputType.T).Elem().Interface()\n}\n\nfunc TestCombineWithTypeDefinition(t *testing.T) {\n\t_, s := beam.NewPipelineWithRoot()\n\tin := beam.Create(s, 1, 2, 3)\n\tstrType := reflect.TypeOf(\"\")\n\tcombineFn := &foolFn{OutputType: beam.EncodedType{T: strType}}\n\toutput := beam.Combine(s, combineFn, in, beam.TypeDefinition{Var: beam.VType, T: strType})\n\tif output.Type().Type() != strType {\n\t\tt.Errorf(\"expect combine output type to be %v, got %v\", strType, output.Type().Type())\n\t}\n}\n"
Review comment:
It seems that we do not need this one.
##########
File path: playground/backend/containers/go/Dockerfile
##########
@@ -56,6 +56,14 @@ ENV BEAM_SDK="SDK_GO"
## Copy build result
COPY src/configs /opt/playground/backend/configs/
+# Temp - create grpcox
+# Update aptitude with new repo
+RUN apt-get update
+# Install software
+RUN apt-get install -y git
+WORKDIR /opt
+RUN git clone https://github.com/gusaul/grpcox.git
Review comment:
Why do we need this one?
##########
File path: playground/backend/internal/executors/executor_test.go
##########
@@ -51,7 +51,10 @@ func BaseExecutorBuilder(envs environment.BeamEnvs, workingDir string, filePath
}
builder := NewExecutorBuilder().
WithExecutableFileName("HelloWorld").
- WithWorkingDir(workingDir).
+ WithWorkingDir(workingDir).WithValidator().
Review comment:
```suggestion
WithWorkingDir(workingDir).
WithValidator().
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] daria-malkova commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
daria-malkova commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r765526011
##########
File path: playground/backend/cmd/server/server.go
##########
@@ -107,8 +108,13 @@ func setupCache(ctx context.Context, appEnv environment.ApplicationEnvs) (cache.
}
func main() {
+ os.Setenv("PROTOCOL_TYPE", "TCP")
Review comment:
Sorry it was draft version, needed it for testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AydarZaynutdinov commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
AydarZaynutdinov commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r765806403
##########
File path: playground/backend/containers/go/Dockerfile
##########
@@ -56,6 +56,4 @@ ENV BEAM_SDK="SDK_GO"
## Copy build result
COPY src/configs /opt/playground/backend/configs/
-ENTRYPOINT ["/opt/playground/backend/server_go_backend"]
-
-
+ENTRYPOINT ["/opt/playground/backend/server_go_backend"]
Review comment:
add an empty string.
##########
File path: playground/backend/internal/code_processing/code_processing.go
##########
@@ -88,14 +88,20 @@ func Process(ctx context.Context, cacheService cache.Cache, lc *fs_tool.LifeCycl
_ = processError(ctxWithTimeout, errorChannel, pipelineId, cacheService, "Validate", pb.Status_STATUS_VALIDATION_ERROR)
return
}
+ // Check if unit test
+ isUnitTest := false
+ valResult, ok := validationResults.Load(validators.UnitTestValidatorName)
+ if ok && valResult.(bool) {
+ isUnitTest = true
+ }
Review comment:
Maybe wrap it like a separate method?
##########
File path: playground/backend/internal/preparators/go_preparators.go
##########
@@ -17,31 +17,51 @@ package preparators
import (
"errors"
+ "fmt"
"os/exec"
"path/filepath"
+ "strings"
)
const (
- nameBinGo = "go"
- fmtArgs = "fmt"
+ goName = "go"
Review comment:
maybe `goFileExtension` instead of `goName`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem merged pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
pabloem merged pull request #16138:
URL: https://github.com/apache/beam/pull/16138
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] daria-malkova commented on a change in pull request #16138: [BEAM-13343][Playground] Support go unit tests
Posted by GitBox <gi...@apache.org>.
daria-malkova commented on a change in pull request #16138:
URL: https://github.com/apache/beam/pull/16138#discussion_r767693085
##########
File path: playground/backend/internal/validators/validator_test.go
##########
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package validators
+
+import (
+ "fmt"
+ "os"
+ "testing"
+)
+
+func TestMain(m *testing.M) {
Review comment:
Thanks Pablo, I've added this to PR https://github.com/apache/beam/pull/16206/
##########
File path: playground/backend/internal/validators/validator_test.go
##########
@@ -0,0 +1,56 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package validators
+
+import (
+ "fmt"
+ "os"
+ "testing"
+)
+
+func TestMain(m *testing.M) {
Review comment:
Thanks, Pablo, I've added this to PR https://github.com/apache/beam/pull/16206/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org