You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/12/29 12:01:35 UTC
[GitHub] [beam] pavel-avilov opened a new pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
pavel-avilov opened a new pull request #16384:
URL: https://github.com/apache/beam/pull/16384
[[BEAM-13308]](https://issues.apache.org/jira/browse/BEAM-13308)
Getting the name of folder in which the pipelines resources are stored
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
`ValidatesRunner` compliance status (on master branch)
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon?subject=Portable+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/badge/icon?subject=Portable">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon?subject=Structured+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon?subject=ValCont">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
</tbody>
</table>
Examples testing status on various runners
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/badge/icon?subject=V1+Java11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
Post-Commit SDK/Transform Integration Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>Go</th>
<th>Java</th>
<th>Python</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon?subject=3.6">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon?subject=3.7">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon?subject=3.8">
</a>
</td>
</tr>
</tbody>
</table>
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>---</th>
<th>Java</th>
<th>Python</th>
<th>Go</th>
<th>Website</th>
<th>Whitespace</th>
<th>Typescript</th>
</tr>
</thead>
<tbody>
<tr>
<td>Non-portable</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Tests">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon?subject=Lint">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon?subject=Docker">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon?subject=Docs">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Portable</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AydarZaynutdinov commented on a change in pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
AydarZaynutdinov commented on a change in pull request #16384:
URL: https://github.com/apache/beam/pull/16384#discussion_r776622798
##########
File path: playground/backend/internal/fs_tool/fs.go
##########
@@ -57,14 +57,14 @@ type LifeCycle struct {
// NewLifeCycle returns a corresponding LifeCycle depending on the given SDK.
// workingDir should be existed and be prepared to create/delete/modify folders into him.
Review comment:
We can remove this comment
##########
File path: playground/backend/internal/fs_tool/fs_test.go
##########
@@ -144,7 +145,7 @@ func TestLifeCycle_CreateExecutableFile(t *testing.T) {
func TestLifeCycle_CreateFolders(t *testing.T) {
pipelineId := uuid.New()
- baseFileFolder := fmt.Sprintf("%s_%s", baseFileFolder, pipelineId)
+ baseFileFolder := fmt.Sprintf("%s_%s", pipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := fmt.Sprintf("%s/%s", pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/fs_tool/fs_test.go
##########
@@ -300,7 +302,7 @@ func TestNewLifeCycle(t *testing.T) {
func TestLifeCycle_GetAbsoluteExecutableFilePath(t *testing.T) {
pipelineId := uuid.New()
- baseFileFolder := fmt.Sprintf("%s_%s", baseFileFolder, pipelineId)
+ baseFileFolder := fmt.Sprintf("%s_%s", pipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := fmt.Sprintf("%s/%s", pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/fs_tool/fs_test.go
##########
@@ -186,7 +187,7 @@ func TestLifeCycle_CreateFolders(t *testing.T) {
func TestLifeCycle_DeleteFolders(t *testing.T) {
pipelineId := uuid.New()
- baseFileFolder := fmt.Sprintf("%s_%s", baseFileFolder, pipelineId)
+ baseFileFolder := fmt.Sprintf("%s_%s", pipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := fmt.Sprintf("%s/%s", pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/fs_tool/fs_test.go
##########
@@ -348,7 +350,7 @@ func TestLifeCycle_GetAbsoluteExecutableFilePath(t *testing.T) {
func TestLifeCycle_GetAbsoluteExecutableFilesFolderPath(t *testing.T) {
pipelineId := uuid.New()
- baseFileFolder := fmt.Sprintf("%s_%s", baseFileFolder, pipelineId)
+ baseFileFolder := fmt.Sprintf("%s_%s", pipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := fmt.Sprintf("%s/%s", pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/fs_tool/go_fs_test.go
##########
@@ -18,20 +18,23 @@ package fs_tool
import (
"fmt"
"github.com/google/uuid"
+ "path/filepath"
"reflect"
"testing"
)
func Test_newGoLifeCycle(t *testing.T) {
pipelineId := uuid.New()
workingDir := "workingDir"
- baseFileFolder := fmt.Sprintf("%s/%s/%s", workingDir, baseFileFolder, pipelineId)
+ preparedPipelinesFolder := filepath.Join(workingDir, pipelinesFolder)
+ baseFileFolder := fmt.Sprintf("%s/%s", preparedPipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := filepath.Join(workingDir, pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/fs_tool/java_fs_test.go
##########
@@ -27,13 +27,15 @@ import (
func Test_newJavaLifeCycle(t *testing.T) {
pipelineId := uuid.New()
workingDir := "workingDir"
- baseFileFolder := fmt.Sprintf("%s/%s/%s", workingDir, baseFileFolder, pipelineId)
+ preparedPipelinesFolder := filepath.Join(workingDir, pipelinesFolder)
+ baseFileFolder := fmt.Sprintf("%s/%s", preparedPipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := filepath.Join(workingDir, pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/fs_tool/java_fs_test.go
##########
@@ -45,8 +47,9 @@ func Test_newJavaLifeCycle(t *testing.T) {
// As a result, want to receive an expected java life cycle.
name: "newJavaLifeCycle",
args: args{
- pipelineId: pipelineId,
- workingDir: workingDir,
+ pipelineId: pipelineId,
+ workingDir: workingDir,
+ pipelinesFolder: preparedPipelinesFolder,
Review comment:
```suggestion
pipelinesFolder: baseFileFolder,
```
##########
File path: playground/backend/internal/preparators/java_preparators_test.go
##########
@@ -37,7 +39,7 @@ func Test_replace(t *testing.T) {
if err != nil {
panic(err)
}
- lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, uuid.New(), filepath.Join(path, "temp"))
+ lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, uuid.New(), filepath.Join(path, "temp", pipelinesFolder))
Review comment:
It is not necessary. `fs_tool.NewLifeCycle` receive any directory as the last argument and it doesn't matter if it will be `temp/executable_files` or just `executable_files`.
##########
File path: playground/backend/internal/fs_tool/fs_test.go
##########
@@ -392,7 +394,7 @@ func TestLifeCycle_GetAbsoluteExecutableFilesFolderPath(t *testing.T) {
func TestLifeCycle_ExecutableName(t *testing.T) {
pipelineId := uuid.New()
workingDir := "workingDir"
Review comment:
It seems that we do not need this variable
##########
File path: playground/backend/internal/fs_tool/fs_test.go
##########
@@ -71,7 +72,7 @@ func teardown() {
func TestLifeCycle_CreateExecutableFile(t *testing.T) {
pipelineId := uuid.New()
- baseFileFolder := fmt.Sprintf("%s_%s", baseFileFolder, pipelineId)
+ baseFileFolder := fmt.Sprintf("%s_%s", pipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := fmt.Sprintf("%s/%s", pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/fs_tool/go_fs_test.go
##########
@@ -43,8 +46,8 @@ func Test_newGoLifeCycle(t *testing.T) {
// As a result, want to receive an expected go life cycle.
name: "newGoLifeCycle",
args: args{
- pipelineId: pipelineId,
- workingDir: workingDir,
+ pipelineId: pipelineId,
+ pipelinesFolder: preparedPipelinesFolder,
Review comment:
```suggestion
pipelinesFolder: baseFileFolder,
```
##########
File path: playground/backend/internal/fs_tool/java_fs_test.go
##########
@@ -45,8 +47,9 @@ func Test_newJavaLifeCycle(t *testing.T) {
// As a result, want to receive an expected java life cycle.
name: "newJavaLifeCycle",
args: args{
- pipelineId: pipelineId,
- workingDir: workingDir,
+ pipelineId: pipelineId,
+ workingDir: workingDir,
Review comment:
This field isn't used in the test case. So we can remove it.
##########
File path: playground/backend/internal/fs_tool/java_fs_test.go
##########
@@ -27,13 +27,15 @@ import (
func Test_newJavaLifeCycle(t *testing.T) {
pipelineId := uuid.New()
workingDir := "workingDir"
- baseFileFolder := fmt.Sprintf("%s/%s/%s", workingDir, baseFileFolder, pipelineId)
+ preparedPipelinesFolder := filepath.Join(workingDir, pipelinesFolder)
+ baseFileFolder := fmt.Sprintf("%s/%s", preparedPipelinesFolder, pipelineId)
srcFileFolder := baseFileFolder + "/src"
binFileFolder := baseFileFolder + "/bin"
type args struct {
- pipelineId uuid.UUID
- workingDir string
+ pipelineId uuid.UUID
+ workingDir string
Review comment:
We do not need this field here.
##########
File path: playground/backend/internal/fs_tool/go_fs_test.go
##########
@@ -18,20 +18,23 @@ package fs_tool
import (
"fmt"
"github.com/google/uuid"
+ "path/filepath"
"reflect"
"testing"
)
func Test_newGoLifeCycle(t *testing.T) {
pipelineId := uuid.New()
workingDir := "workingDir"
- baseFileFolder := fmt.Sprintf("%s/%s/%s", workingDir, baseFileFolder, pipelineId)
+ preparedPipelinesFolder := filepath.Join(workingDir, pipelinesFolder)
+ baseFileFolder := fmt.Sprintf("%s/%s", preparedPipelinesFolder, pipelineId)
srcFileFolder := baseFileFolder + "/src"
binFileFolder := baseFileFolder + "/bin"
type args struct {
- pipelineId uuid.UUID
- workingDir string
+ pipelineId uuid.UUID
+ workingDir string
Review comment:
We do not need this field here.
##########
File path: playground/backend/internal/fs_tool/python_fs_test.go
##########
@@ -41,8 +43,8 @@ func Test_newPythonLifeCycle(t *testing.T) {
// As a result, want to receive an expected python life cycle.
name: "newPythonLifeCycle",
args: args{
- pipelineId: pipelineId,
- workingDir: workingDir,
+ pipelineId: pipelineId,
+ pipelinesFolder: preparedPipelinesFolder,
Review comment:
```suggestion
pipelinesFolder: baseFileFolder,
```
##########
File path: playground/backend/internal/fs_tool/python_fs_test.go
##########
@@ -18,18 +18,20 @@ package fs_tool
import (
"fmt"
"github.com/google/uuid"
+ "path/filepath"
"reflect"
"testing"
)
func Test_newPythonLifeCycle(t *testing.T) {
pipelineId := uuid.New()
workingDir := "workingDir"
- baseFileFolder := fmt.Sprintf("%s/%s/%s", workingDir, baseFileFolder, pipelineId)
+ preparedPipelinesFolder := filepath.Join(workingDir, pipelinesFolder)
+ baseFileFolder := fmt.Sprintf("%s/%s", preparedPipelinesFolder, pipelineId)
Review comment:
```suggestion
baseFileFolder := filepath.Join(workingDir, pipelinesFolder, pipelineId)
```
##########
File path: playground/backend/internal/preparators/java_preparators_test.go
##########
@@ -118,7 +120,7 @@ func Test_changeJavaTestFileName(t *testing.T) {
if err != nil {
panic(err)
}
- lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, uuid.New(), filepath.Join(path, "temp"))
+ lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, uuid.New(), filepath.Join(path, "temp", pipelinesFolder))
Review comment:
ditto
##########
File path: playground/backend/internal/setup_tools/life_cycle/life_cycle_setuper_test.go
##########
@@ -47,16 +48,17 @@ func TestSetup(t *testing.T) {
}
defer os.RemoveAll(workingDir)
- lc, err := fs_tool.NewLifeCycle(playground.Sdk_SDK_JAVA, successPipelineId, workingDir)
+ lc, err := fs_tool.NewLifeCycle(playground.Sdk_SDK_JAVA, successPipelineId, filepath.Join(workingDir, pipelinesFolder))
Review comment:
It is not necessary. `fs_tool.NewLifeCycle` receives any directory as the last argument so it doesn't matter if it will `workingDir/executable_files` or just `workingDir`.
##########
File path: playground/backend/internal/setup_tools/life_cycle/life_cycle_setuper.go
##########
@@ -34,14 +35,13 @@ const (
javaLogFilePlaceholder = "{logFilePath}"
goModFileName = "go.mod"
goSumFileName = "go.sum"
- baseFileFolder = "executable_files"
)
// Setup returns fs_tool.LifeCycle.
// Also, prepares files and folders needed to code processing according to sdk
-func Setup(sdk pb.Sdk, code string, pipelineId uuid.UUID, workingDir string, preparedModDir string) (*fs_tool.LifeCycle, error) {
+func Setup(sdk pb.Sdk, code string, pipelineId uuid.UUID, workingDir, pipelinesFolder, preparedModDir string) (*fs_tool.LifeCycle, error) {
Review comment:
Oh I see that we need to use `workingDir` for the `prepareJavaFiles()`. So yes let's leave it as you did.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #16384:
URL: https://github.com/apache/beam/pull/16384#issuecomment-1009440652
thanks all LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pavel-avilov commented on a change in pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
pavel-avilov commented on a change in pull request #16384:
URL: https://github.com/apache/beam/pull/16384#discussion_r781148643
##########
File path: playground/backend/internal/setup_tools/life_cycle/life_cycle_setuper_test.go
##########
@@ -47,16 +48,17 @@ func TestSetup(t *testing.T) {
}
defer os.RemoveAll(workingDir)
- lc, err := fs_tool.NewLifeCycle(playground.Sdk_SDK_JAVA, successPipelineId, workingDir)
+ lc, err := fs_tool.NewLifeCycle(playground.Sdk_SDK_JAVA, successPipelineId, filepath.Join(workingDir, pipelinesFolder))
Review comment:
The test now checks the path with executable_files, so it matter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem merged pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
pabloem merged pull request #16384:
URL: https://github.com/apache/beam/pull/16384
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pavel-avilov commented on pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
pavel-avilov commented on pull request #16384:
URL: https://github.com/apache/beam/pull/16384#issuecomment-1008831544
@AydarZaynutdinov I added your suggested changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AydarZaynutdinov commented on a change in pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
AydarZaynutdinov commented on a change in pull request #16384:
URL: https://github.com/apache/beam/pull/16384#discussion_r776356060
##########
File path: playground/backend/internal/code_processing/code_processing.go
##########
@@ -219,8 +219,8 @@ func getExecuteCmd(valRes *sync.Map, executor *executors.Executor, ctxWithTimeou
}
// setJavaExecutableFile sets executable file name to runner (JAVA class name is known after compilation step)
-func setJavaExecutableFile(lc *fs_tool.LifeCycle, id uuid.UUID, service cache.Cache, ctx context.Context, executorBuilder *executors.ExecutorBuilder, dir string) (executors.Executor, error) {
- className, err := lc.ExecutableName(id, dir)
+func setJavaExecutableFile(lc *fs_tool.LifeCycle, id uuid.UUID, service cache.Cache, ctx context.Context, executorBuilder *executors.ExecutorBuilder, dir, pipelinesFolder string) (executors.Executor, error) {
+ className, err := lc.ExecutableName(id, dir, pipelinesFolder)
Review comment:
```suggestion
className, err := lc.ExecutableName(id, filepath.Join(dir, pipelinesFolder))
```
##########
File path: playground/backend/internal/code_processing/code_processing_test.go
##########
@@ -242,7 +242,7 @@ func Test_Process(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
- lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, tt.args.pipelineId, os.Getenv("APP_WORK_DIR"))
+ lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, tt.args.pipelineId, os.Getenv("APP_WORK_DIR"), pipelinesFolder)
Review comment:
```suggestion
lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, tt.args.pipelineId, filepath.Join(os.Getenv("APP_WORK_DIR"), pipelinesFolder))
```
##########
File path: playground/backend/internal/code_processing/code_processing_test.go
##########
@@ -560,6 +561,7 @@ func Test_setJavaExecutableFile(t *testing.T) {
ctx: context.Background(),
executorBuilder: &executorBuilder,
dir: "",
+ pipelinesFolder: pipelinesFolder,
Review comment:
```suggestion
dir: pipelinesFolder,
```
##########
File path: playground/backend/internal/code_processing/code_processing_test.go
##########
@@ -677,14 +679,14 @@ func teardownBenchmarks() {
if err != nil {
panic(fmt.Errorf("error during test teardown: %s", err.Error()))
}
- err = os.RemoveAll(baseFileFolder)
+ err = os.RemoveAll(pipelinesFolder)
if err != nil {
panic(fmt.Errorf("error during test teardown: %s", err.Error()))
}
}
func prepareFiles(b *testing.B, pipelineId uuid.UUID, code string, sdk pb.Sdk) *fs_tool.LifeCycle {
- lc, err := fs_tool.NewLifeCycle(sdk, pipelineId, "")
+ lc, err := fs_tool.NewLifeCycle(sdk, pipelineId, "", pipelinesFolder)
Review comment:
We could use the old version with only one argument - `dir`. Method `NewLifeCycle ` shouldn't implement any logic like filepath.Join(dir, pipelinesFolder)
##########
File path: playground/backend/internal/fs_tool/fs.go
##########
@@ -51,20 +51,20 @@ type LifeCycle struct {
folderGlobs []string //folders that should be created to process code
Folder Folder
Extension Extension
- ExecutableName func(uuid.UUID, string) (string, error)
+ ExecutableName func(uuid.UUID, string, string) (string, error)
Review comment:
Do not need to send 2 arguments `(dir1, dir2)` and then use something like `filepath.Join(dir1, dir2)`
##########
File path: playground/backend/internal/fs_tool/fs.go
##########
@@ -51,20 +51,20 @@ type LifeCycle struct {
folderGlobs []string //folders that should be created to process code
Folder Folder
Extension Extension
- ExecutableName func(uuid.UUID, string) (string, error)
+ ExecutableName func(uuid.UUID, string, string) (string, error)
pipelineId uuid.UUID
}
// NewLifeCycle returns a corresponding LifeCycle depending on the given SDK.
// workingDir should be existed and be prepared to create/delete/modify folders into him.
-func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir string) (*LifeCycle, error) {
+func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir, pipelinesFolder string) (*LifeCycle, error) {
switch sdk {
case pb.Sdk_SDK_JAVA:
- return newJavaLifeCycle(pipelineId, workingDir), nil
+ return newJavaLifeCycle(pipelineId, workingDir, pipelinesFolder), nil
Review comment:
```suggestion
return newJavaLifeCycle(pipelineId, pipelinesFolder), nil
```
##########
File path: playground/backend/internal/fs_tool/fs.go
##########
@@ -51,20 +51,20 @@ type LifeCycle struct {
folderGlobs []string //folders that should be created to process code
Folder Folder
Extension Extension
- ExecutableName func(uuid.UUID, string) (string, error)
+ ExecutableName func(uuid.UUID, string, string) (string, error)
pipelineId uuid.UUID
}
// NewLifeCycle returns a corresponding LifeCycle depending on the given SDK.
// workingDir should be existed and be prepared to create/delete/modify folders into him.
-func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir string) (*LifeCycle, error) {
+func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir, pipelinesFolder string) (*LifeCycle, error) {
switch sdk {
case pb.Sdk_SDK_JAVA:
- return newJavaLifeCycle(pipelineId, workingDir), nil
+ return newJavaLifeCycle(pipelineId, workingDir, pipelinesFolder), nil
case pb.Sdk_SDK_GO:
- return newGoLifeCycle(pipelineId, workingDir), nil
+ return newGoLifeCycle(pipelineId, workingDir, pipelinesFolder), nil
Review comment:
```suggestion
return newGoLifeCycle(pipelineId, pipelinesFolder), nil
```
##########
File path: playground/backend/internal/fs_tool/go_fs.go
##########
@@ -25,6 +25,6 @@ const (
)
// newGoLifeCycle creates LifeCycle with go SDK environment.
-func newGoLifeCycle(pipelineId uuid.UUID, workingDir string) *LifeCycle {
- return newCompilingLifeCycle(pipelineId, workingDir, goSourceFileExtension, goExecutableFileExtension)
+func newGoLifeCycle(pipelineId uuid.UUID, workingDir, pipelinesFolder string) *LifeCycle {
+ return newCompilingLifeCycle(pipelineId, workingDir, goSourceFileExtension, goExecutableFileExtension, pipelinesFolder)
Review comment:
```suggestion
func newGoLifeCycle(pipelineId uuid.UUID, pipelinesFolder string) *LifeCycle {
return newCompilingLifeCycle(pipelineId, pipelinesFolder, goSourceFileExtension, goExecutableFileExtension)
```
##########
File path: playground/backend/internal/setup_tools/life_cycle/life_cycle_setuper.go
##########
@@ -34,14 +35,13 @@ const (
javaLogFilePlaceholder = "{logFilePath}"
goModFileName = "go.mod"
goSumFileName = "go.sum"
- baseFileFolder = "executable_files"
)
// Setup returns fs_tool.LifeCycle.
// Also, prepares files and folders needed to code processing according to sdk
-func Setup(sdk pb.Sdk, code string, pipelineId uuid.UUID, workingDir string, preparedModDir string) (*fs_tool.LifeCycle, error) {
+func Setup(sdk pb.Sdk, code string, pipelineId uuid.UUID, workingDir, pipelinesFolder, preparedModDir string) (*fs_tool.LifeCycle, error) {
Review comment:
```suggestion
func Setup(sdk pb.Sdk, code string, pipelineId uuid.UUID, pipelinesFolder, preparedModDir string) (*fs_tool.LifeCycle, error) {
```
##########
File path: playground/backend/internal/fs_tool/python_fs.go
##########
@@ -24,6 +24,6 @@ const (
)
// newPythonLifeCycle creates LifeCycle with go SDK environment.
-func newPythonLifeCycle(pipelineId uuid.UUID, workingDir string) *LifeCycle {
- return newInterpretedLifeCycle(pipelineId, workingDir, pythonExecutableFileExtension)
+func newPythonLifeCycle(pipelineId uuid.UUID, workingDir, pipelinesFolder string) *LifeCycle {
+ return newInterpretedLifeCycle(pipelineId, workingDir, pythonExecutableFileExtension, pipelinesFolder)
Review comment:
```suggestion
func newPythonLifeCycle(pipelineId uuid.UUID, pipelinesFolder string) *LifeCycle {
return newInterpretedLifeCycle(pipelineId, pipelinesFolder, pythonExecutableFileExtension)
```
##########
File path: playground/backend/internal/fs_tool/lc_constructor.go
##########
@@ -47,8 +46,8 @@ func newCompilingLifeCycle(pipelineId uuid.UUID, workingDir string, sourceFileEx
}
// newInterpretedLifeCycle creates LifeCycle for interpreted SDK environment.
-func newInterpretedLifeCycle(pipelineId uuid.UUID, workingDir string, sourceFileExtension string) *LifeCycle {
- sourceFileFolder := filepath.Join(workingDir, baseFileFolder, pipelineId.String())
+func newInterpretedLifeCycle(pipelineId uuid.UUID, workingDir, sourceFileExtension, pipelinesFolder string) *LifeCycle {
+ sourceFileFolder := filepath.Join(workingDir, pipelinesFolder, pipelineId.String())
Review comment:
```suggestion
func newInterpretedLifeCycle(pipelineId uuid.UUID, pipelinesFolder, sourceFileExtension string) *LifeCycle {
sourceFileFolder := filepath.Join(pipelinesFolder, pipelineId.String())
```
##########
File path: playground/backend/internal/setup_tools/life_cycle/life_cycle_setuper.go
##########
@@ -140,6 +140,7 @@ func updateJavaLogConfigFile(lc *fs_tool.LifeCycle) error {
}
if err = os.Rename(updatedFile.Name(), logConfigFilePath); err != nil {
+ fmt.Println(err)
Review comment:
If you want to log some error you need to use `logger`or throw the error to the previous method and log it there.
##########
File path: playground/backend/internal/fs_tool/fs.go
##########
@@ -51,20 +51,20 @@ type LifeCycle struct {
folderGlobs []string //folders that should be created to process code
Folder Folder
Extension Extension
- ExecutableName func(uuid.UUID, string) (string, error)
+ ExecutableName func(uuid.UUID, string, string) (string, error)
pipelineId uuid.UUID
}
// NewLifeCycle returns a corresponding LifeCycle depending on the given SDK.
// workingDir should be existed and be prepared to create/delete/modify folders into him.
-func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir string) (*LifeCycle, error) {
+func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir, pipelinesFolder string) (*LifeCycle, error) {
Review comment:
```suggestion
func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, pipelinesFolder string) (*LifeCycle, error) {
```
##########
File path: playground/backend/internal/code_processing/code_processing_test.go
##########
@@ -572,7 +574,7 @@ func Test_setJavaExecutableFile(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
- got, err := setJavaExecutableFile(tt.args.lc, tt.args.id, tt.args.service, tt.args.ctx, tt.args.executorBuilder, tt.args.dir)
+ got, err := setJavaExecutableFile(tt.args.lc, tt.args.id, tt.args.service, tt.args.ctx, tt.args.executorBuilder, tt.args.dir, tt.args.pipelinesFolder)
Review comment:
We could use the old version with `tt.args.dir`. Method `setJavaExecutableFile()` should receive one argument - directory and doesn't implement any logic like `filepath.Join(dir, pipelinesFolder)`
##########
File path: playground/backend/internal/fs_tool/fs.go
##########
@@ -51,20 +51,20 @@ type LifeCycle struct {
folderGlobs []string //folders that should be created to process code
Folder Folder
Extension Extension
- ExecutableName func(uuid.UUID, string) (string, error)
+ ExecutableName func(uuid.UUID, string, string) (string, error)
pipelineId uuid.UUID
}
// NewLifeCycle returns a corresponding LifeCycle depending on the given SDK.
// workingDir should be existed and be prepared to create/delete/modify folders into him.
-func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir string) (*LifeCycle, error) {
+func NewLifeCycle(sdk pb.Sdk, pipelineId uuid.UUID, workingDir, pipelinesFolder string) (*LifeCycle, error) {
switch sdk {
case pb.Sdk_SDK_JAVA:
- return newJavaLifeCycle(pipelineId, workingDir), nil
+ return newJavaLifeCycle(pipelineId, workingDir, pipelinesFolder), nil
case pb.Sdk_SDK_GO:
- return newGoLifeCycle(pipelineId, workingDir), nil
+ return newGoLifeCycle(pipelineId, workingDir, pipelinesFolder), nil
case pb.Sdk_SDK_PYTHON:
- return newPythonLifeCycle(pipelineId, workingDir), nil
+ return newPythonLifeCycle(pipelineId, workingDir, pipelinesFolder), nil
Review comment:
```suggestion
return newPythonLifeCycle(pipelineId, pipelinesFolder), nil
```
##########
File path: playground/backend/internal/fs_tool/java_fs.go
##########
@@ -29,15 +29,15 @@ const (
)
// newJavaLifeCycle creates LifeCycle with java SDK environment.
-func newJavaLifeCycle(pipelineId uuid.UUID, workingDir string) *LifeCycle {
- javaLifeCycle := newCompilingLifeCycle(pipelineId, workingDir, JavaSourceFileExtension, javaCompiledFileExtension)
+func newJavaLifeCycle(pipelineId uuid.UUID, workingDir, pipelinesFolder string) *LifeCycle {
+ javaLifeCycle := newCompilingLifeCycle(pipelineId, workingDir, JavaSourceFileExtension, javaCompiledFileExtension, pipelinesFolder)
javaLifeCycle.ExecutableName = executableName
return javaLifeCycle
}
// executableName returns name that should be executed (HelloWorld for HelloWorld.class for java SDK)
-func executableName(pipelineId uuid.UUID, workingDir string) (string, error) {
- baseFileFolder := filepath.Join(workingDir, baseFileFolder, pipelineId.String())
+func executableName(pipelineId uuid.UUID, workingDir, pipelinesFolder string) (string, error) {
+ baseFileFolder := filepath.Join(workingDir, pipelinesFolder, pipelineId.String())
Review comment:
```suggestion
func executableName(pipelineId uuid.UUID, pipelinesFolder string) (string, error) {
baseFileFolder := filepath.Join(pipelinesFolder, pipelineId.String())
```
##########
File path: playground/backend/internal/code_processing/code_processing_test.go
##########
@@ -534,7 +534,7 @@ func TestGetLastIndex(t *testing.T) {
func Test_setJavaExecutableFile(t *testing.T) {
pipelineId := uuid.New()
- lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, pipelineId, os.Getenv("APP_WORK_DIR"))
+ lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, pipelineId, os.Getenv("APP_WORK_DIR"), pipelinesFolder)
Review comment:
```suggestion
lc, _ := fs_tool.NewLifeCycle(pb.Sdk_SDK_JAVA, pipelineId, filepath.Join(os.Getenv("APP_WORK_DIR"), pipelinesFolder))
```
##########
File path: playground/backend/internal/fs_tool/java_fs.go
##########
@@ -29,15 +29,15 @@ const (
)
// newJavaLifeCycle creates LifeCycle with java SDK environment.
-func newJavaLifeCycle(pipelineId uuid.UUID, workingDir string) *LifeCycle {
- javaLifeCycle := newCompilingLifeCycle(pipelineId, workingDir, JavaSourceFileExtension, javaCompiledFileExtension)
+func newJavaLifeCycle(pipelineId uuid.UUID, workingDir, pipelinesFolder string) *LifeCycle {
+ javaLifeCycle := newCompilingLifeCycle(pipelineId, workingDir, JavaSourceFileExtension, javaCompiledFileExtension, pipelinesFolder)
Review comment:
```suggestion
func newJavaLifeCycle(pipelineId uuid.UUID, pipelinesFolder string) *LifeCycle {
javaLifeCycle := newCompilingLifeCycle(pipelineId, pipelinesFolder, JavaSourceFileExtension, javaCompiledFileExtension)
```
##########
File path: playground/backend/internal/setup_tools/life_cycle/life_cycle_setuper.go
##########
@@ -34,14 +35,13 @@ const (
javaLogFilePlaceholder = "{logFilePath}"
goModFileName = "go.mod"
goSumFileName = "go.sum"
- baseFileFolder = "executable_files"
)
// Setup returns fs_tool.LifeCycle.
// Also, prepares files and folders needed to code processing according to sdk
-func Setup(sdk pb.Sdk, code string, pipelineId uuid.UUID, workingDir string, preparedModDir string) (*fs_tool.LifeCycle, error) {
+func Setup(sdk pb.Sdk, code string, pipelineId uuid.UUID, workingDir, pipelinesFolder, preparedModDir string) (*fs_tool.LifeCycle, error) {
// create file system service
- lc, err := fs_tool.NewLifeCycle(sdk, pipelineId, workingDir)
+ lc, err := fs_tool.NewLifeCycle(sdk, pipelineId, workingDir, pipelinesFolder)
Review comment:
```suggestion
lc, err := fs_tool.NewLifeCycle(sdk, pipelineId, pipelinesFolder)
```
##########
File path: playground/backend/internal/fs_tool/lc_constructor.go
##########
@@ -21,14 +21,13 @@ import (
)
const (
- baseFileFolder = "executable_files"
sourceFolderName = "src"
compiledFolderName = "bin"
)
// newCompilingLifeCycle creates LifeCycle for compiled SDK environment.
-func newCompilingLifeCycle(pipelineId uuid.UUID, workingDir string, sourceFileExtension string, compiledFileExtension string) *LifeCycle {
- baseFileFolder := filepath.Join(workingDir, baseFileFolder, pipelineId.String())
+func newCompilingLifeCycle(pipelineId uuid.UUID, workingDir, sourceFileExtension, compiledFileExtension, pipelinesFolder string) *LifeCycle {
+ baseFileFolder := filepath.Join(workingDir, pipelinesFolder, pipelineId.String())
Review comment:
```suggestion
func newCompilingLifeCycle(pipelineId uuid.UUID, pipelinesFolder, sourceFileExtension, compiledFileExtension string) *LifeCycle {
baseFileFolder := filepath.Join(pipelinesFolder, pipelineId.String())
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pavel-avilov commented on a change in pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
pavel-avilov commented on a change in pull request #16384:
URL: https://github.com/apache/beam/pull/16384#discussion_r776594944
##########
File path: playground/backend/internal/setup_tools/life_cycle/life_cycle_setuper.go
##########
@@ -140,6 +140,7 @@ func updateJavaLogConfigFile(lc *fs_tool.LifeCycle) error {
}
if err = os.Rename(updatedFile.Name(), logConfigFilePath); err != nil {
+ fmt.Println(err)
Review comment:
Remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AydarZaynutdinov commented on pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
AydarZaynutdinov commented on pull request #16384:
URL: https://github.com/apache/beam/pull/16384#issuecomment-1002946291
> @AydarZaynutdinov I added almost all suggested changes. In the life_cycle.Setup method the `workingDir` is used without `pipelinesFolder` (line 64), so this method will still take 2 arguments(`workingDir`, `pipelinesFolder`). What do you think?
I agree that we need here 2 arguments. So yes let's leave it as you did.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pavel-avilov commented on pull request #16384: [BEAM-13308] [Playground] Getting baseFileFolder from environment
Posted by GitBox <gi...@apache.org>.
pavel-avilov commented on pull request #16384:
URL: https://github.com/apache/beam/pull/16384#issuecomment-1002903005
@AydarZaynutdinov I added almost all suggested changes. In the life_cycle.Setup method the `workingDir` is used without `pipelinesFolder` (line 64), so this method will still take 2 arguments(`workingDir`, `pipelinesFolder`). What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org