You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "TSultanov (via GitHub)" <gi...@apache.org> on 2023/02/23 15:14:17 UTC
[GitHub] [beam] TSultanov opened a new pull request, #25610: [Playground] GetMetadata() endpoint
TSultanov opened a new pull request, #25610:
URL: https://github.com/apache/beam/pull/25610
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1130944407
##########
playground/api/v1/api.proto:
##########
@@ -314,6 +314,17 @@ message GetSnippetResponse {
Complexity complexity = 4;
}
+// GetMetadataRequest represents request for runner metadata
+message GetMetadataRequest {}
+
+// GetMetadataResponse contains metadata about the runner
+message GetMetadataResponse {
+ string runner_sdk = 1;
+ optional string build_commit_hash = 2;
+ optional int64 build_commit_timestamp_seconds_since_epoch = 3;
+ optional string beam_sdk_version = 4;
Review Comment:
Me and @TSultanov agreed to drop `optional` for consistency with other endpoints that do not use it. We will revisit this question in the 3rd phase of Playground.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1122033305
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
Review Comment:
We have this file in the repository so that the project is immediately runnable after checkout. So adding it to .gitignore has no effect. Unfortunately, it will always pollute `git status`.
An alternative would be to write these data to a git-ignored .txt file and to download when the frontend starts. The cost is +1 request on each page load. I would rather not do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1148036035
##########
playground/backend/containers/go/build.gradle:
##########
@@ -83,19 +84,22 @@ task copyDockerfileDependencies(type: Copy) {
}
docker {
- name containerImageName(
- name: project.docker_image_default_repo_prefix + "playground-backend-go",
- root: project.rootProject.hasProperty(["docker-repository-root"]) ?
- project.rootProject["docker-repository-root"] :
- project.docker_image_default_repo_root)
- files "./build/"
- tags containerImageTags()
- buildArgs(['BASE_IMAGE': project.rootProject.hasProperty(["base-image"]) ?
- project.rootProject["base-image"] :
- "golang:1.18-bullseye",
- 'SDK_TAG': project.rootProject.hasProperty(["sdk-tag"]) ?
- project.rootProject["sdk-tag"] : project.rootProject.sdk_version,
- 'SDK_TAG_LOCAL': project.rootProject.sdk_version])
+ name containerImageName(
+ name: project.docker_image_default_repo_prefix + "playground-backend-go",
+ root: project.rootProject.hasProperty(["docker-repository-root"]) ?
+ project.rootProject["docker-repository-root"] :
+ project.docker_image_default_repo_root)
+ files "./build/"
+ tags containerImageTags()
+ buildArgs(
+ ['BASE_IMAGE' : project.rootProject.hasProperty(["base-image"]) ?
+ project.rootProject["base-image"] :
+ "golang:1.18-bullseye",
+ 'SDK_TAG' : project.rootProject.hasProperty(["sdk-tag"]) ?
+ project.rootProject["sdk-tag"] : project.rootProject.sdk_version,
+ 'SDK_TAG_LOCAL': project.rootProject.sdk_version,
+ 'GIT_COMMIT' : getGitCommitHash(),
+ 'GIT_TIMESTAMP': getGitCommitTimestamp()])
Review Comment:
IMO this is ok in order to get the reusability
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1121914445
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
Review Comment:
Also, can we determine the path to the script location and use it to construct the full path to this generated file to drop the requirement of being in the playground_components project root?
Or, pass this value as an argument from a gradle task
Or, why not generate this file from gradle directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1128959040
##########
playground/backend/containers/git-functions.gradle:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ext.getGitCommitHash = () -> {
+ def stdout = new ByteArrayOutputStream()
+ exec {
+ executable('git')
+ args('rev-parse', 'HEAD')
+ standardOutput = stdout
+ }
+ return stdout.toString().trim()
+} as Closure<String>
+
+ext.getGitCommitTimestamp = () -> {
+ def stdout = new ByteArrayOutputStream()
+ exec {
+ executable('git')
+ args('show', '-s', '--format=%ct', 'HEAD')
+ standardOutput = stdout
+ }
+ return stdout.toString().trim()
+} as Closure<String>
Review Comment:
We need this in 6 separate locations:
- Building the frontend container.
- Building the router container.
- Building the 4 runners.
The 5 backends may be similar but frontend and backend are still two rather distinct users of this. This motivates to do this in a single location like this because duplicating `git show -s --format=%ct HEAD` and `git rev-parse HEAD` 2-6 times is more confusing.
Next, we cannot do this in Dockerfile because by the time it is built the directory is unaware of `.git`.
This leads to the requirement of having the values before the 6 containers' builds start. And their builds are triggered from Gradle. So we cannot shorten this to `command $(git rev-parse HEAD)`, we would still have the whole `exec { ... }` thing but in 12 locations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25610: [Playground] GetMetadata() endpoint
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25610:
URL: https://github.com/apache/beam/pull/25610#issuecomment-1456151277
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1126139909
##########
playground/frontend/playground_components/lib/src/repositories/backend_urls.dart:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// ignore_for_file: avoid_print
+
+import '../../playground_components.dart';
+import '../constants/backend_urls.dart';
+
+const _routerNode = 'router';
+
+Future<Uri> getRouterUrl() => _getBackendUrl(_routerNode);
+Future<Uri> getRunnerUrl(Sdk sdk) => _getBackendUrl(sdk.id);
+
+/// Returns options for backend URLs for [node].
+///
+/// If an override is given in [backendUrlOverrides], it is the only option.
+/// This can be used for development if the production backend
+/// must not be used.
+///
+/// Otherwise there are 2 options in the following order:
+/// 1. [node] is prepended to the host of the current URL.
+/// 2. [node] is inserted into [defaultBackendUrlTemplate] (the production URL).
+/// If the two are the same, only one is returned.
+/// This results in the following when looking up "node":
+///
+/// For production Playground:
+/// - node.play.beam.apache.org (the production URL).
+///
+/// For production Tour of Beam:
+/// - node.play.beam.apache.org (as node.tour.beam.apache.org is blacklisted
+/// via [skipBackendUrls]).
+///
+/// For any other stage of Playground or Tour of Beam (my-stage.com):
+/// - node.my-stage.com
+/// - node.play.beam.apache.org
+/// This means that if the stage runs its own container for the given
+/// backend node, it is used. Otherwise the production backend is used.
+List<Uri> _getBackendUrlOptions(String node) {
+ final override = backendUrlOverrides[node];
+ if (override != null) {
+ return [Uri.parse(override)];
+ }
+
+ final currentHost = Uri.base.host;
+ final nodeUriFromCurrentHost = Uri.base.replace(
+ host: '$node.$currentHost',
+ path: '',
+ queryParameters: {},
+ ).removeFragment();
+
+ return {
+ nodeUriFromCurrentHost.toString(),
+ defaultBackendUrlTemplate.replaceAll('{node}', node),
+ }.where(_shouldAttemptUrl).map(Uri.parse).toList();
+}
+
+/// Whether [url] does not match any pattern in [skipBackendUrls].
+bool _shouldAttemptUrl(String url) {
+ return !skipBackendUrls.any((pattern) => pattern.allMatches(url).isNotEmpty);
+}
+
+/// Returns the URL for the backend [node].
+///
+/// Calls [_getBackendUrlOptions] for options.
+///
+/// If only one option exists, returns it.
+/// This ensures fast initialization in production.
+///
+/// If multiple options exist, each of them except the last one is probed
+/// with getMetadata() call. This results in slower initialization for
+/// custom stages but keeps the configuration simple.
+Future<Uri> _getBackendUrl(String node) async {
+ final urls = _getBackendUrlOptions(node);
+
+ if (urls.length == 1) {
+ return urls.first;
+ }
+
+ print('Probing multiple options for $node backend:');
Review Comment:
This code behaves differently locally vs staging vs prod, so better logging is required.
##########
playground/frontend/playground_components/lib/src/repositories/backend_urls.dart:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// ignore_for_file: avoid_print
+
+import '../../playground_components.dart';
+import '../constants/backend_urls.dart';
+
+const _routerNode = 'router';
+
+Future<Uri> getRouterUrl() => _getBackendUrl(_routerNode);
+Future<Uri> getRunnerUrl(Sdk sdk) => _getBackendUrl(sdk.id);
+
+/// Returns options for backend URLs for [node].
+///
+/// If an override is given in [backendUrlOverrides], it is the only option.
+/// This can be used for development if the production backend
+/// must not be used.
+///
+/// Otherwise there are 2 options in the following order:
+/// 1. [node] is prepended to the host of the current URL.
+/// 2. [node] is inserted into [defaultBackendUrlTemplate] (the production URL).
+/// If the two are the same, only one is returned.
+/// This results in the following when looking up "node":
+///
+/// For production Playground:
+/// - node.play.beam.apache.org (the production URL).
+///
+/// For production Tour of Beam:
+/// - node.play.beam.apache.org (as node.tour.beam.apache.org is blacklisted
+/// via [skipBackendUrls]).
+///
+/// For any other stage of Playground or Tour of Beam (my-stage.com):
+/// - node.my-stage.com
+/// - node.play.beam.apache.org
+/// This means that if the stage runs its own container for the given
+/// backend node, it is used. Otherwise the production backend is used.
+List<Uri> _getBackendUrlOptions(String node) {
+ final override = backendUrlOverrides[node];
+ if (override != null) {
+ return [Uri.parse(override)];
+ }
+
+ final currentHost = Uri.base.host;
+ final nodeUriFromCurrentHost = Uri.base.replace(
+ host: '$node.$currentHost',
+ path: '',
+ queryParameters: {},
+ ).removeFragment();
+
+ return {
+ nodeUriFromCurrentHost.toString(),
+ defaultBackendUrlTemplate.replaceAll('{node}', node),
+ }.where(_shouldAttemptUrl).map(Uri.parse).toList();
+}
+
+/// Whether [url] does not match any pattern in [skipBackendUrls].
+bool _shouldAttemptUrl(String url) {
+ return !skipBackendUrls.any((pattern) => pattern.allMatches(url).isNotEmpty);
+}
+
+/// Returns the URL for the backend [node].
+///
+/// Calls [_getBackendUrlOptions] for options.
+///
+/// If only one option exists, returns it.
+/// This ensures fast initialization in production.
+///
+/// If multiple options exist, each of them except the last one is probed
+/// with getMetadata() call. This results in slower initialization for
+/// custom stages but keeps the configuration simple.
+Future<Uri> _getBackendUrl(String node) async {
+ final urls = _getBackendUrlOptions(node);
+
+ if (urls.length == 1) {
+ return urls.first;
+ }
+
+ print('Probing multiple options for $node backend:');
+ urls.forEach(print);
+
+ final lastUrl = urls.removeLast();
+
+ for (final url in urls) {
+ try {
+ final client = GrpcExampleClient(url: url);
+ await client.getMetadata();
+ print('Using $url');
+ return url;
+ } catch (ex) {
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123054451
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE=$(realpath "$(realpath $(dirname $0))/../lib/src/build_metadata.g.dart")
+
+cat > $FILE << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+EOF
+
+printf 'const buildCommitHash = ' >> $FILE
Review Comment:
```suggestion
printf 'const buildCommitHash = %s;\n' $(git rev-parse --sq HEAD) >> $FILE
```
@alexeyinkin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123527885
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,37 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Usage
+# ./generate_build_metadata.sh <build_commit_hash> <build_commit_seconds_since_epoch>
+
+FILE=$(realpath "$(dirname "$0")/../lib/src/build_metadata.g.dart")
+
+cat > "$FILE" << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// Keep these constants \`null\` in the repository.
+EOF
+
+echo "const buildCommitHash = '$1';" >> "$FILE"
Review Comment:
Wonder if moving this into the `cat > .. << EOF` block will work. If so, this will be a nice looking script
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1126138344
##########
playground/frontend/playground_components/lib/src/controllers/build_metadata.dart:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import 'dart:async';
+
+import 'package:flutter/foundation.dart';
+import 'package:get_it/get_it.dart';
+
+import '../api/v1/api.pbgrpc.dart' show GetMetadataResponse;
+import '../models/component_version.dart';
+import '../models/sdk.dart';
+import '../repositories/code_repository.dart';
+import '../repositories/example_repository.dart';
+import '../repositories/get_metadata_response_grpc_extension.dart';
+
+/// Obtains versions from the backend.
+class BuildMetadataController extends ChangeNotifier {
+ ComponentVersion? _routerVersion;
+ Future<GetMetadataResponse>? _routerVersionFuture;
+
+ final _runnerVersions = <Sdk, ComponentVersion>{};
+ final _runnerVersionFutures = <Sdk, Future<GetMetadataResponse>>{};
+
+ /// Returns the router version and starts loading if it is not started yet.
+ ComponentVersion? get routerVersion {
Review Comment:
`+`
##########
playground/frontend/playground_components/lib/src/repositories/backend_urls.dart:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// ignore_for_file: avoid_print
+
+import '../../playground_components.dart';
+import '../constants/backend_urls.dart';
+
+const _routerNode = 'router';
+
+Future<Uri> getRouterUrl() => _getBackendUrl(_routerNode);
+Future<Uri> getRunnerUrl(Sdk sdk) => _getBackendUrl(sdk.id);
+
+/// Returns options for backend URLs for [node].
+///
+/// If an override is given in [backendUrlOverrides], it is the only option.
+/// This can be used for development if the production backend
+/// must not be used.
+///
+/// Otherwise there are 2 options in the following order:
+/// 1. [node] is prepended to the host of the current URL.
+/// 2. [node] is inserted into [defaultBackendUrlTemplate] (the production URL).
+/// If the two are the same, only one is returned.
+/// This results in the following when looking up "node":
+///
+/// For production Playground:
+/// - node.play.beam.apache.org (the production URL).
+///
+/// For production Tour of Beam:
+/// - node.play.beam.apache.org (as node.tour.beam.apache.org is blacklisted
+/// via [skipBackendUrls]).
+///
+/// For any other stage of Playground or Tour of Beam (my-stage.com):
+/// - node.my-stage.com
+/// - node.play.beam.apache.org
+/// This means that if the stage runs its own container for the given
+/// backend node, it is used. Otherwise the production backend is used.
+List<Uri> _getBackendUrlOptions(String node) {
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1127377894
##########
playground/frontend/CONTRIBUTE.md:
##########
@@ -66,25 +66,6 @@ contains decoupled playground page state and module providers like `OutputPlacem
For quick start up, please take a look
to [this guide](https://docs.flutter.dev/development/data-and-backend/state-mgmt/simple)
-## Configuration
Review Comment:
@alexeyinkin can we make sure that we have the naming convention, on which this new configless approach is based, documented as part of the deployment too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1121904414
##########
playground/frontend/Dockerfile:
##########
@@ -23,8 +23,13 @@ ARG FLUTTER_VERSION=3.7.3
# Setting up.
# ------------------------------------------------
+COPY . /app
+WORKDIR /app
+
RUN apt-get update && apt-get install -y wget xz-utils git
+RUN cat /app/playground_components/lib/src/build_metadata.g.dart
Review Comment:
Remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123060713
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE=$(realpath "$(realpath $(dirname $0))/../lib/src/build_metadata.g.dart")
+
+cat > $FILE << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+EOF
+
+printf 'const buildCommitHash = ' >> $FILE
Review Comment:
Or even better, something like
```shell
GIT_COMMIT=$(git rev-parse --sq HEAD)
...
printf 'const buildCommitHash = %s;\n' $GIT_COMMIT >> $FILE
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123054451
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE=$(realpath "$(realpath $(dirname $0))/../lib/src/build_metadata.g.dart")
+
+cat > $FILE << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+EOF
+
+printf 'const buildCommitHash = ' >> $FILE
Review Comment:
```suggestion
printf 'const buildCommitHash = %s;\n' $(git rev-parse --sq HEAD)
```
@alexeyinkin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123048269
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
Review Comment:
`+` determining from the script location.
I would rather not use Gradle for this because Groovy files are currently messy and we cannot do this in Kotlin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123046540
##########
playground/frontend/Dockerfile:
##########
@@ -23,8 +23,13 @@ ARG FLUTTER_VERSION=3.7.3
# Setting up.
# ------------------------------------------------
+COPY . /app
+WORKDIR /app
+
RUN apt-get update && apt-get install -y wget xz-utils git
+RUN cat /app/playground_components/lib/src/build_metadata.g.dart
Review Comment:
I would rather keep it in build logs. If anything goes wrong with generation, it will be problematic to extract this file content.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damondouglas commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "damondouglas (via GitHub)" <gi...@apache.org>.
damondouglas commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1128645798
##########
learning/tour-of-beam/backend/README.md:
##########
@@ -39,6 +39,7 @@ To re-generate:
$ go generate -x ./...
```
+> Note: this requires [`protoc`](https://grpc.io/docs/protoc-installation/) version `>= 3.15` and [`moq`](https://github.com/matryer/moq) tools to be installed
Review Comment:
Does this generate the proto stubs? I think we should either use buf which is already used for the Beam Playground here. Or convert the Beam Playground to this. Either way, both should be the same.
##########
playground/api/v1/api.proto:
##########
@@ -314,6 +314,17 @@ message GetSnippetResponse {
Complexity complexity = 4;
}
+// GetMetadataRequest represents request for runner metadata
+message GetMetadataRequest {}
+
+// GetMetadataResponse contains metadata about the runner
+message GetMetadataResponse {
+ string runner_sdk = 1;
+ optional string build_commit_hash = 2;
+ optional int64 build_commit_timestamp_seconds_since_epoch = 3;
+ optional string beam_sdk_version = 4;
Review Comment:
> Beginning in protobuf v3.14, primitive fields can distinguish between the default value and unset value by using the [optional keyword](https://cloud.google.com/apis/design/design_patterns.md#optional_primitive_fields), although this is generally discouraged.
See: https://cloud.google.com/apis/design/proto3
##########
playground/backend/cmd/server/controller.go:
##########
@@ -529,3 +529,31 @@ func (controller *playgroundController) GetSnippet(ctx context.Context, info *pb
}
return &response, nil
}
+
+// GetMetadata returns runner metadata
+func (controller *playgroundController) GetMetadata(_ context.Context, _ *pb.GetMetadataRequest) (*pb.GetMetadataResponse, error) {
+ commitTimestampInteger := func() *int64 {
+ timestamp, err := strconv.ParseInt(BuildCommitTimestamp, 10, 64)
+ if err != nil {
+ logger.Errorf("GetMetadata(): failed to parse BuildCommitTimestamp (\"%s\"): %s", BuildCommitTimestamp, err.Error())
+ return nil
+ }
+ return ×tamp
+ }()
+
+ buildCommitHash := func() *string {
+ if BuildCommitHash == "" {
+ return nil
+ }
+ return &BuildCommitHash
+ }()
Review Comment:
This unreadable code block is due to the use of the optional keyword [here](https://github.com/akvelon/beam/blob/get-metadata-endpoint-playground/playground/api/v1/api.proto#L323) [which is discouraged](https://cloud.google.com/apis/design/proto3). The optional keyword leads to a string* instead of just a string.
##########
playground/backend/containers/git-functions.gradle:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ext.getGitCommitHash = () -> {
+ def stdout = new ByteArrayOutputStream()
+ exec {
+ executable('git')
+ args('rev-parse', 'HEAD')
+ standardOutput = stdout
+ }
+ return stdout.toString().trim()
+} as Closure<String>
+
+ext.getGitCommitTimestamp = () -> {
+ def stdout = new ByteArrayOutputStream()
+ exec {
+ executable('git')
+ args('show', '-s', '--format=%ct', 'HEAD')
+ standardOutput = stdout
+ }
+ return stdout.toString().trim()
+} as Closure<String>
Review Comment:
Why is a gradle wrapper needed? Why not just run the command in the context where the value is needed?
##########
playground/backend/containers/go/build.gradle:
##########
@@ -83,19 +84,22 @@ task copyDockerfileDependencies(type: Copy) {
}
docker {
- name containerImageName(
- name: project.docker_image_default_repo_prefix + "playground-backend-go",
- root: project.rootProject.hasProperty(["docker-repository-root"]) ?
- project.rootProject["docker-repository-root"] :
- project.docker_image_default_repo_root)
- files "./build/"
- tags containerImageTags()
- buildArgs(['BASE_IMAGE': project.rootProject.hasProperty(["base-image"]) ?
- project.rootProject["base-image"] :
- "golang:1.18-bullseye",
- 'SDK_TAG': project.rootProject.hasProperty(["sdk-tag"]) ?
- project.rootProject["sdk-tag"] : project.rootProject.sdk_version,
- 'SDK_TAG_LOCAL': project.rootProject.sdk_version])
+ name containerImageName(
+ name: project.docker_image_default_repo_prefix + "playground-backend-go",
+ root: project.rootProject.hasProperty(["docker-repository-root"]) ?
+ project.rootProject["docker-repository-root"] :
+ project.docker_image_default_repo_root)
+ files "./build/"
+ tags containerImageTags()
+ buildArgs(
+ ['BASE_IMAGE' : project.rootProject.hasProperty(["base-image"]) ?
+ project.rootProject["base-image"] :
+ "golang:1.18-bullseye",
+ 'SDK_TAG' : project.rootProject.hasProperty(["sdk-tag"]) ?
+ project.rootProject["sdk-tag"] : project.rootProject.sdk_version,
+ 'SDK_TAG_LOCAL': project.rootProject.sdk_version,
+ 'GIT_COMMIT' : getGitCommitHash(),
+ 'GIT_TIMESTAMP': getGitCommitTimestamp()])
Review Comment:
The risk of wrapping more and more commands with gradle, the difficult it will be to troubleshoot in the future.
##########
playground/backend/containers/java/Dockerfile:
##########
@@ -55,6 +57,7 @@ FROM apache/beam_java8_sdk:$BEAM_VERSION
ARG BEAM_VERSION
ARG SPRING_VERSION=5.3.25
ARG KAFKA_CLIENTS_VERSION=2.3.1
+ENV BEAM_VERSION=$BEAM_VERSION
Review Comment:
We should specify environment variables outside the Docker image using ConfigMap
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1130651154
##########
playground/backend/containers/java/Dockerfile:
##########
@@ -55,6 +57,7 @@ FROM apache/beam_java8_sdk:$BEAM_VERSION
ARG BEAM_VERSION
ARG SPRING_VERSION=5.3.25
ARG KAFKA_CLIENTS_VERSION=2.3.1
+ENV BEAM_VERSION=$BEAM_VERSION
Review Comment:
This is not a runtime configuration parameter. This variable holds the value of Beam SDK version with which the image was built. Storing this value outside of the container makes little sense to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1126349574
##########
playground/frontend/playground_components/lib/src/repositories/code_repository.dart:
##########
@@ -35,9 +35,11 @@ const kProcessingStartedText = 'The processing has started\n';
// TODO(alexeyinkin): Rename. This is not a repository but a higher level client.
class CodeRepository {
- final CodeClient _client;
+ final CodeClient client;
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1116811288
##########
playground/backend/containers/git-functions.gradle:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ext.getGitCommitHash = () -> {
+ def stdout = new ByteArrayOutputStream()
+ exec {
+ executable 'git'
Review Comment:
I haven't found any good ways how to import functions defined in Kotlin DSL into Gradle Groovy. If you have suggestions, I'll be glad to hear them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123174786
##########
playground/frontend/playground_components/lib/src/build_metadata.g.dart:
##########
@@ -0,0 +1,9 @@
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+const buildCommitHash = '0d41f157812fdc3227a6ea93429d3096aa03a850' ;
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123061917
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE=$(realpath "$(realpath $(dirname $0))/../lib/src/build_metadata.g.dart")
+
+cat > $FILE << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+EOF
+
+printf 'const buildCommitHash = ' >> $FILE
+git rev-parse --sq HEAD >> $FILE
+echo ';' >> $FILE
+
+printf 'const buildCommitSecondsSinceEpoch = ' >> $FILE
Review Comment:
Or better
```bash
GIT_COMMIT_TIMESTAMP=$(git show -s --format=%ct HEAD)
...
printf 'const buildCommitSecondsSinceEpoch = %s;\n' $GIT_COMMIT_TIMESTAMP >> $FILE
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Malarg commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "Malarg (via GitHub)" <gi...@apache.org>.
Malarg commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1122611303
##########
playground/frontend/playground_components/lib/src/controllers/build_metadata.dart:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import 'dart:async';
+
+import 'package:flutter/foundation.dart';
+import 'package:get_it/get_it.dart';
+
+import '../api/v1/api.pbgrpc.dart' show GetMetadataResponse;
+import '../models/component_version.dart';
+import '../models/sdk.dart';
+import '../repositories/code_repository.dart';
+import '../repositories/example_repository.dart';
+import '../repositories/get_metadata_response_grpc_extension.dart';
+
+/// Obtains versions from the backend.
+class BuildMetadataController extends ChangeNotifier {
+ ComponentVersion? _routerVersion;
+ Future<GetMetadataResponse>? _routerVersionFuture;
+
+ final _runnerVersions = <Sdk, ComponentVersion>{};
+ final _runnerVersionFutures = <Sdk, Future<GetMetadataResponse>>{};
+
+ /// Returns the router version and starts loading if it is not started yet.
+ ComponentVersion? get routerVersion {
Review Comment:
such way don't assure that it returns version and client won't know when it will be available. Suggest make it future and return cached data if the have been loaded
##########
playground/frontend/playground_components/lib/src/repositories/backend_urls.dart:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// ignore_for_file: avoid_print
+
+import '../../playground_components.dart';
+import '../constants/backend_urls.dart';
+
+const _routerNode = 'router';
+
+Future<Uri> getRouterUrl() => _getBackendUrl(_routerNode);
+Future<Uri> getRunnerUrl(Sdk sdk) => _getBackendUrl(sdk.id);
+
+/// Returns options for backend URLs for [node].
+///
+/// If an override is given in [backendUrlOverrides], it is the only option.
+/// This can be used for development if the production backend
+/// must not be used.
+///
+/// Otherwise there are 2 options in the following order:
+/// 1. [node] is prepended to the host of the current URL.
+/// 2. [node] is inserted into [defaultBackendUrlTemplate] (the production URL).
+/// If the two are the same, only one is returned.
+/// This results in the following when looking up "node":
+///
+/// For production Playground:
+/// - node.play.beam.apache.org (the production URL).
+///
+/// For production Tour of Beam:
+/// - node.play.beam.apache.org (as node.tour.beam.apache.org is blacklisted
+/// via [skipBackendUrls]).
+///
+/// For any other stage of Playground or Tour of Beam (my-stage.com):
+/// - node.my-stage.com
+/// - node.play.beam.apache.org
+/// This means that if the stage runs its own container for the given
+/// backend node, it is used. Otherwise the production backend is used.
+List<Uri> _getBackendUrlOptions(String node) {
Review Comment:
place this method under `_getBackendUrl`
##########
playground/frontend/playground_components/lib/src/repositories/code_repository.dart:
##########
@@ -35,9 +35,11 @@ const kProcessingStartedText = 'The processing has started\n';
// TODO(alexeyinkin): Rename. This is not a repository but a higher level client.
class CodeRepository {
- final CodeClient _client;
+ final CodeClient client;
Review Comment:
Client changed to public here just because of one method in `build_metadata.dart`. I suggest to keep this field incapsulated and turn back to private
##########
playground/frontend/playground_components/lib/src/repositories/backend_urls.dart:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// ignore_for_file: avoid_print
+
+import '../../playground_components.dart';
+import '../constants/backend_urls.dart';
+
+const _routerNode = 'router';
+
+Future<Uri> getRouterUrl() => _getBackendUrl(_routerNode);
+Future<Uri> getRunnerUrl(Sdk sdk) => _getBackendUrl(sdk.id);
+
+/// Returns options for backend URLs for [node].
+///
+/// If an override is given in [backendUrlOverrides], it is the only option.
+/// This can be used for development if the production backend
+/// must not be used.
+///
+/// Otherwise there are 2 options in the following order:
+/// 1. [node] is prepended to the host of the current URL.
+/// 2. [node] is inserted into [defaultBackendUrlTemplate] (the production URL).
+/// If the two are the same, only one is returned.
+/// This results in the following when looking up "node":
+///
+/// For production Playground:
+/// - node.play.beam.apache.org (the production URL).
+///
+/// For production Tour of Beam:
+/// - node.play.beam.apache.org (as node.tour.beam.apache.org is blacklisted
+/// via [skipBackendUrls]).
+///
+/// For any other stage of Playground or Tour of Beam (my-stage.com):
+/// - node.my-stage.com
+/// - node.play.beam.apache.org
+/// This means that if the stage runs its own container for the given
+/// backend node, it is used. Otherwise the production backend is used.
+List<Uri> _getBackendUrlOptions(String node) {
+ final override = backendUrlOverrides[node];
+ if (override != null) {
+ return [Uri.parse(override)];
+ }
+
+ final currentHost = Uri.base.host;
+ final nodeUriFromCurrentHost = Uri.base.replace(
+ host: '$node.$currentHost',
+ path: '',
+ queryParameters: {},
+ ).removeFragment();
+
+ return {
+ nodeUriFromCurrentHost.toString(),
+ defaultBackendUrlTemplate.replaceAll('{node}', node),
+ }.where(_shouldAttemptUrl).map(Uri.parse).toList();
+}
+
+/// Whether [url] does not match any pattern in [skipBackendUrls].
+bool _shouldAttemptUrl(String url) {
+ return !skipBackendUrls.any((pattern) => pattern.allMatches(url).isNotEmpty);
+}
+
+/// Returns the URL for the backend [node].
+///
+/// Calls [_getBackendUrlOptions] for options.
+///
+/// If only one option exists, returns it.
+/// This ensures fast initialization in production.
+///
+/// If multiple options exist, each of them except the last one is probed
+/// with getMetadata() call. This results in slower initialization for
+/// custom stages but keeps the configuration simple.
+Future<Uri> _getBackendUrl(String node) async {
+ final urls = _getBackendUrlOptions(node);
+
+ if (urls.length == 1) {
+ return urls.first;
+ }
+
+ print('Probing multiple options for $node backend:');
Review Comment:
all of these `print`s are required?
##########
playground/frontend/playground_components/lib/src/controllers/build_metadata.dart:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import 'dart:async';
+
+import 'package:flutter/foundation.dart';
+import 'package:get_it/get_it.dart';
+
+import '../api/v1/api.pbgrpc.dart' show GetMetadataResponse;
+import '../models/component_version.dart';
+import '../models/sdk.dart';
+import '../repositories/code_repository.dart';
+import '../repositories/example_repository.dart';
+import '../repositories/get_metadata_response_grpc_extension.dart';
+
+/// Obtains versions from the backend.
+class BuildMetadataController extends ChangeNotifier {
+ ComponentVersion? _routerVersion;
+ Future<GetMetadataResponse>? _routerVersionFuture;
+
+ final _runnerVersions = <Sdk, ComponentVersion>{};
+ final _runnerVersionFutures = <Sdk, Future<GetMetadataResponse>>{};
+
+ /// Returns the router version and starts loading if it is not started yet.
+ ComponentVersion? get routerVersion {
Review Comment:
The same about `getRunnerVersion`
##########
playground/frontend/playground_components/lib/src/repositories/backend_urls.dart:
##########
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// ignore_for_file: avoid_print
+
+import '../../playground_components.dart';
+import '../constants/backend_urls.dart';
+
+const _routerNode = 'router';
+
+Future<Uri> getRouterUrl() => _getBackendUrl(_routerNode);
+Future<Uri> getRunnerUrl(Sdk sdk) => _getBackendUrl(sdk.id);
+
+/// Returns options for backend URLs for [node].
+///
+/// If an override is given in [backendUrlOverrides], it is the only option.
+/// This can be used for development if the production backend
+/// must not be used.
+///
+/// Otherwise there are 2 options in the following order:
+/// 1. [node] is prepended to the host of the current URL.
+/// 2. [node] is inserted into [defaultBackendUrlTemplate] (the production URL).
+/// If the two are the same, only one is returned.
+/// This results in the following when looking up "node":
+///
+/// For production Playground:
+/// - node.play.beam.apache.org (the production URL).
+///
+/// For production Tour of Beam:
+/// - node.play.beam.apache.org (as node.tour.beam.apache.org is blacklisted
+/// via [skipBackendUrls]).
+///
+/// For any other stage of Playground or Tour of Beam (my-stage.com):
+/// - node.my-stage.com
+/// - node.play.beam.apache.org
+/// This means that if the stage runs its own container for the given
+/// backend node, it is used. Otherwise the production backend is used.
+List<Uri> _getBackendUrlOptions(String node) {
+ final override = backendUrlOverrides[node];
+ if (override != null) {
+ return [Uri.parse(override)];
+ }
+
+ final currentHost = Uri.base.host;
+ final nodeUriFromCurrentHost = Uri.base.replace(
+ host: '$node.$currentHost',
+ path: '',
+ queryParameters: {},
+ ).removeFragment();
+
+ return {
+ nodeUriFromCurrentHost.toString(),
+ defaultBackendUrlTemplate.replaceAll('{node}', node),
+ }.where(_shouldAttemptUrl).map(Uri.parse).toList();
+}
+
+/// Whether [url] does not match any pattern in [skipBackendUrls].
+bool _shouldAttemptUrl(String url) {
+ return !skipBackendUrls.any((pattern) => pattern.allMatches(url).isNotEmpty);
+}
+
+/// Returns the URL for the backend [node].
+///
+/// Calls [_getBackendUrlOptions] for options.
+///
+/// If only one option exists, returns it.
+/// This ensures fast initialization in production.
+///
+/// If multiple options exist, each of them except the last one is probed
+/// with getMetadata() call. This results in slower initialization for
+/// custom stages but keeps the configuration simple.
+Future<Uri> _getBackendUrl(String node) async {
+ final urls = _getBackendUrlOptions(node);
+
+ if (urls.length == 1) {
+ return urls.first;
+ }
+
+ print('Probing multiple options for $node backend:');
+ urls.forEach(print);
+
+ final lastUrl = urls.removeLast();
+
+ for (final url in urls) {
+ try {
+ final client = GrpcExampleClient(url: url);
+ await client.getMetadata();
+ print('Using $url');
+ return url;
+ } catch (ex) {
Review Comment:
avoid catches without on clauses
##########
playground/frontend/playground_components/lib/src/controllers/build_metadata.dart:
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import 'dart:async';
+
+import 'package:flutter/foundation.dart';
+import 'package:get_it/get_it.dart';
+
+import '../api/v1/api.pbgrpc.dart' show GetMetadataResponse;
+import '../models/component_version.dart';
+import '../models/sdk.dart';
+import '../repositories/code_repository.dart';
+import '../repositories/example_repository.dart';
+import '../repositories/get_metadata_response_grpc_extension.dart';
+
+/// Obtains versions from the backend.
+class BuildMetadataController extends ChangeNotifier {
+ ComponentVersion? _routerVersion;
+ Future<GetMetadataResponse>? _routerVersionFuture;
+
+ final _runnerVersions = <Sdk, ComponentVersion>{};
+ final _runnerVersionFutures = <Sdk, Future<GetMetadataResponse>>{};
+
+ /// Returns the router version and starts loading if it is not started yet.
+ ComponentVersion? get routerVersion {
Review Comment:
The single place, when this getter is used is `_ComponentVersionsWidget` Just use `FutureBuilder` there
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123065497
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
@alexeyinkin also, please run [`shellcheck`](https://www.shellcheck.net/) linter tool on this script
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123048490
##########
playground/frontend/playground_components/lib/src/build_metadata.g.dart:
##########
@@ -0,0 +1,9 @@
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+const buildCommitHash = '0d41f157812fdc3227a6ea93429d3096aa03a850' ;
Review Comment:
@alexeyinkin I don't like that we commit these values into the repo. Can we write something like `unknown` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123048490
##########
playground/frontend/playground_components/lib/src/build_metadata.g.dart:
##########
@@ -0,0 +1,9 @@
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+const buildCommitHash = '0d41f157812fdc3227a6ea93429d3096aa03a850' ;
Review Comment:
I don't like that we commit these values into the repo. Can we write something like `unknown` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1128980772
##########
playground/api/v1/api.proto:
##########
@@ -314,6 +314,17 @@ message GetSnippetResponse {
Complexity complexity = 4;
}
+// GetMetadataRequest represents request for runner metadata
+message GetMetadataRequest {}
+
+// GetMetadataResponse contains metadata about the runner
+message GetMetadataResponse {
+ string runner_sdk = 1;
+ optional string build_commit_hash = 2;
+ optional int64 build_commit_timestamp_seconds_since_epoch = 3;
+ optional string beam_sdk_version = 4;
Review Comment:
From the same link https://cloud.google.com/apis/design/design_patterns.md#optional_primitive_fields
> If not using optional would add complexity or ambiguity, then use optional primitive fields.
This is why I asked for `optional` here. This endpoint exists in both router and runners. The router is unaware of Beam and would have to return a magic value instead if omitting it. I think that omitting it on the protocol level is better.
There is indeed a bug in the current Dart implementation. It parses the raw data correctly, note the `null` in the router's response for Beam version:

But then the getter returns an empty string for it.
The issue to make those getters nullable is the most thumbed-up one in the repository, and members agree: https://github.com/google/protobuf.dart/issues/523
So I believe it will be done at some point. So if the support is almost there, I prefer to provision this right at the high level of the protocol to be able to improve the implementation soon after.
I found no other explanations of discouraging `optional` so I vote for using it.
There is another reason for going `optional`. On the frontend we have many enums coming from the backend: SDK, Complexity, etc. In protobuf they come as magic values like `SDK_UNDEFINED`. So we must maintain two versions of such enum classes: one with this value to be parsed from the protofub, and the other one is without it to be used in business logic. Converting between the two is complexity, so in the next phase of Playground development we would like to switch those to optional too. It would then be consistent with optional fields in this endpoint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1130866232
##########
playground/frontend/CONTRIBUTE.md:
##########
@@ -66,25 +66,6 @@ contains decoupled playground page state and module providers like `OutputPlacem
For quick start up, please take a look
to [this guide](https://docs.flutter.dev/development/data-and-backend/state-mgmt/simple)
-## Configuration
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1130951236
##########
learning/tour-of-beam/backend/README.md:
##########
@@ -39,6 +39,7 @@ To re-generate:
$ go generate -x ./...
```
+> Note: this requires [`protoc`](https://grpc.io/docs/protoc-installation/) version `>= 3.15` and [`moq`](https://github.com/matryer/moq) tools to be installed
Review Comment:
Created task #25775 to resolve this in a separate PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1121902345
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
+
+echo '// GENERATED CODE - DO NOT MODIFY BY HAND' > $FILE
Review Comment:
Use the following pattern to put large snippets of text into files:
```shell
cat > $FILE << EOF
....
EOF
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123187006
##########
playground/frontend/Dockerfile:
##########
@@ -23,8 +23,13 @@ ARG FLUTTER_VERSION=3.7.3
# Setting up.
# ------------------------------------------------
+COPY . /app
+WORKDIR /app
+
RUN apt-get update && apt-get install -y wget xz-utils git
+RUN cat /app/playground_components/lib/src/build_metadata.g.dart
Review Comment:
Deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123055623
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE=$(realpath "$(realpath $(dirname $0))/../lib/src/build_metadata.g.dart")
+
+cat > $FILE << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+EOF
+
+printf 'const buildCommitHash = ' >> $FILE
+git rev-parse --sq HEAD >> $FILE
+echo ';' >> $FILE
+
+printf 'const buildCommitSecondsSinceEpoch = ' >> $FILE
Review Comment:
```suggestion
printf 'const buildCommitSecondsSinceEpoch = %s;\n' $(git show -s --format=%ct HEAD) >> $FILE
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123527885
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,37 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Usage
+# ./generate_build_metadata.sh <build_commit_hash> <build_commit_seconds_since_epoch>
+
+FILE=$(realpath "$(dirname "$0")/../lib/src/build_metadata.g.dart")
+
+cat > "$FILE" << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// Keep these constants \`null\` in the repository.
+EOF
+
+echo "const buildCommitHash = '$1';" >> "$FILE"
Review Comment:
Wonder if moving this into the `cat > ... << EOF` block will work. If so, this will be a nice looking script
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1122035575
##########
playground/frontend/lib/config.g.dart:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// This file was auto-generated and used to contain backend URLs.
+// They are now determined automatically
+// as of https://github.com/apache/beam/pull/25610
+// TODO(alexeyinkin): When it is merged, find a better place for the remaining
+// constant and delete this file.
+
+const String kAnalyticsUA = 'G-BXFP2FNCKC';
Review Comment:
This is to prevent merging conflicts because there are other PRs to change this file. When analytics lands, I will.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123044361
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
+
+echo '// GENERATED CODE - DO NOT MODIFY BY HAND' > $FILE
+echo '' >> $FILE
+
+echo '// This file is generated during deployment to contain data about the commit.' >> $FILE
+echo '// The copy of this file stored in the repository is for development purpose' >> $FILE
+echo '// so the project can be started locally.' >> $FILE
+echo '// It is safe to commit changes here everytime you run code generation.' >> $FILE
+
+echo -n 'const buildCommitHash = ' >> $FILE
+git rev-parse --sq HEAD >> $FILE
+echo ';' >> $FILE
+
+echo -n 'const buildCommitSecondsSinceEpoch = ' >> $FILE
Review Comment:
`+`
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
+
+echo '// GENERATED CODE - DO NOT MODIFY BY HAND' > $FILE
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rshamunov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "rshamunov (via GitHub)" <gi...@apache.org>.
rshamunov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1129262405
##########
playground/backend/internal/environment/environment_service.go:
##########
@@ -35,6 +35,7 @@ const (
serverIpKey = "SERVER_IP"
serverPortKey = "SERVER_PORT"
beamSdkKey = "BEAM_SDK"
+ beamVersionKey = "BEAM_VERSION"
Review Comment:
Fix tabulation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123065497
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
@alexeyinkin also, please run `shellcheck` linter tool on this script
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1121897408
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
Review Comment:
@alexeyinkin Can you also add this file to `.gitignore` so if user would build the project it wouldn't pollute `git status`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25610: [Playground] GetMetadata() endpoint
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25610:
URL: https://github.com/apache/beam/pull/25610#issuecomment-1456287005
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] MakarkinSAkvelon commented on pull request #25610: [Playground] GetMetadata() endpoint
Posted by "MakarkinSAkvelon (via GitHub)" <gi...@apache.org>.
MakarkinSAkvelon commented on PR #25610:
URL: https://github.com/apache/beam/pull/25610#issuecomment-1455990737
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1130926343
##########
playground/api/v1/api.proto:
##########
@@ -314,6 +314,17 @@ message GetSnippetResponse {
Complexity complexity = 4;
}
+// GetMetadataRequest represents request for runner metadata
+message GetMetadataRequest {}
+
+// GetMetadataResponse contains metadata about the runner
+message GetMetadataResponse {
+ string runner_sdk = 1;
+ optional string build_commit_hash = 2;
+ optional int64 build_commit_timestamp_seconds_since_epoch = 3;
+ optional string beam_sdk_version = 4;
Review Comment:
Removed the `optional` fields
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123527885
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,37 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Usage
+# ./generate_build_metadata.sh <build_commit_hash> <build_commit_seconds_since_epoch>
+
+FILE=$(realpath "$(dirname "$0")/../lib/src/build_metadata.g.dart")
+
+cat > "$FILE" << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// Keep these constants \`null\` in the repository.
+EOF
+
+echo "const buildCommitHash = '$1';" >> "$FILE"
Review Comment:
@alexeyinkin Wonder if moving this into the `cat > ... << EOF` block will work. If so, this will be a nice looking script
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123175606
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE=$(realpath "$(realpath $(dirname $0))/../lib/src/build_metadata.g.dart")
+
+cat > $FILE << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+EOF
+
+printf 'const buildCommitHash = ' >> $FILE
Review Comment:
`+`
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE=$(realpath "$(realpath $(dirname $0))/../lib/src/build_metadata.g.dart")
+
+cat > $FILE << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// It is safe to commit changes here everytime you run code generation.
+EOF
+
+printf 'const buildCommitHash = ' >> $FILE
+git rev-parse --sq HEAD >> $FILE
+echo ';' >> $FILE
+
+printf 'const buildCommitSecondsSinceEpoch = ' >> $FILE
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm merged pull request #25610: [Playground] GetMetadata() endpoint
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #25610:
URL: https://github.com/apache/beam/pull/25610
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1115855752
##########
playground/backend/containers/git-functions.gradle:
##########
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+ext.getGitCommitHash = () -> {
+ def stdout = new ByteArrayOutputStream()
+ exec {
+ executable 'git'
Review Comment:
I think we should use Kotlin for new endeavors. Or at least simplify possible migration in the future with
```suggestion
executable("git")
```
##########
playground/backend/containers/java/build.gradle:
##########
@@ -71,7 +72,9 @@ docker {
tags containerImageTags()
buildArgs(['BEAM_VERSION': project.rootProject.hasProperty(["sdk-tag"]) ?
project.rootProject["sdk-tag"] :
- default_beam_version ])
+ default_beam_version,
+ 'GIT_COMMIT': getGitCommitHash(),
+ 'GIT_TIMESTAMP': getGitCommitTimestamp()] )
Review Comment:
Are there auto-formatters or at least guides? This looks random. Maybe at least add a trailing comma?
```suggestion
'GIT_TIMESTAMP': getGitCommitTimestamp(),
])
```
##########
playground/api/v1/api.proto:
##########
@@ -314,6 +314,17 @@ message GetSnippetResponse {
Complexity complexity = 4;
}
+// GetMetadataRequest represents request for runner metadata
+message GetMetadataRequest {}
+
+// GetMetadataResponse contains metadata about the runner
+message GetMetadataResponse {
+ string runner_sdk = 1;
+ string build_commit_hash = 2;
+ string build_commit_timestamp = 3;
Review Comment:
```suggestion
int64 build_commit_seconds_since_epoch = 3;
```
?
In Dart core these are called `millisecondsSinceEpoch`, `microsecondsSinceEpoch`, etc. to avoid confusion since people nowadays expect varying units of timestamp.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1121898421
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run this from the playground_components project root
+FILE='lib/src/build_metadata.g.dart'
+
+echo '// GENERATED CODE - DO NOT MODIFY BY HAND' > $FILE
+echo '' >> $FILE
+
+echo '// This file is generated during deployment to contain data about the commit.' >> $FILE
+echo '// The copy of this file stored in the repository is for development purpose' >> $FILE
+echo '// so the project can be started locally.' >> $FILE
+echo '// It is safe to commit changes here everytime you run code generation.' >> $FILE
+
+echo -n 'const buildCommitHash = ' >> $FILE
+git rev-parse --sq HEAD >> $FILE
+echo ';' >> $FILE
+
+echo -n 'const buildCommitSecondsSinceEpoch = ' >> $FILE
Review Comment:
use `printf` instead of `echo -n`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1121906306
##########
playground/frontend/lib/config.g.dart:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// This file was auto-generated and used to contain backend URLs.
+// They are now determined automatically
+// as of https://github.com/apache/beam/pull/25610
+// TODO(alexeyinkin): When it is merged, find a better place for the remaining
+// constant and delete this file.
+
+const String kAnalyticsUA = 'G-BXFP2FNCKC';
Review Comment:
Rename to `config.dart` or similar as it no longer is a generated file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123058913
##########
playground/frontend/Dockerfile:
##########
@@ -23,8 +23,13 @@ ARG FLUTTER_VERSION=3.7.3
# Setting up.
# ------------------------------------------------
+COPY . /app
+WORKDIR /app
+
RUN apt-get update && apt-get install -y wget xz-utils git
+RUN cat /app/playground_components/lib/src/build_metadata.g.dart
Review Comment:
Aren't we already printing it out in the script generating this file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123063932
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
@alexeyinkin add
```suggestion
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1123186334
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
`+`
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,39 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1126346349
##########
playground/frontend/playground_components/tools/generate_build_metadata.sh:
##########
@@ -0,0 +1,37 @@
+#!/bin/bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Usage
+# ./generate_build_metadata.sh <build_commit_hash> <build_commit_seconds_since_epoch>
+
+FILE=$(realpath "$(dirname "$0")/../lib/src/build_metadata.g.dart")
+
+cat > "$FILE" << EOF
+// GENERATED CODE - DO NOT MODIFY BY HAND
+//
+// This file is generated during deployment to contain data about the commit.
+// The copy of this file stored in the repository is for development purpose
+// so the project can be started locally.
+// Keep these constants \`null\` in the repository.
+EOF
+
+echo "const buildCommitHash = '$1';" >> "$FILE"
Review Comment:
`+`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on pull request #25610: [Playground] GetMetadata() endpoint
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on PR #25610:
URL: https://github.com/apache/beam/pull/25610#issuecomment-1456285402
R: @damondouglas
The only failing check is due to snippet renaming, it is addressed in
- https://github.com/apache/beam/pull/25033
Not merging it here to simplify separate PRs and reduce conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1130651154
##########
playground/backend/containers/java/Dockerfile:
##########
@@ -55,6 +57,7 @@ FROM apache/beam_java8_sdk:$BEAM_VERSION
ARG BEAM_VERSION
ARG SPRING_VERSION=5.3.25
ARG KAFKA_CLIENTS_VERSION=2.3.1
+ENV BEAM_VERSION=$BEAM_VERSION
Review Comment:
This is not a runtime configuration parameter. This variable holds the value of Beam SDK version with which the image was built, so it's a compile-time parameter. Storing this value outside of the container makes little sense to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25610: [Playground] GetMetadata() endpoint
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25610:
URL: https://github.com/apache/beam/pull/25610#discussion_r1130651154
##########
playground/backend/containers/java/Dockerfile:
##########
@@ -55,6 +57,7 @@ FROM apache/beam_java8_sdk:$BEAM_VERSION
ARG BEAM_VERSION
ARG SPRING_VERSION=5.3.25
ARG KAFKA_CLIENTS_VERSION=2.3.1
+ENV BEAM_VERSION=$BEAM_VERSION
Review Comment:
This is not a runtime configuration parameter in the strict sense. This variable holds the value of Beam SDK version with which the image was built. Storing this value outside of the container makes little sense to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org