You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "alexeyinkin (via GitHub)" <gi...@apache.org> on 2023/02/16 09:22:00 UTC
[GitHub] [beam] alexeyinkin opened a new pull request, #25507: Overview of adding snippets to Playground (#25506)
alexeyinkin opened a new pull request, #25507:
URL: https://github.com/apache/beam/pull/25507
- Resolves #25506
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on PR #25507:
URL: https://github.com/apache/beam/pull/25507#issuecomment-1561850667
R: @pabloem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1112742138
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
Review Comment:
`/sdks` contains the whole SDKs. Are its subdirectories really entirely parsed in search for examples? It's a bad coupling to me because an SDK should not be aware of Playground artifacts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on PR #25507:
URL: https://github.com/apache/beam/pull/25507#issuecomment-1502955857
Thinks that are left:
1. How to use external data and code with no internet
2. How to add ToB learning materials in addition to snippets
3. Real example of ToB snippet link
4. Real snippet with named sections
5. Verify metadata reference
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1207406089
##########
playground/frontend/playground_components/lib/src/constants/links.dart:
##########
@@ -24,6 +24,7 @@ class BeamLinks {
// GitHub
static const github = 'https://github.com/apache/beam';
+ static const newExample = 'https://github.com/akvelon/beam/blob/master/playground/load_your_code.md';
Review Comment:
is this link correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1111648440
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
Review Comment:
and `/learning/tour-of-beam/learning-content` for Tour of Beam
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1112932708
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
Review Comment:
1. Every example is also a snippet, so example's code can be obtained both via example-related endpoints and snippet ones. But snippets are not examples and can be only fetched using `GetSnippet()`. Answering your question directly - yes, you should be able to fetch `SDK_JAVA_MinimalWordCount` as a snippet
2. `"SDK_XXX" + "_" + <example_name>` for Playground examples, `"TB_EXAMPLES" +"_" + "SDK_XXX" + "_" + <example_name>` for Tour of Beam examples
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rshamunov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "rshamunov (via GitHub)" <gi...@apache.org>.
rshamunov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1169870976
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,594 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
+- `/examples`
+- `/learning/katas`
+- `/sdks`.
+
+Adding Scala example snippets automatically is not supported,
+and Scala example snippets can be added to the catalog manually.
+
+#### 2. Add metadata to describe the example
+
+Playground relies on metadata comments block to identify and place an example into the database,
+and present in the Examples Catalog.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+Playground automatically removes metadata comments block before storing the example in database
+so the metadata is not visible to end users. The block is in the format of a YAML map:
+
+```yaml
+beam-playground:
+ # Name of the Beam example that will be displayed in the Playground Examples Catalog. Required.
+ name: ""
+ # Description of the Beam example that will be displayed in the Playground Examples Catalog. Required.
+ description: ""
+ # Contains information about pipeline options of the Beam example/test/kata. Optional.
+ pipeline_options: "--name1 value1 --name2 value2"
+ # The line number to scroll to when the snippet is loaded.
+ # Note that lines of the metadata block are cut so line numbers after it are shifted. Required.
+ context_line: 1
+ # Categories this example is included into. Non-existent categories will be created.
+ # Optional, defaults to no categories making the example unlisted.
+ categories:
+ - "Combiners"
+ - "Core Transforms"
+ # Tags by which this snippet can be found in the Example Catalog. Optional.
+ tags:
+ - "numbers"
+ - "count"
+ # Helps user to identify example's complexity. Values: BASIC|MEDIUM|ADVANCED. Required.
+ complexity: BASIC
+ # Specifies the example to be loaded as default when its SDK selected in the Playground.
+ # See section "Default examples" below. Optional, defaults to false.
+ default_example: true
+ # If the snippet has a Colab notebook, can link the URL of the Colab notebook that is based on this snippet.
+ url_notebook: "https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/documentation/transforms/python/elementwise/filter-py.ipynb"
+ # Specifies if the given example consists of multiple files or not. Optional, defaults to false.
+ multifile: true
+ # Specifies example caching by Playground. Optional, defaults to false (the output is cached).
+ always_run: true
+ # Datasets which will be used by emulators. Optional.
+ # Please see section "Kafka emulator" for more information.
+ datasets:
+ # Dataset name
+ CountWords:
+ # Dataset location. Only "local" is supported. Required.
+ location: local
+ # Dataset format. Supported values are "avro" and "json". Required.
+ format: avro
+ # List of emulators to start during pipeline execution. Currently only `kafka` type is supported. Optional.
+ emulators:
+ - type: kafka
+ topic:
+ # Dataset id. Will be used as a topic name.
+ id: dataset
+ # Name of dataset specified in "datasets" section.
+ source_dataset: "CountWords"
+```
+
+For metadata reference see fields in "Tag" class [here](infrastructure/models.py).
+
+##### Default examples
+
+Each SDK must have a single default example.
+If there is none, the user will see the error in the app and face a blank editor.
+If there are more than one, it is not defined which one will be selected.
+
+##### Kafka emulator
+
+Examples which require Kafka server emulator, need to include the `emulators` tag
+and provide `dataset` in the example's tag. You can refer to an example
+[here](/examples/java/src/main/java/org/apache/beam/examples/KafkaWordCountJson.java).
+
+1. Add your dataset in either JSON or Avro format into the `playground/backend/datasets` path.
+
+2. Add the following elements to the example's metadata tag:
+ ```YAML
+ emulators:
+ - type: kafka
+ topic:
+ id: dataset
+ source_dataset: <dataset_name>
+ datasets:
+ <dataset_name>:
+ location: local
+ format: json # or 'avro'
+ ```
+ replace `<dataset_name>` with the name of your dataset file without the file name extension.
+
+3. Use the exact string `"kafka_server:9092"` as a server name in your code snippet.
+ This string will be replaced by the actual host name and port automatically
+ before the compilation step by Playground.
+
+>**Kafka emulator limitations:**
+> - Playground Kafka emulator currently supports only Beam Java SDK
+> - exact string `"kafka_server:9092"` should be present in the code snippet;
+ any other variation like `"kafa_server" + ":9092"` will not work
+
+#### 3. Make a PR
+
+Make a PR with code snippet into [the Apache Beam repository](https://github.com/apache/beam)
+following the [Contribution guide](https://beam.apache.org/contribute/).
+Verify that all pre-commit tests are passing.
+
+Playground CI will verify and deploy the example to Playground Example Catalog when the PR is merged.
+
+#### 4. Save the snippet ID
+
+The snippet will be assigned an ID.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, in this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA_MinimalWordCount&sdk=java
+```
+
+the ID is: `SDK_JAVA_MinimalWordCount`.
+
+You will need the snippet ID to embed the Playground with snippet into a website page.
+
+### Source 2. How to add an unlisted example to Playground
+
+Not all examples must be visible in the example dropdown.
+Some examples are best in the context of Apache Beam documentation.
+To embed them into the documentation use unlisted examples.
+They work and are checked and cached the same way as public examples.
+
+Proceed the same way as with [Source 1. Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog) except:
+1. Use the directory `/learning/beamdoc`
Review Comment:
Replace "use" to something more meaningful, e.g. add an example file to the directory ./beam/learning/beamdoc/<sdk>.
It'd be better to have a dedicated subfolder for each SDK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1112857911
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
Review Comment:
[Yes, they are](https://github.com/apache/beam/blob/921bc7b94708391eb7d980cb8d9ed35c80d3aebb/.github/workflows/playground_examples_ci.yml#L45). I'm not sure why it's made that way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1162981012
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,562 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
+- `/examples`
+- `/learning/katas`
+- `/sdks`.
+
+Adding Scala example snippets automatically is not supported,
+and Scala example snippets can be added to the catalog manually.
+
+#### 2. Add metadata to describe the example
+
+Playground relies on metadata comments block to identify and place an example into the database,
+and present in the Examples Catalog.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+Playground automatically removes metadata comments block before storing the example in database
+so the metadata is not visible to end users. The block is in the format of a YAML map.
+
+Example tag metadata quick reference (all elements are **required**, unless marked **optional**):
+
+| Field | Description |
Review Comment:
Overall I think this will be understandable if it were reformulated as an example of tag with specification in comments. I'll make one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1176299902
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,594 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
Review Comment:
Then what is the new mechanism that picks up the example files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1112846542
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
+```
+
+You will need it to embed the Playground code.
+
+### Source 2. Playground Unlisted Database
+
+If your snippet is less useful to public for learning but still beneficial to some group
+of people, you can put it into the same database as the Playground Visible Catalog
+but make it unlisted.
+
+Advantages:
+
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+Proceed the same way as with Playground Visible Catalog, but do not use these attributes:
+
+- `categories`
+- `default_example`
+- `tags`
+
+**TODO: How to get the path of the example?**
Review Comment:
How to get the ID of a snippet that does not show in the catalog after merging its PR? @TSultanov
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
Review Comment:
Is this complete and true? @TSultanov
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
Review Comment:
1. Are these IDs of the same namespace as those of user-shared snippets? If so, can we call pass any of them to any endpoints like `getSnippet()` for `SDK_JAVA_MinimalWordCount`?
2. Since the ID is user-visible, how is it composed? Is it `"SDK_XXX" + "_" + <name_attribute_from_metadata>`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25507:
URL: https://github.com/apache/beam/pull/25507#issuecomment-1561752708
Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @jrmccluskey added as fallback since no labels match configuration
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25507:
URL: https://github.com/apache/beam/pull/25507#issuecomment-1561851972
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1207407483
##########
website/www/site/content/en/get-started/try-beam-playground.md:
##########
@@ -32,78 +32,7 @@ You can try the available Apache Beam examples at
{{< playground_snippet language="scio" path="SDK_SCIO_MinimalWordCount" >}}
{{< /playground >}}
-## How To Add New Examples
-
-To add an Apache Beam example/test/kata into Beam Playground catalog,
-add the `beam-playground` tag into the file to be added.
-`beam-playground` tag is a yaml format comment:
-
-{{< highlight java >}}
-// beam-playground:
-// name: Name of the example/test/kata
-// description: Description of the example/test/kata
-// multifile: false
-// pipeline_options: --option1 value1 --option2 value2
-// default_example: false
-// context_line: 10
-// categories:
-// - category 1
-// - category 2
-// - category N
-
-// example code
-{{< /highlight >}}
-{{< highlight py >}}
-# beam-playground:
-# name: Name of the example/test/kata
-# description: Description of the example/test/kata
-# multifile: false
-# pipeline_options: --option1 value1 --option2 value2
-# default_example: false
-# context_line: 10
-# categories:
-# - category 1
-# - category 2
-# - category N
-
-# example code
-{{< /highlight >}}
-{{< highlight go >}}
-// beam-playground:
-// name: Name of the example/test/kata
-// description: Description of the example/test/kata
-// multifile: false
-// pipeline_options: --option1 value1 --option2 value2
-// default_example: false
-// context_line: 10
-// categories:
-// - category 1
-// - category 2
-// - category N
-
-// example code
-{{< /highlight >}}
-
-The 'beam-playground' tag consists of the following **required** elements:
-
-- `beam-playground` - tag title.
-- `name` - string field. Name of the Beam example/test/kata that will be displayed in the Beam Playground
-examples catalog.
-- `description` - string field. Description of the Beam example/test/kata that will be displayed in Beam Playground.
-- `multifile` - boolean field. Specifies if the given example consists of multiple files or not.
-- `pipeline_options` - string field (optional). Contains information about pipeline options of the Beam example/test/kata.
-- `default_example` - boolean field (optional). Specifies if the given example is default or not. If some example is tagged
- as default it means that this example is shown when its SDK is chosen in Beam Playground.
- Only one example can be set as a default for each SDK.
-- `context_line` - integer field. The line where the main part of the Beam example/test/kata begins.
-- `categories` - list type field.
-Lists categories this example is included into. Available categories are listed in
-[playground/categories.yaml](https://github.com/apache/beam/blob/master/playground/categories.yaml).
-If some Beam example/kata/test needs to add a new category, then please submit PR with the changes to `categories.yaml`.
-After the category has been added, it can be used in the examples.
-
-More details on examples in Apache Beam Playground can be found
-[here](https://docs.google.com/document/d/1LBeGVTYwJHYbtmLt06OjhBR1CJ1Wgz18MEZjvNkuofc/edit?usp=sharing).
+See [here](https://github.com/akvelon/beam/blob/master/playground/load_your_code.md) for adding new examples.
Review Comment:
should we fix the link please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1112963669
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
+```
+
+You will need it to embed the Playground code.
+
+### Source 2. Playground Unlisted Database
+
+If your snippet is less useful to public for learning but still beneficial to some group
+of people, you can put it into the same database as the Playground Visible Catalog
+but make it unlisted.
+
+Advantages:
+
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+Proceed the same way as with Playground Visible Catalog, but do not use these attributes:
+
+- `categories`
+- `default_example`
+- `tags`
+
+**TODO: How to get the path of the example?**
Review Comment:
It will be available as `SDK_XXXX_<snippet_name>` if somebody will load it into production datastore. Perhaps @MakarkinSAkvelon or @rshamunov can clarify this part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1163007763
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,562 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
+- `/examples`
+- `/learning/katas`
+- `/sdks`.
+
+Adding Scala example snippets automatically is not supported,
+and Scala example snippets can be added to the catalog manually.
+
+#### 2. Add metadata to describe the example
+
+Playground relies on metadata comments block to identify and place an example into the database,
+and present in the Examples Catalog.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+Playground automatically removes metadata comments block before storing the example in database
+so the metadata is not visible to end users. The block is in the format of a YAML map.
+
+Example tag metadata quick reference (all elements are **required**, unless marked **optional**):
+
+| Field | Description |
Review Comment:
Like this:
```yaml
beam-playground:
# Name of the Beam example that will be displayed in the Playground Examples Catalog. Required
name: ""
# Description of the Beam example that will be displayed in the Playground Examples Catalog. Required
description: ""
# Contains information about pipeline options of the Beam example/test/kata. Optional
pipeline_options: ""
# The line number to scroll to when the snippet is loaded. Note that lines of the metadata block are cut so line numbers after it are shifted. Required
context_line: 1
# Lists categories this example is included into. Non-existent categories will be created. Optional
categories:
- ""
# Tags by which this snippet can be found in the Example Catalog. Optional
tags:
- ""
# Helps user to identify example's complexity. Required
complexity: BASIC|MEDIUM|ADVANCED
# Specifies the example to be loaded as default when its SDK selected in the Playground. Only one example can be set as the default for each SDK. Optional
default_example: true|false
# If the snippet has a Colab notebook, can link the URL of the Colab notebook that is based on this snippet.
url_notebook: ""
# Specifies if the given example consists of multiple files or not. False by default. Optional
multifile: true|false
# Specifies example caching by Playground. Default: `false` (the output is cached). Optional
always_run: true|false
# Datasets which will be used by emulators. Optional
# Please see section "Kafka emulator" for more information
datasets:
# Dataset name
CountWords:
# Dataset location. Only "local" is supported
location: local
# Dataset format. Supported values are "avro" and "json"
format: avro
# List of emulators to start during pipeline execution. Currently only `kafka` type is supported. Optional
emulators:
- type: kafka
topic:
# Dataset id. Will be used as a topic name
id: dataset
# Name of dataset soecified in "datasets" section
source_dataset: ""
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1162977014
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,562 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
+- `/examples`
+- `/learning/katas`
+- `/sdks`.
+
+Adding Scala example snippets automatically is not supported,
+and Scala example snippets can be added to the catalog manually.
+
+#### 2. Add metadata to describe the example
+
+Playground relies on metadata comments block to identify and place an example into the database,
+and present in the Examples Catalog.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+Playground automatically removes metadata comments block before storing the example in database
+so the metadata is not visible to end users. The block is in the format of a YAML map.
+
+Example tag metadata quick reference (all elements are **required**, unless marked **optional**):
+
+| Field | Description |
+|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `beam-playground:` | tag title |
+| `name` | string. Name of the Beam example that will be displayed in the Playground Examples Catalog. |
+| `description` | string. Description of the Beam example that will be displayed in the Playground Examples Catalog. |
+| `pipeline_options` | string (optional). Contains information about pipeline options of the Beam example/test/kata. |
+| `context_line` | integer. The line number to scroll to when the snippet is loaded. Note that lines of the metadata block are cut so line numbers after it are shifted. |
+| `categories` | list of strings. Lists categories this example is included into. Non-existent categories will be created. |
+| `tags` | list of strings (optional). Tags by which this snippet can be found in the Example Catalog. |
+| `complexity` | BASIC/MEDIUM/ADVANCED enum. Helps user to identify example's complexity. |
+| `default_example` | boolean (optional). Specifies the example to be loaded as default when its SDK selected in the Playground. Only one example can be set as the default for each SDK. |
+| `url_notebook` | string (optional). If the snippet has a Colab notebook, can link the URL of the Colab notebook that is based on this snippet. |
+| `multifile` | boolean (optional). Specifies if the given example consists of multiple files or not. Default: `false`. |
+| `always_run` | boolean (optional). Specifies example caching by Playground. Default: `false` (the output is cached). |
+| `datasets` | enum (optional). Datasets which will be used by emulators. Please see `Dataset` in Tag class model [here](infrastructure/models.py) for reference. |
Review Comment:
I think we should refer to the "Kafka emulator" section here for clarification
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on PR #25507:
URL: https://github.com/apache/beam/pull/25507#issuecomment-1577179522
Run Website_Stage_GCS PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1112963669
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
+```
+
+You will need it to embed the Playground code.
+
+### Source 2. Playground Unlisted Database
+
+If your snippet is less useful to public for learning but still beneficial to some group
+of people, you can put it into the same database as the Playground Visible Catalog
+but make it unlisted.
+
+Advantages:
+
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+Proceed the same way as with Playground Visible Catalog, but do not use these attributes:
+
+- `categories`
+- `default_example`
+- `tags`
+
+**TODO: How to get the path of the example?**
Review Comment:
It should be available as `SDK_XXXX_<snippet_name>`
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
+```
+
+You will need it to embed the Playground code.
+
+### Source 2. Playground Unlisted Database
+
+If your snippet is less useful to public for learning but still beneficial to some group
+of people, you can put it into the same database as the Playground Visible Catalog
+but make it unlisted.
+
+Advantages:
+
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+Proceed the same way as with Playground Visible Catalog, but do not use these attributes:
+
+- `categories`
+- `default_example`
+- `tags`
+
+**TODO: How to get the path of the example?**
Review Comment:
It will be available as `SDK_XXXX_<snippet_name>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1111655814
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
+```
+
+You will need it to embed the Playground code.
+
+### Source 2. Playground Unlisted Database
+
+If your snippet is less useful to public for learning but still beneficial to some group
+of people, you can put it into the same database as the Playground Visible Catalog
+but make it unlisted.
+
+Advantages:
+
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+Proceed the same way as with Playground Visible Catalog, but do not use these attributes:
+
+- `categories`
+- `default_example`
+- `tags`
+
+**TODO: How to get the path of the example?**
+
+### Source 3. Tour of Beam unit
+
+If your snippet fills a gap in the Tour of Beam tutorial, you can add it there
+as a learning unit. This also requires a textual learning material that your snippet will accompany.
+
+Advantages:
+
+- Anyone can find your snippet in the Tour of Beam tutorial.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+**TODO: HOWTO**
+
+### Source 4. User-shared Code
+
+Code can be uploaded directly to the database with "Share my code" button in Playground:
+
+
+
+Advantages:
+
+- No approval required.
+- Fast.
+
+Drawbacks:
+
+- The code may be deleted if it is not loaded by anyone during a 6 months rolling window.
+- No validation.
+- No cached output and graph.
+- A snippet is immutable. If you edit the code and re-share, you get a new link.
+- You cannot delete the code uploaded by accident.
+
+After clicking the button you will get a shareable link or embeddable HTML code.
+Note the `shared` parameter in the link query string.
+It contains the ID of your snippet that you can later use with other sharing methods.
+
+### Source 5. HTTPS
+
+You can upload a snippet file to any HTTPS-server you have access to.
+Then you refer to it by a URL to load into Playground.
+
+Advantages:
+
+- No approval required.
+- Fast.
+- You can change the snippet without changing a link.
+
+Drawbacks:
+
+- No validation.
+- No cached output and graph.
+
+For Playground to be able to load the snippet over HTTPS,
+your server needs to allow the access by sending the following header:
+
+```
+Access-Control-Allow-Origin: *
+```
+
+at least when requested with `*.beam.apache.org` as
+[`referer`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer).
+
+To understand the reasons, read about
+[CORS (Cross-Origin Resource Sharing)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS).
+
+Many people prefer to host their snippets in their GitHub repositories.
+GitHub is known to allow cross-origin access on direct links to raw file content.
+
+## Step 3. Create a link or embed
+
+Once you uploaded or otherwise prepared your code, you can get it shown in the Playground.
+Choose any of the following ways.
+
+### Direct Link to the Standalone Playground Web App
+
+You can link to the Playground app so that it opens your code. The link depends on
+where you placed the code.
+
+#### 1. Link for a snippet from Playground Visible Catalog
+
+1. Open your snippet in the dropdown menu.
+2. Without changing it, click "Share my code".
+3. Copy the link.
+
+The link contains the `path` to your snippet in the database. It is in the following format:
+```
+https://play.beam.apache.org/?path=SDK_JAVA/PRECOMPILED_OBJECT_TYPE_KATA/AggregationMax&sdk=java
+```
+
+A special case is the default snippet for an SDK. It can be loaded by the following link:
+
+```
+`https://play.beam.apache.org/?sdk=python&default=true`
+```
+
+This way if another snippet is ever made default, the links you shared will lead
+to the new snippet.
+
+#### 2. Link for a snippet from Playground Unlisted Database
+
+The code can be accessed with the same link as with Playground Visible Catalog.
+Since the snippet is unlisted, you cannot select it in the dropdown and so you should
+manually edit the link. Use the example above and replace the `path` and `sdk` with yours.
+
+#### 3. Link for a Tour of Beam Unit
+
+**TODO**
+
+#### 4. Link for User-shared Code
+
+You get the link when you click "Share my code" button. It is in the following format:
+
+```
+https://play.beam.apache.org/?sdk=java&shared=SNIPPET_ID
+```
+
+#### 5. Link for an HTTPS-served snippet
+
+Add the URL to the `url` parameter, for example:
+
+```
+https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/golang/go/master/src/fmt/format.go
+```
+
+#### 6. Link to an empty editor
+
+You can link to an empty editor to make uses start their snippets from scratch:
+
+```
+https://play.beam.apache.org/?sdk=go&empty=true
+```
+
+#### Passing View Options
+
+If your code contains named sections as described in the beginning of this page,
+you can apply view options to those sections. Otherwise skip this.
+
+##### Read-only sections
+
+Add `readonly` parameter with comma-separated section names:
+
+`https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go&readonly=iam_get_role`
+
+##### Folding everything except sections
+
+Add `unfold` parameter with comma-separated section names:
+
+`https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go&unfold=iam_get_role`
+
+This folds all foldable blocks that do not overlap with
+any of the given sections.
+
+##### Hiding everything except a section
+
+Add `show` parameter with a single section name:
+
+`https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go&show=iam_get_role`
+
+It is still the whole snippet that is sent for execution although only the given section
+is visible.
+
+This also makes the editor read-only so the user cannot add code that conflicts
+with the hidden text.
+
+#### Linking to multiple examples
+
+The above URLs load a snippet that you want. But what happens if the user switches SDK?
+Normally this will be shown:
+
+- The catalog default example for the new SDK in the Standalone Playground.
+- The empty editor for the new SDK in the Embedded Playground.
+
+This can be changed by linking to multiple examples, up to one per SDK.
+
+For this purpose, make a JSON array with any combination of parameters that
+are allowed for loading single examples, for instance:
+
+```json
+[
+ {
+ "sdk": "java",
+ "path": "SDK_JAVA/PRECOMPILED_OBJECT_TYPE_KATA/AggregationMax"
Review Comment:
`SDK_JAVA_AggregationMax`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1162979161
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,562 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
+- `/examples`
+- `/learning/katas`
+- `/sdks`.
+
+Adding Scala example snippets automatically is not supported,
+and Scala example snippets can be added to the catalog manually.
+
+#### 2. Add metadata to describe the example
+
+Playground relies on metadata comments block to identify and place an example into the database,
+and present in the Examples Catalog.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+Playground automatically removes metadata comments block before storing the example in database
+so the metadata is not visible to end users. The block is in the format of a YAML map.
+
+Example tag metadata quick reference (all elements are **required**, unless marked **optional**):
+
+| Field | Description |
+|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `beam-playground:` | tag title |
+| `name` | string. Name of the Beam example that will be displayed in the Playground Examples Catalog. |
+| `description` | string. Description of the Beam example that will be displayed in the Playground Examples Catalog. |
+| `pipeline_options` | string (optional). Contains information about pipeline options of the Beam example/test/kata. |
+| `context_line` | integer. The line number to scroll to when the snippet is loaded. Note that lines of the metadata block are cut so line numbers after it are shifted. |
+| `categories` | list of strings. Lists categories this example is included into. Non-existent categories will be created. |
+| `tags` | list of strings (optional). Tags by which this snippet can be found in the Example Catalog. |
+| `complexity` | BASIC/MEDIUM/ADVANCED enum. Helps user to identify example's complexity. |
+| `default_example` | boolean (optional). Specifies the example to be loaded as default when its SDK selected in the Playground. Only one example can be set as the default for each SDK. |
+| `url_notebook` | string (optional). If the snippet has a Colab notebook, can link the URL of the Colab notebook that is based on this snippet. |
+| `multifile` | boolean (optional). Specifies if the given example consists of multiple files or not. Default: `false`. |
+| `always_run` | boolean (optional). Specifies example caching by Playground. Default: `false` (the output is cached). |
+| `datasets` | enum (optional). Datasets which will be used by emulators. Please see `Dataset` in Tag class model [here](infrastructure/models.py) for reference. |
+| `emulators` | list (optional). List of emulators to start during pipeline execution. Currently only `kafka` type is supported. Please see `Emulator` in Tag class model [here](infrastructure/models.py) for reference. |
Review Comment:
Same. This really requires an example to be understandable. I think `See [Kafka emulator](#kafka-emulator) section for clarification` should work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1111641317
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
Review Comment:
`/learning/katas`, `/examples` and `/sdks` for all three SDKs,files with examples are chosen based on filename extension
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
Review Comment:
We use IDs now instead of paths, like `SDK_JAVA_MinimalWordCount`
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
Review Comment:
`SDK_JAVA_MinimalWordCount`
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
Review Comment:
and `/learning/tour-of-beam/learning-content" for Tour of Beam
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
Review Comment:
`tour_of_beam_examples_ci.yml` for Tour of Beam, `playground_examples_ci.yml` for Playground
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1111656167
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
+```
+
+You will need it to embed the Playground code.
+
+### Source 2. Playground Unlisted Database
+
+If your snippet is less useful to public for learning but still beneficial to some group
+of people, you can put it into the same database as the Playground Visible Catalog
+but make it unlisted.
+
+Advantages:
+
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+Proceed the same way as with Playground Visible Catalog, but do not use these attributes:
+
+- `categories`
+- `default_example`
+- `tags`
+
+**TODO: How to get the path of the example?**
+
+### Source 3. Tour of Beam unit
+
+If your snippet fills a gap in the Tour of Beam tutorial, you can add it there
+as a learning unit. This also requires a textual learning material that your snippet will accompany.
+
+Advantages:
+
+- Anyone can find your snippet in the Tour of Beam tutorial.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+**TODO: HOWTO**
+
+### Source 4. User-shared Code
+
+Code can be uploaded directly to the database with "Share my code" button in Playground:
+
+
+
+Advantages:
+
+- No approval required.
+- Fast.
+
+Drawbacks:
+
+- The code may be deleted if it is not loaded by anyone during a 6 months rolling window.
+- No validation.
+- No cached output and graph.
+- A snippet is immutable. If you edit the code and re-share, you get a new link.
+- You cannot delete the code uploaded by accident.
+
+After clicking the button you will get a shareable link or embeddable HTML code.
+Note the `shared` parameter in the link query string.
+It contains the ID of your snippet that you can later use with other sharing methods.
+
+### Source 5. HTTPS
+
+You can upload a snippet file to any HTTPS-server you have access to.
+Then you refer to it by a URL to load into Playground.
+
+Advantages:
+
+- No approval required.
+- Fast.
+- You can change the snippet without changing a link.
+
+Drawbacks:
+
+- No validation.
+- No cached output and graph.
+
+For Playground to be able to load the snippet over HTTPS,
+your server needs to allow the access by sending the following header:
+
+```
+Access-Control-Allow-Origin: *
+```
+
+at least when requested with `*.beam.apache.org` as
+[`referer`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer).
+
+To understand the reasons, read about
+[CORS (Cross-Origin Resource Sharing)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS).
+
+Many people prefer to host their snippets in their GitHub repositories.
+GitHub is known to allow cross-origin access on direct links to raw file content.
+
+## Step 3. Create a link or embed
+
+Once you uploaded or otherwise prepared your code, you can get it shown in the Playground.
+Choose any of the following ways.
+
+### Direct Link to the Standalone Playground Web App
+
+You can link to the Playground app so that it opens your code. The link depends on
+where you placed the code.
+
+#### 1. Link for a snippet from Playground Visible Catalog
+
+1. Open your snippet in the dropdown menu.
+2. Without changing it, click "Share my code".
+3. Copy the link.
+
+The link contains the `path` to your snippet in the database. It is in the following format:
+```
+https://play.beam.apache.org/?path=SDK_JAVA/PRECOMPILED_OBJECT_TYPE_KATA/AggregationMax&sdk=java
+```
+
+A special case is the default snippet for an SDK. It can be loaded by the following link:
+
+```
+`https://play.beam.apache.org/?sdk=python&default=true`
+```
+
+This way if another snippet is ever made default, the links you shared will lead
+to the new snippet.
+
+#### 2. Link for a snippet from Playground Unlisted Database
+
+The code can be accessed with the same link as with Playground Visible Catalog.
+Since the snippet is unlisted, you cannot select it in the dropdown and so you should
+manually edit the link. Use the example above and replace the `path` and `sdk` with yours.
+
+#### 3. Link for a Tour of Beam Unit
+
+**TODO**
+
+#### 4. Link for User-shared Code
+
+You get the link when you click "Share my code" button. It is in the following format:
+
+```
+https://play.beam.apache.org/?sdk=java&shared=SNIPPET_ID
+```
+
+#### 5. Link for an HTTPS-served snippet
+
+Add the URL to the `url` parameter, for example:
+
+```
+https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/golang/go/master/src/fmt/format.go
+```
+
+#### 6. Link to an empty editor
+
+You can link to an empty editor to make uses start their snippets from scratch:
+
+```
+https://play.beam.apache.org/?sdk=go&empty=true
+```
+
+#### Passing View Options
+
+If your code contains named sections as described in the beginning of this page,
+you can apply view options to those sections. Otherwise skip this.
+
+##### Read-only sections
+
+Add `readonly` parameter with comma-separated section names:
+
+`https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go&readonly=iam_get_role`
+
+##### Folding everything except sections
+
+Add `unfold` parameter with comma-separated section names:
+
+`https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go&unfold=iam_get_role`
+
+This folds all foldable blocks that do not overlap with
+any of the given sections.
+
+##### Hiding everything except a section
+
+Add `show` parameter with a single section name:
+
+`https://play.beam.apache.org/?sdk=go&url=https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go&show=iam_get_role`
+
+It is still the whole snippet that is sent for execution although only the given section
+is visible.
+
+This also makes the editor read-only so the user cannot add code that conflicts
+with the hidden text.
+
+#### Linking to multiple examples
+
+The above URLs load a snippet that you want. But what happens if the user switches SDK?
+Normally this will be shown:
+
+- The catalog default example for the new SDK in the Standalone Playground.
+- The empty editor for the new SDK in the Embedded Playground.
+
+This can be changed by linking to multiple examples, up to one per SDK.
+
+For this purpose, make a JSON array with any combination of parameters that
+are allowed for loading single examples, for instance:
+
+```json
+[
+ {
+ "sdk": "java",
+ "path": "SDK_JAVA/PRECOMPILED_OBJECT_TYPE_KATA/AggregationMax"
+ },
+ {
+ "sdk": "go",
+ "url": "https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go",
+ "readonly": "iam_get_role"
+ }
+]
+```
+
+Then pass it in`examples` query parameter like this:
+
+`https://play.beam.apache.org/?sdk=go&examples=[{"sdk":"java","path":"SDK_JAVA/PRECOMPILED_OBJECT_TYPE_KATA/AggregationMax"},{"sdk":"go","url":"https://raw.githubusercontent.com/GoogleCloudPlatform/golang-samples/main/iam/snippets/roles_get.go","readonly":"iam_get_role"}]`
Review Comment:
`SDK_JAVA_AggregationMax`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1112961391
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
Review Comment:
Full list can be seen [here](https://github.com/apache/beam/blob/921bc7b94708391eb7d980cb8d9ed35c80d3aebb/playground/infrastructure/helper.py#L69) and [here](https://github.com/apache/beam/blob/921bc7b94708391eb7d980cb8d9ed35c80d3aebb/playground/infrastructure/models.py#L70).
So, missing attributes are:
- `pipeline_options` - options which will be passed to the pipeline executor, e.g. `--output output.txt`
- `emulators` - list of emulators to start during pipeline execution. Currently only `kafka` type is supported. Example:
```yaml
emulators:
- type: kafka
topic:
id: dataset
source_dataset: CountWords
```
- `datasets` - specify datasets which will be used by emulators, e.g.:
```yaml
datasets:
CountWords:
location: local
format: json
```
here name of dataset should be the name of the file with the data, possible values of `format` are `avro` or `json` and `location` can be either `local` or `GCS` (only `local` is supported by playground backend).
- `multifile` - indicates whether given example has additional files associated with it
- `files` - list of files to include together with the example if `multifile` is `true`
- `url_notebook` - ???? can't find usage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rshamunov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "rshamunov (via GitHub)" <gi...@apache.org>.
rshamunov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1169854206
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,594 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
Review Comment:
This yml and the workflow will be removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TSultanov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "TSultanov (via GitHub)" <gi...@apache.org>.
TSultanov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1111654996
##########
playground/get_your_code.md:
##########
@@ -0,0 +1,449 @@
+# How to Get Your Code into Apache Beam Playground
+
+Getting your code into the Playground is a three-step process:
+
+1. Prepare your code.
+2. Place your code somewhere the Playground can load it from.
+3. Create a link to view it or a code to embed.
+
+For the last two steps there are multiple options -- pick any source and use any embedding:
+
+
+
+They can be combined, e.g.
+"put the code to the Playground catalog, embed the Playground to any HTML", or
+"upload the code to any website, embed the Playground to Apache Beam docs with Markdown".
+
+# Table of Contents
+- [Step 1. Prepare Your Code](#step-1-prepare-your-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Place your code somewhere the Playground can load it from](#step-2-place-your-code-somewhere-the-playground-can-load-it-from)
+ * [Source 1. Playground Visible Catalog](#source-1-playground-visible-catalog)
+ + [1. Put the file to the directory](#1-put-the-file-to-the-directory)
+ + [2. Add metadata](#2-add-metadata)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet path](#4-save-the-snippet-path)
+ * [Source 2. Playground Unlisted Database](#source-2-playground-unlisted-database)
+ * [Source 3. Tour of Beam unit](#source-3-tour-of-beam-unit)
+ * [Source 4. User-shared Code](#source-4-user-shared-code)
+ * [Source 5. HTTPS](#source-5-https)
+- [Step 3. Create a link or embed](#step-3-create-a-link-or-embed)
+ * [Direct Link to the Standalone Playground Web App](#direct-link-to-the-standalone-playground-web-app)
+ + [1. Link for a snippet from Playground Visible Catalog](#1-link-for-a-snippet-from-playground-visible-catalog)
+ + [2. Link for a snippet from Playground Unlisted Database](#2-link-for-a-snippet-from-playground-unlisted-database)
+ + [3. Link for a Tour of Beam Unit](#3-link-for-a-tour-of-beam-unit)
+ + [4. Link for User-shared Code](#4-link-for-user-shared-code)
+ + [5. Link for an HTTPS-served snippet](#5-link-for-an-https-served-snippet)
+ + [6. Link to an empty editor](#6-link-to-an-empty-editor)
+ + [Passing View Options](#passing-view-options)
+ - [Read-only sections](#read-only-sections)
+ - [Folding everything except sections](#folding-everything-except-sections)
+ - [Hiding everything except a section](#hiding-everything-except-a-section)
+ + [Linking to multiple examples](#linking-to-multiple-examples)
+ + [Embedded vs Standalone Playground URLs](#embedded-vs-standalone-playground-urls)
+ * [Embed into HTML](#embed-into-html)
+ + [1. Embed a snippet from Playground Visible Catalog](#1-embed-a-snippet-from-playground-visible-catalog)
+ + [2. Embed User-shared Code](#2-embed-user-shared-code)
+ + [3. Embed a snippet from any other source](#3-embed-a-snippet-from-any-other-source)
+ * [Embedding into the Beam documentation](#embedding-into-the-beam-documentation)
+
+
+## Step 1. Prepare Your Code
+
+Playground has multiple features to focus users to certain parts of the code.
+It does the following to all snippets:
+
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+Additionally it can apply the following view options:
+
+- Fold all blocks except certain ones.
+- Completely hide all code except certain parts.
+- Make certain parts read-only.
+
+If you do not need any of those view options, skip to the [next step](#step-2-place-your-code-somewhere-the-playground-can-load-it-from).
+
+### Named Sections
+
+For those features to work, the snippet must contain sections with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+See more on the syntax and limitations in the
+[README of the editor](https://pub.dev/packages/flutter_code_editor)
+that Playground uses.
+
+Create a named section for each part of your code that you want that features for.
+
+## Step 2. Place your code somewhere the Playground can load it from
+
+### Source 1. Playground Visible Catalog
+
+If you think your snippet is beneficial to many users, it's a good idea to add it to
+the Playground Catalog.
+
+Advantages:
+
+- Anyone can find your snippet in the dropdown when they open Playground.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+#### 1. Put the file to the directory
+
+Snippets for the Playground Catalog are automatically picked from predefined categories
+by a workflow **(WHICH ONE?)** after merging a PR.
+
+This directories are:
+
+| SDK | Snippet path root |
+|-|-|
+| Java | **?** |
+| Python | **?** |
+| Go | **?** |
+
+Place the file anywhere under the given directory.
+
+**TODO: NAMING, DIRECTORY STRUCTURE?**
+
+Snippets in Scala are not yet supported for by that workflow.
+All existing Scala examples are added manually by the team.
+Please use other options to place your Scala snippets.
+
+#### 2. Add metadata
+
+Playground needs metadata to put a snippet to the database. These are stored as a comment block
+anywhere in the snippet.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+This comment block is cut from the text before putting it to the database and so is not visible
+to end users. The block is in the format of a YAML map.
+
+The following attributes are required:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `description` | Description text to show on mouse over. | Text | Required |
+| `multifile` | Whether this is a file of a multi-file example. | `true`, `false` | Required |
+| `name` | Name to show in the dropdown. | Text| Required |
+
+The following optional attributes are supported:
+
+| Attribute | Description | Values |
+|-|-|-|
+| `categories` | Titles of categories to list this snippet in. Non-existent categories will be created. | Array of strings |
+| `complexity` | How hard is the snippet to understand. | `BASIC`, `MEDIUM`, `ADVANCED` |
+| `context_line` | The line number to scroll to when the snippet is loaded. This applies after the metadata block is removed from the file, so discount for those lines. | Integer, 1-based |
+| `default_example` | Whether this is the default example in its SDK. If multiple snippets set this to `true` the behavior is undefined. | `false` (default), `true` |
+| `tags` | Tags by which this snippet can be found in the dropdown. | Array of strings |
+
+**TODO: ARE THE REQUIRED AND OPTIONAL ATTRIBUTES IDENTIFIED CORRECTLY?**
+
+**TODO: ARE THERE ANY OTHER SUPPORTED ATTRIBUTES?**
+
+#### 3. Make a PR
+
+Make a PR into [the Beam repository](https://github.com/apache/beam).
+Make sure all checks are green.
+
+A reviewer will be assigned. After merge the snippet will be visible in the Playground dropdown.
+
+The CI we have will also check that the example compiles and runs alright.
+
+#### 4. Save the snippet path
+
+The snippet will be assigned a path to identify it.
+You can find it in the address bar of the browser when you select it in the dropdown.
+
+For example, this URL:
+
+```
+https://play.beam.apache.org/?path=SDK_JAVA%2FPRECOMPILED_OBJECT_TYPE_EXAMPLE%2FMinimalWordCount&sdk=java
+```
+
+after [URL decoding](https://www.urldecoder.org) indicates this snippet path:
+
+```
+SDK_JAVA/PRECOMPILED_OBJECT_TYPE_EXAMPLE/MinimalWordCount
+```
+
+You will need it to embed the Playground code.
+
+### Source 2. Playground Unlisted Database
+
+If your snippet is less useful to public for learning but still beneficial to some group
+of people, you can put it into the same database as the Playground Visible Catalog
+but make it unlisted.
+
+Advantages:
+
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+Proceed the same way as with Playground Visible Catalog, but do not use these attributes:
+
+- `categories`
+- `default_example`
+- `tags`
+
+**TODO: How to get the path of the example?**
+
+### Source 3. Tour of Beam unit
+
+If your snippet fills a gap in the Tour of Beam tutorial, you can add it there
+as a learning unit. This also requires a textual learning material that your snippet will accompany.
+
+Advantages:
+
+- Anyone can find your snippet in the Tour of Beam tutorial.
+- The CI of the Beam repository guarantees your snippet builds and runs correctly.
+- Output and graph are cached so the viewers of your snippet will not wait when they run it.
+
+**TODO: HOWTO**
+
+### Source 4. User-shared Code
+
+Code can be uploaded directly to the database with "Share my code" button in Playground:
+
+
+
+Advantages:
+
+- No approval required.
+- Fast.
+
+Drawbacks:
+
+- The code may be deleted if it is not loaded by anyone during a 6 months rolling window.
+- No validation.
+- No cached output and graph.
+- A snippet is immutable. If you edit the code and re-share, you get a new link.
+- You cannot delete the code uploaded by accident.
+
+After clicking the button you will get a shareable link or embeddable HTML code.
+Note the `shared` parameter in the link query string.
+It contains the ID of your snippet that you can later use with other sharing methods.
+
+### Source 5. HTTPS
+
+You can upload a snippet file to any HTTPS-server you have access to.
+Then you refer to it by a URL to load into Playground.
+
+Advantages:
+
+- No approval required.
+- Fast.
+- You can change the snippet without changing a link.
+
+Drawbacks:
+
+- No validation.
+- No cached output and graph.
+
+For Playground to be able to load the snippet over HTTPS,
+your server needs to allow the access by sending the following header:
+
+```
+Access-Control-Allow-Origin: *
+```
+
+at least when requested with `*.beam.apache.org` as
+[`referer`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer).
+
+To understand the reasons, read about
+[CORS (Cross-Origin Resource Sharing)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS).
+
+Many people prefer to host their snippets in their GitHub repositories.
+GitHub is known to allow cross-origin access on direct links to raw file content.
+
+## Step 3. Create a link or embed
+
+Once you uploaded or otherwise prepared your code, you can get it shown in the Playground.
+Choose any of the following ways.
+
+### Direct Link to the Standalone Playground Web App
+
+You can link to the Playground app so that it opens your code. The link depends on
+where you placed the code.
+
+#### 1. Link for a snippet from Playground Visible Catalog
+
+1. Open your snippet in the dropdown menu.
+2. Without changing it, click "Share my code".
+3. Copy the link.
+
+The link contains the `path` to your snippet in the database. It is in the following format:
+```
+https://play.beam.apache.org/?path=SDK_JAVA/PRECOMPILED_OBJECT_TYPE_KATA/AggregationMax&sdk=java
Review Comment:
This should now be simply `SDK_JAVA_AggregationMax`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rshamunov commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "rshamunov (via GitHub)" <gi...@apache.org>.
rshamunov commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1169855614
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,594 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
+- `/examples`
+- `/learning/katas`
+- `/sdks`.
+
+Adding Scala example snippets automatically is not supported,
Review Comment:
We need a description for Scala examples too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] alexeyinkin commented on a diff in pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "alexeyinkin (via GitHub)" <gi...@apache.org>.
alexeyinkin commented on code in PR #25507:
URL: https://github.com/apache/beam/pull/25507#discussion_r1163821775
##########
playground/load_your_code.md:
##########
@@ -0,0 +1,562 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to Add an Example/Snippet/Learning Content into Apache Beam Playground
+
+Adding a new example, code snippet or Tour of Beam learning unit into the Playground is a three-step process:
+
+1. Prepare a code snippet.
+2. Add the code snippet to Apache Beam and/or Playground.
+3. Create a link to view the code snippet in Playground or to embed in a website page.
+
+
+Playground sources and output presentation formats:
+
+
+
+The guide will walk through all steps.
+
+
+# Table of Contents
+
+- [Step 1. Prepare a code snippet](#step-1-prepare-a-code-snippet)
+ * [Data access limitations](#data-access-limitations)
+ * [Emphasizing parts of code](#emphasizing-parts-of-code)
+ * [Named Sections](#named-sections)
+- [Step 2. Add the code snippet to the Apache Beam repo and/or Playground](#step-2-add-the-code-snippet-to-the-apache-beam-repo-andor-playground)
+ * [Source 1. How to add an example to Playground Examples Catalog](#source-1-how-to-add-an-example-to-playground-examples-catalog)
+ + [1. Add the file with example to a directory](#1-add-the-file-with-example-to-a-directory)
+ + [2. Add metadata to describe example](#2-add-metadata-to-describe-the-example)
+ - [Kafka emulator](#kafka-emulator)
+ + [3. Make a PR](#3-make-a-pr)
+ + [4. Save the snippet ID](#4-save-the-snippet-id)
+ * [Source 2. How to add an unlisted example to Playground](#source-2-how-to-add-an-unlisted-example-to-playground)
+ * [Source 3. How to add a Tour of Beam unit](#source-3-how-to-add-a-tour-of-beam-unit)
+ * [Source 4. How to add a snippet with the app alone](#source-4-how-to-add-a-snippet-with-the-app-alone)
+ * [Source 5. How to load code from GitHub or other HTTPS URL](#source-5-how-to-load-code-from-github-or-other-https-url)
+- [Step 3. Create a link or embed snippet](#step-3-create-a-link-or-embed-snippet)
+ * [Link to a snippet](#link-to-a-snippet)
+ + [Link to an example from the Playground Examples Catalog](#link-to-an-example-from-the-playground-examples-catalog)
+ + [Link to snippet (`unlisted` Playground Examples Catalog item)](#link-to-snippet---unlisted--playground-examples-catalog-item-)
+ + [Link to a Tour of Beam unit](#link-to-a-tour-of-beam-unit)
+ + [Link to a user-shared snippet](#link-to-a-user-shared-snippet)
+ + [Link to a GitHub or HTTPS URL snippet](#link-to-a-github-or-https-url-snippet)
+ + [Link to an empty editor](#link-to-an-empty-editor)
+ * [Embedding a snippet into HTML](#embedding-a-snippet-into-html)
+ + [Embedding a snippet from Playground Examples Catalog](#embedding-a-snippet-from-playground-examples-catalog)
+ + [Embedding a user-shared snippet](#embedding-a-user-shared-snippet)
+ + [Embedding a snippet from other sources](#embedding-a-snippet-from-other-sources)
+ * [Embedding a snippet into the Apache Beam website](#embedding-a-snippet-into-the-apache-beam-website)
+- [Snippet view options](#snippet-view-options)
+ * [Read-only sections](#read-only-sections)
+ * [Folding everything except named sections](#folding-everything-except-named-sections)
+ * [Hiding everything except a named section](#hiding-everything-except-a-named-section)
+ * [Linking to multiple examples](#linking-to-multiple-examples)
+
+
+## Step 1. Prepare a code snippet
+
+Playground runs example code snippets using Apache Beam Direct Runner
+and requires that a code snippet is a complete runnable code.
+
+### Data access limitations
+
+For security reasons, the snippets in Playground cannot access arbitrary internet resources.
+Depending on your data source you should do the following to access data:
+
+| Source | Notes |
+|----------|-----------------------------------------------------------|
+| File | Put to a GCS bucket in `apache-beam-testing` project. |
+| BigQuery | The table must be added to `apache-beam-testing` project. |
+
+Machine Learning: PyTorch, SKLearn -- change the playground itself. (request for enhancement to add a dependency to the Python image).
+To request adding, please send an email to dev@...
+
+CLONING 3RD PARTY REPOS.
+IF ML MODELS -- FROM PLAYGROUND BEAM ML EXAMPLES SUPPORT
+
+### Emphasizing parts of code
+
+Playground provides multiple features to help focus users on certain parts of the code.
+
+Playground automatically applies the following to all snippets:
+- Folds a comment if a snippet starts with one.
+- Folds imports.
+
+### Named Sections
+
+Playground supports *Named Sections* to tag code blocks and provide the following view options:
+- Fold all blocks except tagged code blocks.
+ This can be useful to help user focus on specific code blocks and features presented in a snippet.
+- Hide all code except tagged code blocks.
+ This can be useful to create runnable snippets illustrating specific concepts or transforms,
+ and hide all non-essential code blocks.
+ Such snippet can be embedded on a website to make examples in documentation and tutorials runnable.
+- Make certain code parts read-only.
+ This feature can be useful to create learning units where user modifications are desired
+ only in certain parts of the code.
+Please see [Snippet View Options](#snippet-view-options) section for details how different view options can be used.
+
+If you do not need any of those view options, skip to the [next step](#step-2-add-the-code-snippet-to-the-apache-beam-repo-and-playground).
+
+*Named Sections* are defined with the following syntax:
+
+```
+// [START section_name]
+void method() {
+...
+}
+// [END section_name]
+```
+
+Create a named section for each part of your code that you want the above features for.
+To learn more details about the syntax please see
+the [README of the editor](https://pub.dev/packages/flutter_code_editor) that Playground uses.
+
+## Step 2. Add the code snippet to the Apache Beam repo and/or Playground
+
+There are several types of code snippets in the Playground:
+
+1. Public Example — a code snippet displayed in the Playground Examples Catalog.
+ See [how to add a new public example here](#source-1-how-to-add-an-example-to-playground-examples-catalog).
+2. Unlisted Example — the same as a public example, but it does not show in the example dropdown
+ and can only be accessed by direct linking. These are typically embedded on a website.
+ See [how to add a new unlisted example here](#source-2-how-to-add-an-unlisted-example-to-playground).
+3. [Tour of Beam](https://github.com/apache/beam/tree/master/learning/tour-of-beam) learning unit.
+ See [how to add a new Tour of Beam unit here](#source-3-how-to-add-a-tour-of-beam-unit).
+4. User-shared code snippets do not require a PR and should be used for code
+ not displayed on Beam resources.
+ See [how to add a snippet with the app alone here](#source-4-how-to-add-a-snippet-with-the-app-alone).
+5. GitHub or other HTTPS URL sources.
+ See [how to load a snippet from external GitHub or other HTTPS URL here](#source-5-how-to-load-code-from-github-or-other-https-url).
+
+See the [workflow above](#how-to-add-an-examplesnippetlearning-content-into-apache-beam-playground) how artifacts map to these sources.
+
+### Source 1. How to add an example to Playground Examples Catalog
+
+Playground Examples Catalog helps users discover example snippets
+and is a recommended way to add examples. Playground automatically scans,
+verifies and deploys example snippets from the directories listed below.
+
+#### 1. Add the file with example to a directory
+
+Playground Java, Python, and Go examples are automatically picked from these
+predefined directories by the `playground_examples_ci.yml` GitHub workflow
+after a PR is merged to Beam repo:
+- `/examples`
+- `/learning/katas`
+- `/sdks`.
+
+Adding Scala example snippets automatically is not supported,
+and Scala example snippets can be added to the catalog manually.
+
+#### 2. Add metadata to describe the example
+
+Playground relies on metadata comments block to identify and place an example into the database,
+and present in the Examples Catalog.
+See [this](https://github.com/apache/beam/blob/3e080ff212d8ed7208c8486b515bb73c5d294475/examples/java/src/main/java/org/apache/beam/examples/MinimalWordCount.java#L20-L36) for an example.
+Playground automatically removes metadata comments block before storing the example in database
+so the metadata is not visible to end users. The block is in the format of a YAML map.
+
+Example tag metadata quick reference (all elements are **required**, unless marked **optional**):
+
+| Field | Description |
Review Comment:
Nice. But looks like the optionality is not exactly that.
- `name` is optional according to `models.py` and defaults to `Field(..., min_length=1)` whatever it means. Should we make it required?
- `categories` is required. So in instructions for unlisted examples and ToB snippets we instruct folks to use `categories: []` which looks redundant. Should we make it optional?
- Also `context_line` is required for some reason. But we have a plan to use a comment instead of the line number: https://github.com/apache/beam/issues/23774 Making this optional now will simplify the transition. May default to 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem commented on PR #25507:
URL: https://github.com/apache/beam/pull/25507#issuecomment-1577179377
Run Website PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem merged pull request #25507: Overview of adding snippets to Playground (#25506)
Posted by "pabloem (via GitHub)" <gi...@apache.org>.
pabloem merged PR #25507:
URL: https://github.com/apache/beam/pull/25507
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org