You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "jojochuang (via GitHub)" <gi...@apache.org> on 2026/01/03 03:38:42 UTC

[PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

jojochuang opened a new pull request, #206:
URL: https://github.com/apache/ozone-site/pull/206

   ## What changes were proposed in this pull request?
   
   HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration
   
   Please describe your PR in detail:
   - A self contained tutorial to guide users to set up Flink to access Ozone.
   - Draft. I tested the steps myself but need one more iteration to clean it up.
   
   ## What is the link to the Apache Jira?
   
   https://issues.apache.org/jira/browse/HDDS-14340
   
   ## How was this patch tested?
   
   Check off which of the following tests were done on this change. If additional testing was done, please elaborate here as well.
   
   - [ ] The CI checks on my fork are passing
   - [ ] I verified the rendered content using a local preview
   - [ ] I manually verified the steps provided in this change work as described
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748144081


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together
+
+With both `docker-compose.yaml` (for Ozone) and `docker-compose-flink.yml` (for Flink) in the same directory,
+you can start both services together, sharing the same network, using:
+
+```bash
+export COMPOSE_FILE=docker-compose.yaml:docker-compose-flink.yml
+docker compose up -d
+```
+
+Verify containers are running:
+
+```bash
+docker ps
+```
+
+## Step 3 — Create an Ozone bucket
+
+You need to connect to Ozone (for example, `s3g`) to create a OBS bucket:
+
+```bash
+docker compose exec -it s3g ozone sh bucket create s3v/bucket1 -l obs
+```
+
+## Step 4 — Copy the Flink S3 filesystem plugin
+
+The official Flink Docker image does not enable S3 by default.
+You must copy the plugin JAR into both JobManager and TaskManager.
+
+Copy into JobManager
+
+```bash
+docker compose exec -it jobmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Copy into TaskManager
+
+```bash
+docker compose exec -it taskmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Verify:
+
+```bash
+docker compose exec -it jobmanager ls /opt/flink/plugins/s3-fs-hadoop
+docker compose exec -it taskmanager ls /opt/flink/plugins/s3-fs-hadoop
+```
+
+## Step 5 — Restart Flink containers (required)
+
+Plugins are loaded only at startup.
+
+```bash
+docker compose restart jobmanager taskmanager
+```
+
+## Step 6 — Start Flink SQL client
+
+```bash
+docker compose exec -it jobmanager ./bin/sql-client.sh
+```
+
+You should now be in:
+
+```text
+Flink SQL>
+```
+
+## Step 7 — Create a table backed by Ozone S3
+
+Important: Must use BATCH mode otherwise multi-part upload fails.
+
+```sql
+SET 'execution.runtime-mode' = 'BATCH';
+
+CREATE TABLE ozone_sink (
+  id STRING,
+  ts TIMESTAMP(3)
+) WITH (
+  'connector' = 'filesystem',
+  'path' = 's3a://bucket1/ozone_sink/',
+  'format' = 'csv'
+);
+```
+
+Insert data:
+
+```sql
+INSERT INTO ozone_sink VALUES ('hello', CURRENT_TIMESTAMP);
+```
+
+Query it:
+
+```sql
+SELECT * FROM ozone_sink;
+```
+
+If this works, Flink is successfully reading/writing Ozone via S3.
+
+## Step 8 — Check Flink job status in the Web UI

Review Comment:
   ```suggestion
   ### Step 7 — Check Flink job status in the Web UI
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748054683


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager

Review Comment:
   ```suggestion
       command: >
         bash -c "mkdir -p /opt/flink/plugins/s3-fs-hadoop &&
         cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/ &&
         /docker-entrypoint.sh jobmanager"
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748058876


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together
+
+With both `docker-compose.yaml` (for Ozone) and `docker-compose-flink.yml` (for Flink) in the same directory,
+you can start both services together, sharing the same network, using:
+
+```bash
+export COMPOSE_FILE=docker-compose.yaml:docker-compose-flink.yml
+docker compose up -d
+```
+
+Verify containers are running:
+
+```bash
+docker ps
+```
+
+## Step 3 — Create an Ozone bucket

Review Comment:
   ```suggestion
   ### Step 4 — Create an Ozone bucket
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748027058


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,190 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+This tutorial shows how to connect Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+

Review Comment:
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748056725


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together

Review Comment:
   ```suggestion
   ### Step 3 — Start Flink and Ozone together
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748065293


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together
+
+With both `docker-compose.yaml` (for Ozone) and `docker-compose-flink.yml` (for Flink) in the same directory,
+you can start both services together, sharing the same network, using:
+
+```bash
+export COMPOSE_FILE=docker-compose.yaml:docker-compose-flink.yml
+docker compose up -d
+```
+
+Verify containers are running:
+
+```bash
+docker ps
+```
+
+## Step 3 — Create an Ozone bucket
+
+You need to connect to Ozone (for example, `s3g`) to create a OBS bucket:
+
+```bash
+docker compose exec -it s3g ozone sh bucket create s3v/bucket1 -l obs
+```
+
+## Step 4 — Copy the Flink S3 filesystem plugin
+
+The official Flink Docker image does not enable S3 by default.
+You must copy the plugin JAR into both JobManager and TaskManager.
+
+Copy into JobManager
+
+```bash
+docker compose exec -it jobmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Copy into TaskManager
+
+```bash
+docker compose exec -it taskmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Verify:
+
+```bash
+docker compose exec -it jobmanager ls /opt/flink/plugins/s3-fs-hadoop
+docker compose exec -it taskmanager ls /opt/flink/plugins/s3-fs-hadoop
+```
+
+## Step 5 — Restart Flink containers (required)
+
+Plugins are loaded only at startup.
+
+```bash
+docker compose restart jobmanager taskmanager
+```
+
+## Step 6 — Start Flink SQL client
+
+```bash
+docker compose exec -it jobmanager ./bin/sql-client.sh
+```
+
+You should now be in:
+
+```text
+Flink SQL>
+```
+
+## Step 7 — Create a table backed by Ozone S3

Review Comment:
   ```suggestion
   ### Step 6 — Create a table backed by Ozone S3
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748185767


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+

Review Comment:
   ```suggestion
   
   Edit the `docker-compose.yaml`:
   
   Append the last 2 SCM safemode configurations to the `x-common-config:` section to enable starting with a single datanode.
   
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2747922326


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,190 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+This tutorial shows how to connect Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.

Review Comment:
   ```suggestion
   [Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
   
   Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
   
   ## Quickstart
   
   This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #206:
URL: https://github.com/apache/ozone-site/pull/206


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748065293


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together
+
+With both `docker-compose.yaml` (for Ozone) and `docker-compose-flink.yml` (for Flink) in the same directory,
+you can start both services together, sharing the same network, using:
+
+```bash
+export COMPOSE_FILE=docker-compose.yaml:docker-compose-flink.yml
+docker compose up -d
+```
+
+Verify containers are running:
+
+```bash
+docker ps
+```
+
+## Step 3 — Create an Ozone bucket
+
+You need to connect to Ozone (for example, `s3g`) to create a OBS bucket:
+
+```bash
+docker compose exec -it s3g ozone sh bucket create s3v/bucket1 -l obs
+```
+
+## Step 4 — Copy the Flink S3 filesystem plugin
+
+The official Flink Docker image does not enable S3 by default.
+You must copy the plugin JAR into both JobManager and TaskManager.
+
+Copy into JobManager
+
+```bash
+docker compose exec -it jobmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Copy into TaskManager
+
+```bash
+docker compose exec -it taskmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Verify:
+
+```bash
+docker compose exec -it jobmanager ls /opt/flink/plugins/s3-fs-hadoop
+docker compose exec -it taskmanager ls /opt/flink/plugins/s3-fs-hadoop
+```
+
+## Step 5 — Restart Flink containers (required)
+
+Plugins are loaded only at startup.
+
+```bash
+docker compose restart jobmanager taskmanager
+```
+
+## Step 6 — Start Flink SQL client
+
+```bash
+docker compose exec -it jobmanager ./bin/sql-client.sh
+```
+
+You should now be in:
+
+```text
+Flink SQL>
+```
+
+## Step 7 — Create a table backed by Ozone S3

Review Comment:
   ```suggestion
   ### Step 6 — Create and Query a table backed by Ozone S3
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748185767


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+

Review Comment:
   ````suggestion
   
   Edit the `docker-compose.yaml`:
   
   Append the last 2 SCM safemode configurations to the `x-common-config:` section to enable starting with a single datanode.
   
   ```bash
   x-common-config:
      &common-config
      ...
      no_proxy: "om,recon,scm,s3g,localhost,127.0.0.1"
      OZONE-SITE.XML_hdds.scm.safemode.min.datanode: "1"
      OZONE-SITE.XML_hdds.scm.safemode.healthy.pipeline.pct: "0" 
   ```
   
   ````
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #206:
URL: https://github.com/apache/ozone-site/pull/206#issuecomment-3829287706

   push a commit to fix markdownlink and spelling error.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748062524


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together
+
+With both `docker-compose.yaml` (for Ozone) and `docker-compose-flink.yml` (for Flink) in the same directory,
+you can start both services together, sharing the same network, using:
+
+```bash
+export COMPOSE_FILE=docker-compose.yaml:docker-compose-flink.yml
+docker compose up -d
+```
+
+Verify containers are running:
+
+```bash
+docker ps
+```
+
+## Step 3 — Create an Ozone bucket
+
+You need to connect to Ozone (for example, `s3g`) to create a OBS bucket:
+
+```bash
+docker compose exec -it s3g ozone sh bucket create s3v/bucket1 -l obs
+```
+
+## Step 4 — Copy the Flink S3 filesystem plugin
+
+The official Flink Docker image does not enable S3 by default.
+You must copy the plugin JAR into both JobManager and TaskManager.
+
+Copy into JobManager
+
+```bash
+docker compose exec -it jobmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Copy into TaskManager
+
+```bash
+docker compose exec -it taskmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Verify:
+
+```bash
+docker compose exec -it jobmanager ls /opt/flink/plugins/s3-fs-hadoop
+docker compose exec -it taskmanager ls /opt/flink/plugins/s3-fs-hadoop
+```
+
+## Step 5 — Restart Flink containers (required)
+
+Plugins are loaded only at startup.
+
+```bash
+docker compose restart jobmanager taskmanager
+```
+

Review Comment:
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748055260


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager

Review Comment:
   ```suggestion
       command: >
         bash -c "mkdir -p /opt/flink/plugins/s3-fs-hadoop &&
         cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/ &&
         /docker-entrypoint.sh taskmanager"
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #206:
URL: https://github.com/apache/ozone-site/pull/206#issuecomment-3825968329

   I'll just commit the recommended changes and let's see how it renders.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748174587


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.

Review Comment:
   ```suggestion
   Edit the `docker-compose.yaml`:
   
   Append the last 2 SCM safemode configurations to the `x-common-config:` section to enable starting with a single datanode.
   ```
   x-common-config:
      &common-config
      ...
      no_proxy: "om,recon,scm,s3g,localhost,127.0.0.1"
      OZONE-SITE.XML_hdds.scm.safemode.min.datanode: "1"
      OZONE-SITE.XML_hdds.scm.safemode.healthy.pipeline.pct: "0" 
   ```
   
   Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748174587


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.

Review Comment:
   ```suggestion
   Edit the `docker-compose.yaml`:
   
   Append the last 2 SCM safemode configurations to the `x-common-config:` section to enable starting with a single datanode.
   ```
   x-common-config:
      &common-config
      ...
      no_proxy: "om,recon,scm,s3g,localhost,127.0.0.1"
      OZONE-SITE.XML_hdds.scm.safemode.min.datanode: "1"
      OZONE-SITE.XML_hdds.scm.safemode.healthy.pipeline.pct: "0" 
   ```
   
   Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748052502


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink

Review Comment:
   ```suggestion
   ## Step 2 — Create `docker-compose-flink.yml` for Flink
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748052502


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink

Review Comment:
   ```suggestion
   ### Step 2 — Create `docker-compose-flink.yml` for Flink
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748185767


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+

Review Comment:
   ````suggestion
   
   Edit the `docker-compose.yaml`:
   
   Append the last 2 SCM safemode configurations to the `x-common-config:` section to enable starting with a single datanode.
   
   ```bash
   x-common-config:
      &common-config
      ...
      no_proxy: "om,recon,scm,s3g,localhost,127.0.0.1"
      OZONE-SITE.XML_hdds.scm.safemode.min.datanode: "1"
      OZONE-SITE.XML_hdds.scm.safemode.healthy.pipeline.pct: "0" 
   ```
   ````
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748025645


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,190 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+This tutorial shows how to connect Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:

Review Comment:
   ```suggestion
   ### Quickstart environment 
   
   - Unsecure Ozone and Flink clusters.
   - Ozone S3G enables path style access. To enable virtual-host style addressing see [here](../../01-client-interfaces/03-s3/01-s3-api.md#url-schema).
   - Flink accesses Ozone via S3 Gateway.
   
   ### Step 1 — Download Ozone's `docker-compose.yaml`
   
   First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2747922326


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,190 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+This tutorial shows how to connect Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.

Review Comment:
   ```suggestion
   [Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
   
   Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
   
   This tutorial shows how to connect Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748063591


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together
+
+With both `docker-compose.yaml` (for Ozone) and `docker-compose-flink.yml` (for Flink) in the same directory,
+you can start both services together, sharing the same network, using:
+
+```bash
+export COMPOSE_FILE=docker-compose.yaml:docker-compose-flink.yml
+docker compose up -d
+```
+
+Verify containers are running:
+
+```bash
+docker ps
+```
+
+## Step 3 — Create an Ozone bucket
+
+You need to connect to Ozone (for example, `s3g`) to create a OBS bucket:
+
+```bash
+docker compose exec -it s3g ozone sh bucket create s3v/bucket1 -l obs
+```
+
+## Step 4 — Copy the Flink S3 filesystem plugin
+
+The official Flink Docker image does not enable S3 by default.
+You must copy the plugin JAR into both JobManager and TaskManager.
+
+Copy into JobManager
+
+```bash
+docker compose exec -it jobmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Copy into TaskManager
+
+```bash
+docker compose exec -it taskmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Verify:
+
+```bash
+docker compose exec -it jobmanager ls /opt/flink/plugins/s3-fs-hadoop
+docker compose exec -it taskmanager ls /opt/flink/plugins/s3-fs-hadoop
+```
+
+## Step 5 — Restart Flink containers (required)
+
+Plugins are loaded only at startup.
+
+```bash
+docker compose restart jobmanager taskmanager
+```
+
+## Step 6 — Start Flink SQL client

Review Comment:
   ```suggestion
   ### Step 5 — Start Flink SQL client
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14340. [Website v2] [Docs] [User Guide] Flink integration [ozone-site]

Posted by "SaketaChalamchala (via GitHub)" <gi...@apache.org>.
SaketaChalamchala commented on code in PR #206:
URL: https://github.com/apache/ozone-site/pull/206#discussion_r2748148677


##########
docs/04-user-guide/03-integrations/09-flink.md:
##########
@@ -0,0 +1,196 @@
+---
+sidebar_label: Flink
+---
+
+# Apache Flink
+
+[Apache Flink](https://flink.apache.org/) is a powerful, open-source distributed processing framework designed for stateful computations over both bounded and unbounded data streams at any scale. It enables high-throughput, low-latency, and fault-tolerant processing while offering elastic scaling capabilities to handle millions of events per second across thousands of cores. 
+
+Apache Flink can use Apache Ozone for reading and writing data, and for storing essential operational components like application state checkpoints and savepoints.
+
+## Quickstart
+
+This tutorial shows how to get started with connecting Apache Flink to Apache Ozone using the S3 Gateway, with Docker Compose.
+
+First, obtain Ozone's sample Docker Compose configuration and save it as `docker-compose.yaml`:
+
+```bash
+curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml
+```
+
+Refer to the [Docker quick start page](../../02-quick-start/01-installation/01-docker.md) for details.
+
+## Assumptions
+
+- Flink accesses Ozone through S3 Gateway instead of ofs.
+- Ozone S3G listens on port 9878
+- Ozone S3G enables path style access.
+- Ozone S3G does not enable security, therefore any S3 access key and secret key is accepted.
+- Flink Docker image tag `flink:scala_2.12-java17`
+
+## Step 1 — Create `docker-compose-flink.yml` for Flink
+
+```yaml
+services:
+  jobmanager:
+    image: flink:scala_2.12-java17
+    command: jobmanager
+    ports:
+      - "8081:8081"
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+
+  taskmanager:
+    image: flink:scala_2.12-java17
+    command: taskmanager
+    depends_on:
+      - jobmanager
+    environment:
+      AWS_ACCESS_KEY_ID: ozone
+      AWS_SECRET_ACCESS_KEY: ozone
+      FLINK_PROPERTIES: |
+        jobmanager.rpc.address: jobmanager
+        taskmanager.numberOfTaskSlots: 4
+        fs.s3a.endpoint: http://s3g:9878
+        fs.s3a.path.style.access: true
+        fs.s3a.connection.ssl.enabled: false
+        fs.s3a.access.key: ozone
+        fs.s3a.secret.key: ozone
+```
+
+## Step 2 — Start Flink and Ozone together
+
+With both `docker-compose.yaml` (for Ozone) and `docker-compose-flink.yml` (for Flink) in the same directory,
+you can start both services together, sharing the same network, using:
+
+```bash
+export COMPOSE_FILE=docker-compose.yaml:docker-compose-flink.yml
+docker compose up -d
+```
+
+Verify containers are running:
+
+```bash
+docker ps
+```
+
+## Step 3 — Create an Ozone bucket
+
+You need to connect to Ozone (for example, `s3g`) to create a OBS bucket:
+
+```bash
+docker compose exec -it s3g ozone sh bucket create s3v/bucket1 -l obs
+```
+
+## Step 4 — Copy the Flink S3 filesystem plugin
+
+The official Flink Docker image does not enable S3 by default.
+You must copy the plugin JAR into both JobManager and TaskManager.
+
+Copy into JobManager
+
+```bash
+docker compose exec -it jobmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Copy into TaskManager
+
+```bash
+docker compose exec -it taskmanager bash -lc \
+  "mkdir -p /opt/flink/plugins/s3-fs-hadoop && \\
+   cp /opt/flink/opt/flink-s3-fs-hadoop-*.jar /opt/flink/plugins/s3-fs-hadoop/"
+```
+
+Verify:
+
+```bash
+docker compose exec -it jobmanager ls /opt/flink/plugins/s3-fs-hadoop
+docker compose exec -it taskmanager ls /opt/flink/plugins/s3-fs-hadoop
+```
+
+## Step 5 — Restart Flink containers (required)
+
+Plugins are loaded only at startup.
+
+```bash
+docker compose restart jobmanager taskmanager
+```
+
+## Step 6 — Start Flink SQL client
+
+```bash
+docker compose exec -it jobmanager ./bin/sql-client.sh
+```
+
+You should now be in:
+
+```text
+Flink SQL>
+```
+
+## Step 7 — Create a table backed by Ozone S3
+
+Important: Must use BATCH mode otherwise multi-part upload fails.
+
+```sql
+SET 'execution.runtime-mode' = 'BATCH';
+
+CREATE TABLE ozone_sink (
+  id STRING,
+  ts TIMESTAMP(3)
+) WITH (
+  'connector' = 'filesystem',
+  'path' = 's3a://bucket1/ozone_sink/',
+  'format' = 'csv'
+);
+```
+
+Insert data:
+
+```sql
+INSERT INTO ozone_sink VALUES ('hello', CURRENT_TIMESTAMP);
+```
+
+Query it:
+
+```sql
+SELECT * FROM ozone_sink;
+```
+
+If this works, Flink is successfully reading/writing Ozone via S3.
+
+## Step 8 — Check Flink job status in the Web UI
+
+Open your browser:
+
+```text
+http://localhost:8081/
+```
+
+Here you can:
+
+- See running and completed jobs
+- Inspect TaskManagers
+- Debug failures visually
+
+This is the first place to look if something goes wrong.
+
+## Key takeaways (important)
+
+- Flink Docker images do not ship with S3 enabled
+- The S3 plugin must exist in both JM and TM
+- Flink and Ozone should be started using a combined Docker Compose file (`COMPOSE_FILE`) to ensure they share the same network.
+- Always use `s3a://` with `flink-s3-fs-hadoop`
+- Restart Flink after copying plugins

Review Comment:
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org