You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "yangguangfu007 (via GitHub)" <gi...@apache.org> on 2023/10/19 15:00:34 UTC
[PR] [Feature] Submit flink Job base implementation [incubator-paimon-webui]
yangguangfu007 opened a new pull request, #71:
URL: https://github.com/apache/incubator-paimon-webui/pull/71
fix: https://github.com/apache/incubator-paimon-webui/issues/23
### Purpose
Submit flink Job base implementation
### Tests
<!-- List UT and IT cases to verify this change -->
### API and Format
<!-- Does this change affect API or storage format -->
### Documentation
<!-- Does this change introduce a new feature -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature] Support Flink job submission baseline implementation [incubator-paimon-webui]
Posted by "yangguangfu007 (via GitHub)" <gi...@apache.org>.
yangguangfu007 commented on code in PR #71:
URL: https://github.com/apache/incubator-paimon-webui/pull/71#discussion_r1367688445
##########
paimon-web-flink/paimon-web-flink-common/pom.xml:
##########
@@ -0,0 +1,85 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.paimon</groupId>
+ <artifactId>paimon-web-flink</artifactId>
+ <version>0.1-SNAPSHOT</version>
+ </parent>
+
+ <packaging>jar</packaging>
+
+ <artifactId>paimon-web-flink-common</artifactId>
+ <name>Paimon : Web : Flink: Common</name>
+
+ <properties>
+ <maven.compiler.source>${target.java.version}</maven.compiler.source>
+ <maven.compiler.target>${target.java.version}</maven.compiler.target>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <flink.version>1.17.1</flink.version>
+ </properties>
+
+ <dependencies>
+ <!-- Flink Base -->
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-java</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-streaming-java</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-clients</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-core</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <!-- Flink Table API -->
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-table-api-java-bridge</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-table-planner_${scala.binary.version}</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ </dependencies>
+</project>
Review Comment:
Thanks to the review, I will rewrite the code using sql-gateway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature] Support Flink job submission baseline implementation [incubator-paimon-webui]
Posted by "yangguangfu007 (via GitHub)" <gi...@apache.org>.
yangguangfu007 closed pull request #71: [Feature] Support Flink job submission baseline implementation
URL: https://github.com/apache/incubator-paimon-webui/pull/71
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature] Support Flink job submission baseline implementation [incubator-paimon-webui]
Posted by "SteNicholas (via GitHub)" <gi...@apache.org>.
SteNicholas commented on code in PR #71:
URL: https://github.com/apache/incubator-paimon-webui/pull/71#discussion_r1366411380
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/ApplicationExecutorContext.java:
##########
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context;
+
+import org.apache.flink.api.common.RuntimeExecutionMode;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+
+/**
+ * The ApplicationExecutorContext class is an extension of the ExecutorContext, designed to work
Review Comment:
```suggestion
* This application executor context implementation is an extension of {@link ExecutorContext}, which provides executor context
```
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/ExecutorContext.java:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context;
+
+import org.apache.flink.api.common.RuntimeExecutionMode;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.table.api.EnvironmentSettings;
+import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;
+
+/** The ExecutorContext class provides the context for creating an Executor. */
Review Comment:
```suggestion
/** The context interface provides the context to create an Executor. */
```
##########
paimon-web-flink/paimon-web-flink-common/pom.xml:
##########
@@ -0,0 +1,85 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.paimon</groupId>
+ <artifactId>paimon-web-flink</artifactId>
+ <version>0.1-SNAPSHOT</version>
+ </parent>
+
+ <packaging>jar</packaging>
+
+ <artifactId>paimon-web-flink-common</artifactId>
+ <name>Paimon : Web : Flink: Common</name>
+
+ <properties>
+ <maven.compiler.source>${target.java.version}</maven.compiler.source>
+ <maven.compiler.target>${target.java.version}</maven.compiler.target>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <flink.version>1.17.1</flink.version>
+ </properties>
+
+ <dependencies>
+ <!-- Flink Base -->
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-java</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-streaming-java</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-clients</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-core</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+
+ <!-- Flink Table API -->
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-table-api-java-bridge</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-table-planner_${scala.binary.version}</artifactId>
+ <version>${flink.version}</version>
+ <scope>provided</scope>
+ </dependency>
+ </dependencies>
+</project>
Review Comment:
New line at the end of file.
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/params/RemoteParams.java:
##########
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context.params;
+
+/** This class represents the parameters required to connect to a remote Flink cluster. */
+public class RemoteParams {
+
+ private final String host;
+ private final int port;
+ private String[] jarFilePath;
+
+ public RemoteParams(String host, int port, String[] jarFilePath) {
Review Comment:
Why does the `jarFilePath` use string array?
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/executor/Executor.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.executor;
+
+import org.apache.flink.api.dag.Transformation;
+import org.apache.flink.configuration.PipelineOptions;
+import org.apache.flink.runtime.jobgraph.JobGraph;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.streaming.api.graph.StreamGraph;
+import org.apache.flink.table.api.TableException;
+import org.apache.flink.table.api.TableResult;
+import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;
+import org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl;
+import org.apache.flink.table.api.internal.TableEnvironmentImpl;
+import org.apache.flink.table.delegation.Parser;
+import org.apache.flink.table.delegation.Planner;
+import org.apache.flink.table.operations.ModifyOperation;
+import org.apache.flink.table.operations.Operation;
+
+import java.lang.reflect.Method;
+import java.util.ArrayList;
+import java.util.List;
+
+/** The Executor interface. */
+public interface Executor {
+
+ TableResult executeSql(String statement);
+
+ TableResult executeStatementSet(List<String> statements);
+
+ JobGraph getJobGraph(List<String> statements);
+
+ default StreamGraph getStreamGraph(
+ StreamExecutionEnvironment env,
+ StreamTableEnvironment tableEnv,
+ List<String> statements) {
+ try {
+ Method getParserMethod = TableEnvironmentImpl.class.getDeclaredMethod("getParser");
+ getParserMethod.setAccessible(true);
+ Parser parser = (Parser) getParserMethod.invoke(tableEnv);
+
+ List<ModifyOperation> modifyOperations = new ArrayList<>();
+ for (String statement : statements) {
+ List<Operation> operations = parser.parse(statement);
+ if (operations.size() != 1) {
+ throw new TableException("Single statement required.");
+ } else {
+ Operation operation = operations.get(0);
+ if (operation instanceof ModifyOperation) {
+ modifyOperations.add((ModifyOperation) operation);
+ } else {
+ throw new TableException("INSERT statement required.");
Review Comment:
Why does the message use `INSERT statement`?
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/operation/FlinkSqlOperationType.java:
##########
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.operation;
+
+/**
+ * This enum represents the types of operations that can be performed in Flink SQL. It includes
+ * operations like SELECT, CREATE, DROP, ALTER, INSERT, etc. Each operation type is associated with
+ * a string value, and there are methods to compare the operation type with a string, check if the
+ * operation type is INSERT, and get the operation type from a SQL statement.
+ *
+ * <p>Typically, you would use this enum to categorize a SQL statement and perform different actions
+ * based on the operation type.
+ */
+public enum FlinkSqlOperationType {
Review Comment:
Could this directly use the SqlOperation of Flink SQL?
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/operation/SqlCategory.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.operation;
+
+/** Enum representing the category of SQL statements. */
+public enum SqlCategory {
Review Comment:
Could the enum directly use the enum of Flink table?
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/ApplicationExecutorContext.java:
##########
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context;
+
+import org.apache.flink.api.common.RuntimeExecutionMode;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+
+/**
+ * The ApplicationExecutorContext class is an extension of the ExecutorContext, designed to work
+ * with Flink applications running in YARN or Kubernetes application mode.
+ *
+ * <p>In this mode, Flink automatically identifies the runtime environment and returns the
+ * appropriate StreamExecutionEnvironment instance.
Review Comment:
```suggestion
* appropriated {@link StreamExecutionEnvironment} instance.
```
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/executor/ExecutorFactory.java:
##########
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.executor;
+
+import org.apache.paimon.web.flink.context.ExecutorContext;
+
+/** The ExecutorFactory interface provides a method to create an Executor. */
+public interface ExecutorFactory<C extends ExecutorContext> {
+
+ Executor createExecutor(C context);
Review Comment:
Adds the comments of this interface.
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/LocalExecutorContext.java:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context;
+
+import org.apache.flink.api.common.RuntimeExecutionMode;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+
+/** The LocalExecutorContext class provides the context for creating a LocalExecutor. */
Review Comment:
```suggestion
/** This local executor context implementation provides the context for creating a {@link LocalExecutor}. */
```
##########
paimon-web-flink/paimon-web-flink-common/src/test/java/org/apache/paimon/web/flink/parser/StatementParserTest.java:
##########
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.parser;
+
+import org.apache.paimon.web.flink.TestBase;
+
+import org.junit.jupiter.api.Test;
+
+import java.util.Arrays;
+import java.util.stream.Collectors;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+/** The test of {@link StatementParser}. */
+public class StatementParserTest extends TestBase {
+
+ @Test
+ public void testParse() {
+ String[] statements = StatementParser.parse(statement);
+ assertThat(statements.length).isEqualTo(5);
+ System.out.println(Arrays.stream(statements).collect(Collectors.toList()));
Review Comment:
Could this assert the parsed content?
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/ApplicationExecutorContext.java:
##########
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context;
+
+import org.apache.flink.api.common.RuntimeExecutionMode;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+
+/**
+ * The ApplicationExecutorContext class is an extension of the ExecutorContext, designed to work
+ * with Flink applications running in YARN or Kubernetes application mode.
Review Comment:
```suggestion
* with Flink jobs running in application mode.
```
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/ExecutorContext.java:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context;
+
+import org.apache.flink.api.common.RuntimeExecutionMode;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+import org.apache.flink.table.api.EnvironmentSettings;
+import org.apache.flink.table.api.bridge.java.StreamTableEnvironment;
+
+/** The ExecutorContext class provides the context for creating an Executor. */
+public abstract class ExecutorContext {
+
+ private StreamExecutionEnvironment environment;
+ private StreamTableEnvironment tableEnv;
+
+ protected final Configuration configuration;
+
+ private final RuntimeExecutionMode executionMode;
+
+ protected ExecutorContext(Configuration configuration, RuntimeExecutionMode executionMode) {
+ this.configuration = configuration;
+ this.executionMode = executionMode;
+ }
+
+ public void init() {
+ this.environment = createEnvironment();
+ EnvironmentSettings settings = createEnvironmentSettings(executionMode);
+ this.tableEnv = StreamTableEnvironment.create(environment, settings);
+ }
+
+ protected abstract StreamExecutionEnvironment createEnvironment();
+
+ public StreamExecutionEnvironment getEnvironment() {
+ return environment;
+ }
+
+ public StreamTableEnvironment getTableEnvironment() {
+ return tableEnv;
+ }
+
+ private EnvironmentSettings createEnvironmentSettings(RuntimeExecutionMode mode) {
+ return mode == RuntimeExecutionMode.BATCH
Review Comment:
What's the return value of `RuntimeExecutionMode.DYNAMIC`?
##########
paimon-web-flink/pom.xml:
##########
@@ -0,0 +1,66 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.paimon</groupId>
+ <artifactId>paimon-webui</artifactId>
+ <version>0.1-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>paimon-web-flink</artifactId>
+ <name>Paimon : Web : Flink</name>
+
+ <packaging>pom</packaging>
+
+ <modules>
+ <module>paimon-web-flink-common</module>
+ </modules>
+
+ <properties>
+ <maven.compiler.source>${target.java.version}</maven.compiler.source>
+ <maven.compiler.target>${target.java.version}</maven.compiler.target>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <scala.binary.version>2.12</scala.binary.version>
+ </properties>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.paimon</groupId>
+ <artifactId>paimon-web-common</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.assertj</groupId>
+ <artifactId>assertj-core</artifactId>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+
+</project>
Review Comment:
New line at the end of file.
##########
paimon-web-flink/paimon-web-flink-common/src/main/java/org/apache/paimon/web/flink/context/RemoteExecutorContext.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.paimon.web.flink.context;
+
+import org.apache.paimon.web.flink.context.params.RemoteParams;
+
+import org.apache.commons.lang3.ArrayUtils;
+import org.apache.flink.api.common.RuntimeExecutionMode;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
+
+/** The RemoteExecutorContext class provides the context for creating a RemoteExecutor. */
Review Comment:
```suggestion
/** The remote executor context implementation provides the context for creating a {@link RemoteExecutor}. */
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Feature] Support Flink job submission baseline implementation [incubator-paimon-webui]
Posted by "yangguangfu007 (via GitHub)" <gi...@apache.org>.
yangguangfu007 commented on PR #71:
URL: https://github.com/apache/incubator-paimon-webui/pull/71#issuecomment-1773714785
Thanks to the review, I will rewrite the code using sql-gateway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org