You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by "ashishm07 (via GitHub)" <gi...@apache.org> on 2025/12/01 14:35:18 UTC

[PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

ashishm07 opened a new pull request, #7122:
URL: https://github.com/apache/ignite-3/pull/7122

   Deployment Unit - Complete automation and standardization of all examples containing Deployment Unit
   Minor bug fixes to support complete automation.
   
    
   Tested on : apacheignite/ignite:3.1.0
    
   Repo : https://github.com/apache/ignite-3/tree/main/examples
    
   Absolute Path of affected files :
    
   examples/java/src/main/java/org/apache/ignite/example/compute/*  – All files under compute 
   
   examples/java/src/main/java/org/apache/ignite/example/code/deployment/CodeDeploymentExample.java
   
   examples/java/src/main/java/org/apache/ignite/example/serialization/SerializationExample.java
   
   examples/java/src/main/java/org/apache/ignite/example/streaming/DistributedComputeWithReceiverExample.java
   
   examples/java/src/main/java/org/apache/ignite/example/streaming/MultiTableDataStreamerExample.java
   
   Sample Error : 
   Exception in thread "main" org.apache.ignite.compute.ComputeException: IGN-COMPUTE-9 Job execution failed: java.lang.ClassNotFoundException: org.apache.ignite.example.compute.ComputeColocatedExample$PrintAccountInfoJob. Deployment unit computeExampleUnit:1.0.0 doesn't exist TraceId:e232314f
   
   Solution : Completely automated deployment unit dependent examples and made minor bug fixes to support the same


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashish-zs (via GitHub)" <gi...@apache.org>.
ashish-zs commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2642460499


##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,263 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+/**
+ * Utility class for building and deploying Ignite compute units.
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+
+    /**
+     * Builds a JAR file by packaging all compiled classes present in the given directory.
+     *
+     * @param classesDir Directory containing compiled .class files.
+     * @param jarPath Target JAR file path to create.
+     * @throws IOException If building the JAR fails.
+     */
+    public static void buildJar(Path classesDir, Path jarPath) throws IOException {
+        if (!Files.exists(classesDir)) {
+            throw new IllegalArgumentException("Compiled classes not found: " + classesDir);
+        }
+
+        Files.createDirectories(jarPath.getParent());
+
+        try (OutputStream fos = Files.newOutputStream(jarPath);
+                JarOutputStream jar = new JarOutputStream(fos, createManifest())) {
+
+            Files.walk(classesDir).filter(Files::isRegularFile).forEach(path -> {
+                String entry = classesDir.relativize(path).toString().replace("\\", "/");
+                try (InputStream is = Files.newInputStream(path)) {
+                    jar.putNextEntry(new JarEntry(entry));
+                    is.transferTo(jar);
+                    jar.closeEntry();
+                } catch (Exception e) {
+                    throw new RuntimeException(e);
+                }
+            });
+        }
+
+        System.out.println("JAR built: " + jarPath);
+    }
+
+    /**
+     * Creates a simple manifest declaring manifest version.
+     *
+     * @return Manifest object.
+     */
+    private static Manifest createManifest() {
+        Manifest m = new Manifest();
+        m.getMainAttributes().putValue("Manifest-Version", "1.0");
+        return m;
+    }
+
+    /**
+     * Deploys a unit only if it is not already deployed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnitIfNeeded(String unitId, String version, Path jar) throws Exception {
+        if (deploymentExists(unitId, version)) {
+            System.out.println("Deployment unit already active. Skipping deployment.");
+            return;
+        }
+        deployUnit(unitId, version, jar);
+        System.out.println("Deployment completed.");
+    }
+
+    /**
+     * Checks if a deployment unit already exists on the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if active deployment exists.
+     * @throws Exception If request fails.
+     */
+    public static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET().build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        Thread.sleep(500);

Review Comment:
   The deployment unit often remains in the UPLOADING state for a short period.
   This sleep was added to ensure that subsequent operations are only executed after the unit has been successfully deployed and transitions out of the UPLOADING state to DEPLOYED state.
   
   This can be revisited in the future and replaced with a polling or event-based approach once a reliable deployment status signal is available.
   



##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,263 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+/**
+ * Utility class for building and deploying Ignite compute units.
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+
+    /**
+     * Builds a JAR file by packaging all compiled classes present in the given directory.
+     *
+     * @param classesDir Directory containing compiled .class files.
+     * @param jarPath Target JAR file path to create.
+     * @throws IOException If building the JAR fails.
+     */
+    public static void buildJar(Path classesDir, Path jarPath) throws IOException {
+        if (!Files.exists(classesDir)) {
+            throw new IllegalArgumentException("Compiled classes not found: " + classesDir);
+        }
+
+        Files.createDirectories(jarPath.getParent());
+
+        try (OutputStream fos = Files.newOutputStream(jarPath);
+                JarOutputStream jar = new JarOutputStream(fos, createManifest())) {
+
+            Files.walk(classesDir).filter(Files::isRegularFile).forEach(path -> {
+                String entry = classesDir.relativize(path).toString().replace("\\", "/");
+                try (InputStream is = Files.newInputStream(path)) {
+                    jar.putNextEntry(new JarEntry(entry));
+                    is.transferTo(jar);
+                    jar.closeEntry();
+                } catch (Exception e) {
+                    throw new RuntimeException(e);
+                }
+            });
+        }
+
+        System.out.println("JAR built: " + jarPath);
+    }
+
+    /**
+     * Creates a simple manifest declaring manifest version.
+     *
+     * @return Manifest object.
+     */
+    private static Manifest createManifest() {
+        Manifest m = new Manifest();
+        m.getMainAttributes().putValue("Manifest-Version", "1.0");
+        return m;
+    }
+
+    /**
+     * Deploys a unit only if it is not already deployed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnitIfNeeded(String unitId, String version, Path jar) throws Exception {
+        if (deploymentExists(unitId, version)) {
+            System.out.println("Deployment unit already active. Skipping deployment.");
+            return;
+        }
+        deployUnit(unitId, version, jar);
+        System.out.println("Deployment completed.");
+    }
+
+    /**
+     * Checks if a deployment unit already exists on the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if active deployment exists.
+     * @throws Exception If request fails.
+     */
+    public static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET().build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        Thread.sleep(500);
+
+        if (resp.statusCode() != 200 && resp.statusCode() != 409) {
+            throw new RuntimeException("Deployment failed: " + resp.statusCode() + "\n" + resp.body());
+        }
+    }
+
+    /**
+     * Undeploys the given deployment unit from the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @throws Exception If undeployment fails.
+     */
+    public static void undeployUnit(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version))
+                .DELETE()
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        if (resp.statusCode() != 200 && resp.statusCode() != 404) {
+            throw new RuntimeException("Undeploy failed: " + resp.statusCode() + "\n" + resp.body());
+        }
+
+        for (int i = 0; i < 10; i++) {
+            if (!deploymentExists(unitId, version)) {
+                System.out.println("Unit successfully undeployed.");
+                return;
+            }
+            Thread.sleep(300);

Review Comment:
   The deployment unit often remains in the UPLOADING state for a short period.
   This sleep was added to ensure that subsequent operations are only executed after the unit has been successfully deployed and transitions out of the UPLOADING state to DEPLOYED state.
   
   This can be revisited in the future and replaced with a polling or event-based approach once a reliable deployment status signal is available.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2589089296


##########
examples/java/src/main/java/org/apache/ignite/example/serialization/AutoSerializableArg.java:
##########
@@ -1,33 +1,28 @@
-/*

Review Comment:
   Have added the licensing info back.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2589005364


##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeJobPriorityExample.java:
##########
@@ -122,13 +187,19 @@ public static void main(String[] args) {
             //--------------------------------------------------------------------------------------
 
             CompletableFuture.allOf(jobFutures.toArray(new CompletableFuture[0])).join();
+        } catch (Exception e) {
+            throw new RuntimeException(e);
+        } finally {
+
+            System.out.println("Cleaning up resources");
+            undeployUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION);
         }
     }
 
     /**
      * High-priority job.
      */
-    private static class HighPriorityJob implements ComputeJob<Integer, String> {
+    public static class HighPriorityJob implements ComputeJob<Integer, String> {

Review Comment:
   Made it public/static so Ignite can access and create the job class remotely without instantiation errors. 
   DeploymentUnit is not able to find these jobs unless we make it accessible remotely



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "iartiukhov (via GitHub)" <gi...@apache.org>.
iartiukhov commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2601428380


##########
examples/java/src/main/java/org/apache/ignite/example/serialization/AutoSerializableArg.java:
##########
@@ -1,5 +1,5 @@
 /*
- * Licensed to the Apache Software Foundation (ASF) under one or more
+ *Licensed to the Apache Software Foundation (ASF) under one or more

Review Comment:
   This line should not be changed



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;

Review Comment:
   Please add the ASF license header as in other files. 



##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,223 @@
+package org.apache.ignite.example.util;

Review Comment:
   Please add the ASF license header as in other files. 



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {
+
+    // Root path of ignite-examples/
+    protected static final Path projectRoot =
+            Paths.get("").toAbsolutePath();
+
+    // Compiled class output when running from IDE/CLI
+    protected static final Path DEFAULT_CLASSES_DIR =
+            projectRoot.resolve("examples/java/build/classes/java/main");
+
+    // Default JAR output
+    protected static final Path DEFAULT_JAR_PATH =
+            Path.of("build/libs/deploymentunit-example-1.0.0.jar");
+
+    protected static volatile String jarPathAsString = "";
+    protected static volatile Path jarPath = DEFAULT_JAR_PATH;
+    protected static volatile boolean runFromIDE = true;
+    // ---------------------------------------------------
+
+    /**
+     * Processes the deployment unit.
+     *
+     * @param args Arguments passed to the deployment process.
+     * @throws IOException if any error occurs.
+     */
+    protected static synchronized void processDeploymentUnit(String[] args)

Review Comment:
   What's the reason behind using the `synchronized` keyword? W do not execute several examples in parallel using more than one thread.



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/CodeDeploymentExample.java:
##########
@@ -17,32 +17,105 @@
 
 package org.apache.ignite.example.code.deployment;
 
+import static org.apache.ignite.example.util.DeployComputeUnit.deployUnit;
+import static org.apache.ignite.example.util.DeployComputeUnit.deploymentExists;
+import static org.apache.ignite.example.util.DeployComputeUnit.undeployUnit;
+
 import org.apache.ignite.client.IgniteClient;
+import org.apache.ignite.compute.BroadcastJobTarget;
+import org.apache.ignite.compute.IgniteCompute;
 import org.apache.ignite.compute.JobDescriptor;
 import org.apache.ignite.compute.JobTarget;
 import org.apache.ignite.deployment.DeploymentUnit;
 
-public class CodeDeploymentExample {
+/**
+ * This example demonstrates the usage of the {@link IgniteCompute#execute(BroadcastJobTarget, JobDescriptor, Object)} API.
+ *
+ * <p>Find instructions on how to run the example in the {@code README.md}
+ * file located in the {@code examples} directory root.</p>
+ *
+ * <h2>Execution Modes</h2>
+ *
+ * <p>There are two modes of execution:</p>
+ *
+ * <h3>1. Automated : The JAR Deployment for  deployment unit is automated </h3>
+ *
+ * <h4>1.1 With IDE</h4>
+ * <ul>
+ *   <li>
+ *     <b>Run from an IDE</b><br>
+ *     Launch the example directly from the IDE. If the required deployment
+ *     unit is not present, the example automatically builds and deploys the
+ *     necessary JAR.
+ *   </li>
+ * </ul>
+ *
+ * <h3>1.2 Without IDE</h3>
+ * <ul>
+ *   <li>
+ *     <b>Run from the command line</b><br>
+ *     Start the example using a Java command where the classpath includes
+ *     all required dependencies:
+ *
+ *     <pre>{@code
+ * java -cp "{user.home}\\.m2\\repository\\org\\apache\\ignite\\ignite-core\\3.1.0-SNAPSHOT\\
+ * ignite-core-3.1.0-SNAPSHOT.jar{other required jars}"
+ * <example-main-class> runFromIDE=false jarPath="{path-to-examples-jar}"
+ *     }</pre>
+ *
+ *     In this mode, {@code runFromIDE=false} indicates command-line execution,
+ *     and {@code jarPath} must reference the examples JAR used as the
+ *     deployment unit.
+ *   </li>
+ * </ul>
+ *
+ * <h2>2. Manual (with IDE): The JAR deployment for the deployment unit is manual</h2>
+ *
+ * <p>Before running this example, complete the following steps related to
+ * code deployment:</p>
+ *
+ * <ol>
+ *   <li>
+ *     Build the {@code ignite-examples-x.y.z.jar} file:<br>
+ *     {@code ./gradlew :ignite-examples:jar}
+ *   </li>
+ *   <li>
+ *     Deploy the generated JAR as a deployment unit using the CLI:<br>
+ *     <pre>{@code
+ * cluster unit deploy computeExampleUnit \
+ *     --version 1.0.0 \
+ *     --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar
+ *     }</pre>
+ *   </li>
+ * </ol>
+ */
+
+
+public class CodeDeploymentExample extends AbstractDeploymentUnitExample {
 
     /** Deployment unit name. */
     private static final String DEPLOYMENT_UNIT_NAME = "codeDeploymentExampleUnit";
 
     /** Deployment unit version. */
     private static final String DEPLOYMENT_UNIT_VERSION = "1.0.0";
 
-    /**
-     * Main method of the example.
-     *
-     * @param args The command line arguments.
-     */
-    public static void main(String[] args) {
 
+    public static void main(String[] args) throws Exception {
+        processDeploymentUnit(args);
         System.out.println("\nConnecting to server...");
 
         try (IgniteClient client = IgniteClient.builder().addresses("127.0.0.1:10800").build()) {
 
             System.out.println("\nConfiguring compute job...");
 
+            if (deploymentExists(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION)) {

Review Comment:
   Here and everywhere: let's extract this code block to an utility method to avoid massive code duplication.



##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeCancellationExample.java:
##########
@@ -29,42 +34,86 @@
 import org.apache.ignite.compute.JobExecutionContext;
 import org.apache.ignite.compute.JobTarget;
 import org.apache.ignite.deployment.DeploymentUnit;
+import org.apache.ignite.example.code.deployment.AbstractDeploymentUnitExample;
 import org.apache.ignite.lang.CancelHandle;
 import org.apache.ignite.lang.CancellationToken;
 
 /**
- * This example demonstrates the usage of the
- * {@link IgniteCompute#executeAsync(JobTarget, JobDescriptor, Object, CancellationToken)} API.
+ * This example demonstrates the usage of the {@link IgniteCompute#executeAsync(JobTarget, JobDescriptor, Object, CancellationToken)} API.
  *
- * <p>Find instructions on how to run the example in the README.md file located in the "examples" directory root.
+ * <p>Find instructions on how to run the example in the <code>README.md</code>
+ * file located in the "examples" directory root.</p>
+ *
+ * <h2>Execution Modes</h2>
+ *
+ * <p>There are two modes of execution:</p>
+ *
+ * <h3>1. Automated : The JAR Deployment for  deployment unit is automated </h3>
+ *
+ * <h4>1.1 With IDE</h4>
+ * <ul>
+ *     <li>
+ *         <b>Run from an IDE</b><br>
+ *         Launch the example directly from the IDE. If the required deployment
+ *         unit is not present, the example automatically builds and deploys the
+ *         necessary JAR.
+ *     </li>
+ * </ul>
+ *
+ * <h4>1.2 Without IDE</h4>
+ * <ul>
+ *     <li>
+ *         <b>Run from the command line</b><br>
+ *         Start the example using a Java command where the classpath includes all required
+ *         dependencies:<br>
+ *         {@code
+ *         java -cp "{user.home}\.m2\repository\org\apache\ignite\ignite-core\3.1.0-SNAPSHOT\
+ *         ignite-core-3.1.0-SNAPSHOT.jar{other required jars}"
+ *         <example-main-class> runFromIDE=false jarPath="{path-to-examples-jar}"}
+ *         <br>
+ *         In this mode, {@code runFromIDE=false} indicates command-line execution, and
+ *         {@code jarPath} must reference the examples JAR used as the deployment unit.
+ *     </li>
+ * </ul>
+ *
+ * <h3>2. Manual (with IDE) :  The JAR Deployment for  deployment unit is manual</h3>
+ *
+ * <p>Before running this example, complete the following steps related to
+ * code deployment:</p>
  *
- * <p>The following steps related to code deployment should be additionally executed before running the current example:
  * <ol>
  *     <li>
- *         Build "ignite-examples-x.y.z.jar" using the next command:<br>
+ *         Build the <code>ignite-examples-x.y.z.jar</code> file:<br>
  *         {@code ./gradlew :ignite-examples:jar}
  *     </li>
  *     <li>
- *         Create a new deployment unit using the CLI tool:<br>
- *         {@code cluster unit deploy computeExampleUnit \
- *          --version 1.0.0 \
- *          --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar}
+ *         Deploy the generated JAR as a deployment unit using the CLI:<br>
+ *         {@code
+ *         cluster unit deploy computeExampleUnit \
+ *         --version 1.0.0 \
+ *         --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar}
  *     </li>
  * </ol>
  */
-public class ComputeCancellationExample {
+
+public class ComputeCancellationExample  extends AbstractDeploymentUnitExample {
     /** Deployment unit name. */
     private static final String DEPLOYMENT_UNIT_NAME = "computeExampleUnit";
 
     /** Deployment unit version. */
     private static final String DEPLOYMENT_UNIT_VERSION = "1.0.0";
+    private static final Path JAR_PATH = Path.of("build/libs/codeDeploymentExampleUnit-1.0.0.jar"); // Output jar
 
     /**
      * Main method of the example.
      *
      * @param args The command line arguments.
+     * @throws Exception if any error occurs.
      */
-    public static void main(String[] args) {
+    public static void main(String[] args) throws Exception {
+
+        processDeploymentUnit(args);

Review Comment:
   Here and in all other examples where the specific JAR_PATH is used. The example does not work because this method builds a deployment JAR to the default path instead of the  `JAR_PATH` specified in the example.



##########
examples/java/src/main/java/org/apache/ignite/example/serialization/SerializationExample.java:
##########
@@ -17,19 +17,145 @@
 
 package org.apache.ignite.example.serialization;
 
+import static org.apache.ignite.example.util.DeployComputeUnit.deployUnit;
+import static org.apache.ignite.example.util.DeployComputeUnit.deploymentExists;
+import static org.apache.ignite.example.util.DeployComputeUnit.undeployUnit;
+
+import java.nio.file.Path;
+import java.nio.file.Paths;
 import org.apache.ignite.client.IgniteClient;
+import org.apache.ignite.example.code.deployment.AbstractDeploymentUnitExample;
+import org.apache.ignite.example.util.DeployComputeUnit;
 
-public class SerializationExample {
+/**
+ * This example demonstrates the usage of the { @link IgniteCompute#executeAsync(JobTarget, JobDescriptor, Object)} API.
+ *
+ * <p>Find instructions on how to run the example in the {@code README.md}
+ * file located in the {@code examples} directory root.</p>
+ *
+ * <h2>Execution Modes</h2>
+ *
+ * <p>There are two modes of execution:</p>
+ *
+ * <h3>1. Automated : The JAR Deployment for  deployment unit is automated </h3>
+ *
+ * <h4>1.1 With IDE</h4>
+ * <ul>
+ *   <li>
+ *     <b>Run from an IDE</b><br>
+ *     Launch the example directly from the IDE. If the required deployment
+ *     unit is not present, the example automatically builds and deploys the
+ *     necessary JAR.
+ *   </li>
+ * </ul>
+ *
+ * <h3>1.2 Without IDE</h3>
+ * <ul>
+ *   <li>
+ *     <b>Run from the command line</b><br>
+ *     Start the example using a Java command where the classpath includes
+ *     all required dependencies:
+ *
+ *     <pre>{@code
+ * java -cp "{user.home}\\.m2\\repository\\org\\apache\\ignite\\ignite-core\\3.1.0-SNAPSHOT\\
+ * ignite-core-3.1.0-SNAPSHOT.jar{other required jars}"
+ * <example-main-class> runFromIDE=false jarPath="{path-to-examples-jar}"
+ *     }</pre>
+ *
+ *     In this mode, {@code runFromIDE=false} indicates command-line execution,
+ *     and {@code jarPath} must reference the examples JAR used as the
+ *     deployment unit.
+ *   </li>
+ * </ul>
+ *
+ * <h2>2. Manual (with IDE): The JAR deployment for the deployment unit is manual</h2>
+ *
+ * <p>Before running this example, complete the following steps related to
+ * code deployment:</p>
+ *
+ * <ol>
+ *   <li>
+ *     Build the {@code ignite-examples-x.y.z.jar} file:<br>
+ *     {@code ./gradlew :ignite-examples:jar}
+ *   </li>
+ *   <li>
+ *     Deploy the generated JAR as a deployment unit using the CLI:<br>
+ *     <pre>{@code
+ * cluster unit deploy computeExampleUnit \
+ *     --version 1.0.0 \
+ *     --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar
+ *     }</pre>
+ *   </li>
+ * </ol>
+ */
+
+public class SerializationExample extends AbstractDeploymentUnitExample {
+
+    private static final String DEPLOYMENT_UNIT_NATIVE = "nativeSerializationExampleUnit";
+    private static final String DEPLOYMENT_UNIT_CUSTOM = "customPojoSerializationExampleUnit";
+    private static final String DEPLOYMENT_UNIT_AUTO = "pojoAutoSerializationExampleUnit";
+    private static final String DEPLOYMENT_UNIT_TUPLE = "tupleSerializationExampleUnit";
+    private static final String VERSION = "1.0.0";
+    private static final Path JAR_PATH = Path.of("build/libs/serialization-example-1.0.0.jar"); // Output jar
 
     public static void main(String[] args) throws Exception {
         try (IgniteClient client = IgniteClient.builder()
                 .addresses("127.0.0.1:10800")
                 .build()) {
 
+            processDeploymentUnit(args);

Review Comment:
   This example does not work because the `processDeploymentUnit()` builds a JAR to the default path (`build/libs/deploymentunit-example-1.0.0.jar`), not to the path used by the example (`build/libs/serialization-example-1.0.0.jar`).



##########
examples/java/src/main/java/org/apache/ignite/example/serialization/AutoSerializableArg.java:
##########
@@ -13,21 +13,34 @@
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  * See the License for the specific language governing permissions and
  * limitations under the License.
+ *

Review Comment:
   This line should not be added.



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {
+
+    // Root path of ignite-examples/
+    protected static final Path projectRoot =
+            Paths.get("").toAbsolutePath();
+
+    // Compiled class output when running from IDE/CLI
+    protected static final Path DEFAULT_CLASSES_DIR =
+            projectRoot.resolve("examples/java/build/classes/java/main");
+
+    // Default JAR output
+    protected static final Path DEFAULT_JAR_PATH =
+            Path.of("build/libs/deploymentunit-example-1.0.0.jar");
+
+    protected static volatile String jarPathAsString = "";

Review Comment:
   The `volatile` keyword is not needed because we do not execute examples in parallel using more than one thread.



##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeBroadcastExample.java:
##########
@@ -28,43 +36,89 @@
 import org.apache.ignite.compute.JobDescriptor;
 import org.apache.ignite.compute.JobExecutionContext;
 import org.apache.ignite.deployment.DeploymentUnit;
+import org.apache.ignite.example.code.deployment.AbstractDeploymentUnitExample;
 import org.apache.ignite.table.QualifiedName;
 
 /**
- * This example demonstrates the usage of the
- * {@link IgniteCompute#execute(BroadcastJobTarget, JobDescriptor, Object)} API.
+ * This example demonstrates the usage of the {@link IgniteCompute#execute(BroadcastJobTarget, JobDescriptor, Object)} API.
+ *
+ *
+ * <p>Find instructions on how to run the example in the {@code README.md}
+ * file located in the {@code examples} directory root.</p>
+ *
+ * <h2>Execution Modes</h2>
+ *
+ * <p>There are two modes of execution:</p>
+ *
+ * <h3>1. Automated : The JAR Deployment for  deployment unit is automated </h3>
  *
- * <p>Find instructions on how to run the example in the README.md file located in the "examples" directory root.
+ * <h4>1.1 With IDE</h4>
+ * <ul>
+ *   <li>
+ *     <b>Run from an IDE</b><br>
+ *     Launch the example directly from the IDE. If the required deployment
+ *     unit is not present, the example automatically builds and deploys the
+ *     necessary JAR.
+ *   </li>
+ * </ul>
  *
- * <p>This example is intended to be run on a cluster with more than one node to show that the job is broadcast to each node.
+ * <h3>1.2 Without IDE</h3>
+ * <ul>
+ *   <li>
+ *     <b>Run from the command line</b><br>
+ *     Start the example using a Java command where the classpath includes
+ *     all required dependencies:
+ *
+ *     <pre>{@code
+ * java -cp "{user.home}\\.m2\\repository\\org\\apache\\ignite\\ignite-core\\3.1.0-SNAPSHOT\\
+ * ignite-core-3.1.0-SNAPSHOT.jar{other required jars}"
+ * <example-main-class> runFromIDE=false jarPath="{path-to-examples-jar}"
+ *     }</pre>
+ *
+ *     In this mode, {@code runFromIDE=false} indicates command-line execution,
+ *     and {@code jarPath} must reference the examples JAR used as the
+ *     deployment unit.
+ *   </li>
+ * </ul>
+ *
+ * <h2>2. Manual (with IDE): The JAR deployment for the deployment unit is manual</h2>
+ *
+ * <p>Before running this example, complete the following steps related to
+ * code deployment:</p>
  *
- * <p>The following steps related to code deployment should be additionally executed before running the current example:
  * <ol>
- *     <li>
- *         Build "ignite-examples-x.y.z.jar" using the next command:<br>
- *         {@code ./gradlew :ignite-examples:jar}
- *     </li>
- *     <li>
- *         Create a new deployment unit using the CLI tool:<br>
- *         {@code cluster unit deploy computeExampleUnit \
- *          --version 1.0.0 \
- *          --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar}
- *     </li>
+ *   <li>
+ *     Build the {@code ignite-examples-x.y.z.jar} file:<br>
+ *     {@code ./gradlew :ignite-examples:jar}
+ *   </li>
+ *   <li>
+ *     Deploy the generated JAR as a deployment unit using the CLI:<br>
+ *     <pre>{@code
+ * cluster unit deploy computeExampleUnit \
+ *     --version 1.0.0 \
+ *     --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar
+ *     }</pre>
+ *   </li>
  * </ol>
  */
-public class ComputeBroadcastExample {
+public class ComputeBroadcastExample extends AbstractDeploymentUnitExample {
     /** Deployment unit name. */
     private static final String DEPLOYMENT_UNIT_NAME = "computeExampleUnit";
 
     /** Deployment unit version. */
     private static final String DEPLOYMENT_UNIT_VERSION = "1.0.0";
+    private static final Path JAR_PATH = Path.of("build/libs/codeDeploymentExampleUnit-1.0.0.jar"); // Output jar
 
     /**
      * Main method of the example.
      *
      * @param args The command line arguments.
+     * @throws Exception if any error occurs.
      */
-    public static void main(String[] args) {
+    public static void main(String[] args) throws Exception {
+
+        processDeploymentUnit(args);

Review Comment:
   The example does not work because this method builds a deployment JAR to the default path instead of the  `JAR_PATH` specified in the example.
   
   ```
   Caused by: java.nio.file.NoSuchFileException: build/libs/codeDeploymentExampleUnit-1.0.0.jar
   ```



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/MyJob.java:
##########
@@ -68,4 +67,4 @@ public CompletableFuture<String> executeAsync(JobExecutionContext ctx, String ar
             throw new RuntimeException("Failed to run script", e);
         }
     }
-}
+}

Review Comment:
   Let's revert the file completely.



##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeAsyncExample.java:
##########
@@ -32,40 +37,89 @@
 import org.apache.ignite.compute.JobExecutionContext;
 import org.apache.ignite.compute.JobTarget;
 import org.apache.ignite.deployment.DeploymentUnit;
+import org.apache.ignite.example.code.deployment.AbstractDeploymentUnitExample;
 
 /**
- * This example demonstrates the usage of the
- * {@link IgniteCompute#executeAsync(JobTarget, JobDescriptor, Object)} API.
+ * This example demonstrates the usage of the {@link IgniteCompute#executeAsync(JobTarget, JobDescriptor, Object)} API.
  *
- * <p>Find instructions on how to run the example in the README.md file located in the "examples" directory root.
  *
- * <p>The following steps related to code deployment should be additionally executed before running the current example:
+ * <p>Find instructions on how to run the example in the {@code README.md}
+ * file located in the {@code examples} directory root.</p>
+ *
+ * <h2>Execution Modes</h2>
+ *
+ * <p>There are two modes of execution:</p>
+ *
+ * <h3>1. Automated : The JAR Deployment for  deployment unit is automated </h3>
+ *
+ * <h4>1.1 With IDE</h4>
+ * <ul>
+ *   <li>
+ *     <b>Run from an IDE</b><br>
+ *     Launch the example directly from the IDE. If the required deployment
+ *     unit is not present, the example automatically builds and deploys the
+ *     necessary JAR.
+ *   </li>
+ * </ul>
+ *
+ * <h3>1.2 Without IDE</h3>
+ * <ul>
+ *   <li>
+ *     <b>Run from the command line</b><br>
+ *     Start the example using a Java command where the classpath includes
+ *     all required dependencies:
+ *
+ *     <pre>{@code
+ * java -cp "{user.home}\\.m2\\repository\\org\\apache\\ignite\\ignite-core\\3.1.0-SNAPSHOT\\
+ * ignite-core-3.1.0-SNAPSHOT.jar{other required jars}"
+ * <example-main-class> runFromIDE=false jarPath="{path-to-examples-jar}"
+ *     }</pre>
+ *
+ *     In this mode, {@code runFromIDE=false} indicates command-line execution,
+ *     and {@code jarPath} must reference the examples JAR used as the
+ *     deployment unit.
+ *   </li>
+ * </ul>
+ *
+ * <h2>2. Manual (with IDE): The JAR deployment for the deployment unit is manual</h2>
+ *
+ * <p>Before running this example, complete the following steps related to
+ * code deployment:</p>
+ *
  * <ol>
- *     <li>
- *         Build "ignite-examples-x.y.z.jar" using the next command:<br>
- *         {@code ./gradlew :ignite-examples:jar}
- *     </li>
- *     <li>
- *         Create a new deployment unit using the CLI tool:<br>
- *         {@code cluster unit deploy computeExampleUnit \
- *          --version 1.0.0 \
- *          --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar}
- *     </li>
+ *   <li>
+ *     Build the {@code ignite-examples-x.y.z.jar} file:<br>
+ *     {@code ./gradlew :ignite-examples:jar}
+ *   </li>
+ *   <li>
+ *     Deploy the generated JAR as a deployment unit using the CLI:<br>
+ *     <pre>{@code
+ * cluster unit deploy computeExampleUnit \
+ *     --version 1.0.0 \
+ *     --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar
+ *     }</pre>
+ *   </li>
  * </ol>
  */
-public class ComputeAsyncExample {
+
+public class ComputeAsyncExample extends AbstractDeploymentUnitExample {
     /** Deployment unit name. */
     private static final String DEPLOYMENT_UNIT_NAME = "computeExampleUnit";
 
     /** Deployment unit version. */
     private static final String DEPLOYMENT_UNIT_VERSION = "1.0.0";
+    private static final Path JAR_PATH = Path.of("build/libs/computeExampleUnit-1.0.0.jar"); // Output jar
 
     /**
      * Main method of the example.
      *
      * @param args The command line arguments.
+     * @throws Exception if any error occurs.
      */
-    public static void main(String[] args) throws ExecutionException, InterruptedException {
+    public static void main(String[] args) throws Exception {
+
+        processDeploymentUnit(args);

Review Comment:
   The example does not work because this method builds a deployment JAR to the default path instead of the  `JAR_PATH` specified in the example. Hence the following error arises:
   
   ```
   Caused by: java.nio.file.NoSuchFileException: build/libs/computeExampleUnit-1.0.0.jar
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "iartiukhov (via GitHub)" <gi...@apache.org>.
iartiukhov commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2642353873


##########
examples/java/src/main/java/org/apache/ignite/example/table/RecordViewPojoExample.java:
##########
@@ -42,81 +39,73 @@ public static void main(String[] args) throws Exception {
         //
         //--------------------------------------------------------------------------------------
 
-        try (
-                Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                Statement stmt = conn.createStatement()
-        ) {
-            stmt.executeUpdate(
-                    "CREATE TABLE accounts ("
-                            + "accountNumber INT PRIMARY KEY,"
-                            + "firstName     VARCHAR,"
-                            + "lastName      VARCHAR,"
-                            + "balance       DOUBLE)"
-            );
-        }
-
-        //--------------------------------------------------------------------------------------
-        //
-        // Creating a client to connect to the cluster.
-        //
-        //--------------------------------------------------------------------------------------
-
-        System.out.println("\nConnecting to server...");
-
         try (IgniteClient client = IgniteClient.builder()
                 .addresses("127.0.0.1:10800")
                 .build()
         ) {
-            //--------------------------------------------------------------------------------------
-            //
-            // Creating a record view for the 'accounts' table.
-            //
-            //--------------------------------------------------------------------------------------
-
-            RecordView<Account> accounts = client.tables()
-                    .table("accounts")
-                    .recordView(Account.class);
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'insert' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nInserting a record into the 'accounts' table...");
-
-            Account newAccount = new Account(
-                    123456,
-                    "Val",
-                    "Kulichenko",
-                    100.00d
-            );
-
-            accounts.insert(null, newAccount);
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'get' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nRetrieving a record using RecordView API...");
-
-            Account account = accounts.get(null, new Account(123456));
-
-            System.out.println(
-                    "\nRetrieved record:\n"
-                        + "    Account Number: " + account.accountNumber + '\n'
-                        + "    Owner: " + account.firstName + " " + account.lastName + '\n'
-                        + "    Balance: $" + account.balance);
-        } finally {
-            System.out.println("\nDropping the table...");
-
-            try (
-                    Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                    Statement stmt = conn.createStatement()
-            ) {
-                stmt.executeUpdate("DROP TABLE accounts");
+            try {

Review Comment:
   Please do not change unrelated examples. This example should be reverted. If you want to overcome the broken compatibility issue (https://issues.apache.org/jira/browse/IGNITE-27280) then please use the latest Ignite 3 build from `main`.



##########
examples/java/src/main/java/org/apache/ignite/example/table/KeyValueViewPojoExample.java:
##########
@@ -42,82 +39,75 @@ public static void main(String[] args) throws Exception {
         //
         //--------------------------------------------------------------------------------------
 
-        try (
-                Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                Statement stmt = conn.createStatement()
-        ) {
-            stmt.executeUpdate(
-                    "CREATE TABLE accounts ("
-                            + "accountNumber INT PRIMARY KEY,"
-                            + "firstName     VARCHAR,"
-                            + "lastName      VARCHAR,"
-                            + "balance       DOUBLE)"
-            );
-        }
-
-        //--------------------------------------------------------------------------------------
-        //
-        // Creating a client to connect to the cluster.
-        //
-        //--------------------------------------------------------------------------------------
-
-        System.out.println("\nConnecting to server...");
-
         try (IgniteClient client = IgniteClient.builder()
                 .addresses("127.0.0.1:10800")
                 .build()
         ) {
-            //--------------------------------------------------------------------------------------
-            //
-            // Creating a key-value view for the 'accounts' table.
-            //
-            //--------------------------------------------------------------------------------------
-
-            KeyValueView<AccountKey, Account> kvView = client.tables()
-                    .table("accounts")
-                    .keyValueView(AccountKey.class, Account.class);
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'put' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nInserting a key-value pair into the 'accounts' table...");
-
-            AccountKey key = new AccountKey(123456);
-
-            Account value = new Account(
-                    "Val",
-                    "Kulichenko",
-                    100.00d
-            );
-
-            kvView.put(null, key, value);
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'get' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nRetrieving a value using KeyValueView API...");
-
-            value = kvView.get(null, key);
-
-            System.out.println(
-                    "\nRetrieved value:\n"
-                        + "    Account Number: " + key.accountNumber + '\n'
-                        + "    Owner: " + value.firstName + " " + value.lastName + '\n'
-                        + "    Balance: $" + value.balance);
-        } finally {
-            System.out.println("\nDropping the table...");
-
-            try (
-                    Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                    Statement stmt = conn.createStatement()
-            ) {
-                stmt.executeUpdate("DROP TABLE accounts");
+            try {

Review Comment:
   Please do not change unrelated examples. This example should be reverted. If you want to overcome the broken compatibility issue (https://issues.apache.org/jira/browse/IGNITE-27280) then please use the latest Ignite 3 build from `main`.



##########
examples/java/src/main/java/org/apache/ignite/example/table/KeyValueViewExample.java:
##########
@@ -43,80 +40,73 @@ public static void main(String[] args) throws Exception {
         //
         //--------------------------------------------------------------------------------------
 
-        try (
-                Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                Statement stmt = conn.createStatement()
-        ) {
-            stmt.executeUpdate(
-                    "CREATE TABLE accounts ("
-                            + "accountNumber INT PRIMARY KEY,"
-                            + "firstName     VARCHAR,"
-                            + "lastName      VARCHAR,"
-                            + "balance       DOUBLE)"
-            );
-        }
-
-        //--------------------------------------------------------------------------------------
-        //
-        // Creating a client to connect to the cluster.
-        //
-        //--------------------------------------------------------------------------------------
-
-        System.out.println("\nConnecting to server...");
-
         try (IgniteClient client = IgniteClient.builder()
                 .addresses("127.0.0.1:10800")
                 .build()
         ) {
-            //--------------------------------------------------------------------------------------
-            //
-            // Creating a key-value view for the 'accounts' table.
-            //
-            //--------------------------------------------------------------------------------------
-
-            KeyValueView<Tuple, Tuple> kvView = client.tables().table("accounts").keyValueView();
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'put' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nInserting a key-value pair into the 'accounts' table...");
-
-            Tuple key = Tuple.create()
-                    .set("accountNumber", 123456);
-
-            Tuple value = Tuple.create()
-                    .set("firstName", "Val")
-                    .set("lastName", "Kulichenko")
-                    .set("balance", 100.00d);
-
-            kvView.put(null, key, value);
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'get' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nRetrieving a value using KeyValueView API...");
-
-            value = kvView.get(null, key);
-
-            System.out.println(
-                    "\nRetrieved value:\n"
-                            + "    Account Number: " + key.intValue("accountNumber") + '\n'
-                            + "    Owner: " + value.stringValue("firstName") + " " + value.stringValue("lastName") + '\n'
-                            + "    Balance: $" + value.doubleValue("balance"));
-        } finally {
-            System.out.println("\nDropping the table...");
-
-            try (
-                    Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                    Statement stmt = conn.createStatement()
-            ) {
-                stmt.executeUpdate("DROP TABLE accounts");
+            try {

Review Comment:
   Please do not change unrelated examples. This example should be reverted. If you want to overcome the broken compatibility issue (https://issues.apache.org/jira/browse/IGNITE-27280) then please use the latest Ignite 3 build from `main`.



##########
examples/java/src/main/java/org/apache/ignite/example/table/RecordViewExample.java:
##########
@@ -43,80 +40,72 @@ public static void main(String[] args) throws Exception {
         //
         //--------------------------------------------------------------------------------------
 
-        try (
-                Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                Statement stmt = conn.createStatement()
-        ) {
-            stmt.executeUpdate(
-                    "CREATE TABLE accounts ("
-                            + "accountNumber INT PRIMARY KEY,"
-                            + "firstName     VARCHAR,"
-                            + "lastName      VARCHAR,"
-                            + "balance       DOUBLE)"
-            );
-        }
-
-        //--------------------------------------------------------------------------------------
-        //
-        // Creating a client to connect to the cluster.
-        //
-        //--------------------------------------------------------------------------------------
-
-        System.out.println("\nConnecting to server...");
-
         try (IgniteClient client = IgniteClient.builder()
                 .addresses("127.0.0.1:10800")
                 .build()
         ) {
-            //--------------------------------------------------------------------------------------
-            //
-            // Creating a record view for the 'accounts' table.
-            //
-            //--------------------------------------------------------------------------------------
-
-            RecordView<Tuple> accounts = client.tables().table("accounts").recordView();
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'insert' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nInserting a record into the 'accounts' table...");
-
-            Tuple newAccountTuple = Tuple.create()
-                    .set("accountNumber", 123456)
-                    .set("firstName", "Val")
-                    .set("lastName", "Kulichenko")
-                    .set("balance", 100.00d);
-
-            accounts.insert(null, newAccountTuple);
-
-            //--------------------------------------------------------------------------------------
-            //
-            // Performing the 'get' operation.
-            //
-            //--------------------------------------------------------------------------------------
-
-            System.out.println("\nRetrieving a record using RecordView API...");
-
-            Tuple accountNumberTuple = Tuple.create().set("accountNumber", 123456);
-
-            Tuple accountTuple = accounts.get(null, accountNumberTuple);
-
-            System.out.println(
-                    "\nRetrieved record:\n"
-                            + "    Account Number: " + accountTuple.intValue("accountNumber") + '\n'
-                            + "    Owner: " + accountTuple.stringValue("firstName") + " " + accountTuple.stringValue("lastName") + '\n'
-                            + "    Balance: $" + accountTuple.doubleValue("balance"));
-        } finally {
-            System.out.println("\nDropping the table...");
-
-            try (
-                    Connection conn = DriverManager.getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
-                    Statement stmt = conn.createStatement()
-            ) {
-                stmt.executeUpdate("DROP TABLE accounts");
+            try {

Review Comment:
   Please do not change unrelated examples. This example should be reverted. If you want to overcome the broken compatibility issue (https://issues.apache.org/jira/browse/IGNITE-27280) then please use the latest Ignite 3 build from `main`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashish-zs (via GitHub)" <gi...@apache.org>.
ashish-zs commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2642706625


##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,263 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+/**
+ * Utility class for building and deploying Ignite compute units.
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+
+    /**
+     * Builds a JAR file by packaging all compiled classes present in the given directory.
+     *
+     * @param classesDir Directory containing compiled .class files.
+     * @param jarPath Target JAR file path to create.
+     * @throws IOException If building the JAR fails.
+     */
+    public static void buildJar(Path classesDir, Path jarPath) throws IOException {
+        if (!Files.exists(classesDir)) {
+            throw new IllegalArgumentException("Compiled classes not found: " + classesDir);
+        }
+
+        Files.createDirectories(jarPath.getParent());
+
+        try (OutputStream fos = Files.newOutputStream(jarPath);
+                JarOutputStream jar = new JarOutputStream(fos, createManifest())) {
+
+            Files.walk(classesDir).filter(Files::isRegularFile).forEach(path -> {
+                String entry = classesDir.relativize(path).toString().replace("\\", "/");
+                try (InputStream is = Files.newInputStream(path)) {
+                    jar.putNextEntry(new JarEntry(entry));
+                    is.transferTo(jar);
+                    jar.closeEntry();
+                } catch (Exception e) {
+                    throw new RuntimeException(e);
+                }
+            });
+        }
+
+        System.out.println("JAR built: " + jarPath);
+    }
+
+    /**
+     * Creates a simple manifest declaring manifest version.
+     *
+     * @return Manifest object.
+     */
+    private static Manifest createManifest() {
+        Manifest m = new Manifest();
+        m.getMainAttributes().putValue("Manifest-Version", "1.0");
+        return m;
+    }
+
+    /**
+     * Deploys a unit only if it is not already deployed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnitIfNeeded(String unitId, String version, Path jar) throws Exception {
+        if (deploymentExists(unitId, version)) {
+            System.out.println("Deployment unit already active. Skipping deployment.");
+            return;
+        }
+        deployUnit(unitId, version, jar);
+        System.out.println("Deployment completed.");
+    }
+
+    /**
+     * Checks if a deployment unit already exists on the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if active deployment exists.
+     * @throws Exception If request fails.
+     */
+    public static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET().build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        Thread.sleep(500);

Review Comment:
   As mentioned earlier, I plan to revise this to replace the sleep with a polling approach that waits for the deployment unit to reach the DEPLOYED state



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2599234654


##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeAsyncExample.java:
##########
@@ -86,6 +140,15 @@ public static void main(String[] args) throws ExecutionException, InterruptedExc
 
             System.out.println("\nConfiguring compute job...");
 
+            // 1) Check if deployment unit already exists

Review Comment:
   Removing all comments as previously advised



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2599233761


##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeBroadcastExample.java:
##########
@@ -83,8 +137,40 @@ public static void main(String[] args) {
             //
             //--------------------------------------------------------------------------------------
 
+            try (
+                    Connection conn = getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
+                    Statement stmt = conn.createStatement()
+            ) {
+                stmt.execute("CREATE SCHEMA IF NOT EXISTS CUSTOM_SCHEMA");
+                stmt.execute("CREATE TABLE IF NOT EXISTS CUSTOM_SCHEMA.MY_QUALIFIED_TABLE (" +
+                        "ID INT PRIMARY KEY, MESSAGE VARCHAR(255))");
+
+                stmt.execute("CREATE SCHEMA IF NOT EXISTS PUBLIC");
+                stmt.execute("CREATE TABLE IF NOT EXISTS PUBLIC.MY_TABLE (" +
+                        "ID INT PRIMARY KEY, MESSAGE VARCHAR(255))");
+
+                stmt.execute("CREATE TABLE IF NOT EXISTS PERSON (" +
+                        "ID INT PRIMARY KEY, FIRST_NAME VARCHAR(100)," +
+                        "LAST_NAME VARCHAR(100), AGE INT)");
+
+                stmt.execute("INSERT INTO PERSON VALUES " +
+                        "(1, 'John', 'Doe', 36)," +
+                        "(2, 'Jane', 'Smith', 35)," +
+                        "(3, 'Robert', 'Johnson', 25)");
+
+            }
+
             System.out.println("\nConfiguring compute job...");
 
+            // 1) Check if deployment unit already exists

Review Comment:
   Removing all comments as previously advised



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashish-zs (via GitHub)" <gi...@apache.org>.
ashish-zs commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2642706625


##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,263 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+/**
+ * Utility class for building and deploying Ignite compute units.
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+
+    /**
+     * Builds a JAR file by packaging all compiled classes present in the given directory.
+     *
+     * @param classesDir Directory containing compiled .class files.
+     * @param jarPath Target JAR file path to create.
+     * @throws IOException If building the JAR fails.
+     */
+    public static void buildJar(Path classesDir, Path jarPath) throws IOException {
+        if (!Files.exists(classesDir)) {
+            throw new IllegalArgumentException("Compiled classes not found: " + classesDir);
+        }
+
+        Files.createDirectories(jarPath.getParent());
+
+        try (OutputStream fos = Files.newOutputStream(jarPath);
+                JarOutputStream jar = new JarOutputStream(fos, createManifest())) {
+
+            Files.walk(classesDir).filter(Files::isRegularFile).forEach(path -> {
+                String entry = classesDir.relativize(path).toString().replace("\\", "/");
+                try (InputStream is = Files.newInputStream(path)) {
+                    jar.putNextEntry(new JarEntry(entry));
+                    is.transferTo(jar);
+                    jar.closeEntry();
+                } catch (Exception e) {
+                    throw new RuntimeException(e);
+                }
+            });
+        }
+
+        System.out.println("JAR built: " + jarPath);
+    }
+
+    /**
+     * Creates a simple manifest declaring manifest version.
+     *
+     * @return Manifest object.
+     */
+    private static Manifest createManifest() {
+        Manifest m = new Manifest();
+        m.getMainAttributes().putValue("Manifest-Version", "1.0");
+        return m;
+    }
+
+    /**
+     * Deploys a unit only if it is not already deployed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnitIfNeeded(String unitId, String version, Path jar) throws Exception {
+        if (deploymentExists(unitId, version)) {
+            System.out.println("Deployment unit already active. Skipping deployment.");
+            return;
+        }
+        deployUnit(unitId, version, jar);
+        System.out.println("Deployment completed.");
+    }
+
+    /**
+     * Checks if a deployment unit already exists on the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if active deployment exists.
+     * @throws Exception If request fails.
+     */
+    public static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET().build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        Thread.sleep(500);

Review Comment:
   As mentioned earlier, I plan to revise this to replace the sleep with a polling approach that waits for the deployment unit to reach the DEPLOYED state
   
   I’ll use the existing approach in the tests (e.g. Awaitility) for consistency and to avoid fixed delays.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2812300004


##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,52 @@
+package org.apache.ignite.example.code.deployment;

Review Comment:
   Missing ASF license header. All source files in the project must include the Apache License header (this was already mentioned by @iartiukhov but appears to still be missing).



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,52 @@
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {

Review Comment:
   Using inheritance purely for sharing static utility methods is an anti-pattern. All examples extend this class only to access `jarPath` and `processDeploymentUnit()`, but these could simply be static methods/fields in `DeployComputeUnit` (which already exists as a utility class). This would remove the need for the entire class hierarchy.
   
   Suggestion: move `processDeploymentUnit()` and the JAR path logic into `DeployComputeUnit`, then delete this class and remove `extends AbstractDeploymentUnitExample` from all examples.



##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeAsyncExample.java:
##########
@@ -135,13 +188,21 @@ public static void main(String[] args) throws ExecutionException, InterruptedExc
             //--------------------------------------------------------------------------------------
 
             System.out.println("\nTotal number of characters in the words is '" + sum + "'.");
+        } catch (Exception e) {

Review Comment:
   This pattern is repeated in every single example and is redundant:
   ```java
   } catch (Exception e) {
       throw new RuntimeException(e);
   }
   ```
   Since `main` already declares `throws Exception`, the catch block is unnecessary — it only wraps a checked exception into an unchecked one for no benefit. The `finally` block alone would suffice.
   
   This applies to all modified examples.



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/CodeDeploymentExample.java:
##########
@@ -53,6 +119,11 @@ public static void main(String[] args) {
             String result = client.compute().execute(target, job, "Hello from job");
 
             System.out.println("\n=== Result ===\n" + result);
+        }catch (Exception e) {

Review Comment:
   Missing space before `catch`:
   ```java
   }catch (Exception e) {
   ```
   Should be:
   ```java
   } catch (Exception e) {
   ```



##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,284 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.awaitility.Awaitility.await;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Utility class for deploying Ignite compute units.
+ * <p>
+ * Note: The deployment unit JAR is now built at compile time via the deploymentUnitJar Gradle task,
+ * not at runtime. This eliminates the need for runtime JAR building.
+ * </p>
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
+    private static final int DEPLOYMENT_TIMEOUT_SECONDS = 30;
+
+
+    /**
+     * Checks if a deployment unit already exists on the cluster with DEPLOYED status.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if deployment exists with DEPLOYED status.
+     * @throws Exception If request fails.
+     */
+    private static boolean isDeployed(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET()
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        System.out.println("[DEBUG] Checking deployment status for " + unitId + " version " + version);

Review Comment:
   There are ~15 `System.out.println("[DEBUG]")` statements throughout `isDeployed()` (lines 66-68, 72, 83-84, 88, 99, 101, 108, 111, 118). These are clearly leftover debug prints and should be removed before merging. If logging is needed, consider using a proper logger or at least a consistent non-debug format.



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,52 @@
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {
+
+    // Root path of the project
+    private static final Path projectRoot = Paths.get("").toAbsolutePath();
+
+    // Pre-built JAR from deploymentUnitJar task (built at compile time)
+    private static final Path DEFAULT_JAR_PATH =
+            projectRoot.resolve("examples/java/build/libs/deploymentunit-example-1.0.0.jar");
+
+    protected static String jarPathAsString = "";

Review Comment:
   Non-final `protected static` mutable fields exposed via inheritance are fragile. Any subclass can modify them, and the state is shared across all examples. Consider encapsulating these in a return object from `processDeploymentUnit()` or making them private with accessor methods.



##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,284 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.awaitility.Awaitility.await;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Utility class for deploying Ignite compute units.
+ * <p>
+ * Note: The deployment unit JAR is now built at compile time via the deploymentUnitJar Gradle task,
+ * not at runtime. This eliminates the need for runtime JAR building.
+ * </p>
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
+    private static final int DEPLOYMENT_TIMEOUT_SECONDS = 30;
+
+
+    /**
+     * Checks if a deployment unit already exists on the cluster with DEPLOYED status.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if deployment exists with DEPLOYED status.
+     * @throws Exception If request fails.
+     */
+    private static boolean isDeployed(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET()
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        System.out.println("[DEBUG] Checking deployment status for " + unitId + " version " + version);
+        System.out.println("[DEBUG] HTTP Status: " + resp.statusCode());
+        System.out.println("[DEBUG] Response body: " + resp.body());
+
+        if (resp.statusCode() == 404) {
+            // Unit doesn't exist yet
+            System.out.println("[DEBUG] Unit not found (404)");
+            return false;
+        }
+
+        if (resp.statusCode() != 200) {
+            throw new RuntimeException("Failed to check deployment status. HTTP " + resp.statusCode() + ": " + resp.body());
+        }
+
+        // Parse JSON response - the API returns a Collection<UnitStatus>
+        JsonNode root = OBJECT_MAPPER.readTree(resp.body());
+
+        System.out.println("[DEBUG] Parsed JSON root: " + root);
+        System.out.println("[DEBUG] Is array: " + root.isArray() + ", Is object: " + root.isObject());
+
+        // Handle empty response (unit exists but no matching status)
+        if (root.isArray() && root.isEmpty()) {
+            System.out.println("[DEBUG] Empty array response");
+            return false;
+        }
+
+        // The response is an array of UnitStatus objects
+        if (!root.isArray()) {
+            throw new RuntimeException("Unexpected response format. Expected array, got: " + root);
+        }
+
+        // Check if any node has this version deployed
+        for (JsonNode unitStatus : root) {
+            System.out.println("[DEBUG] Processing UnitStatus: " + unitStatus);
+            JsonNode versionToStatus = unitStatus.path("versionToStatus");
+            System.out.println("[DEBUG] versionToStatus: " + versionToStatus);
+
+            if (versionToStatus.isArray()) {
+                for (JsonNode versionStatus : versionToStatus) {
+                    String versionValue = versionStatus.path("version").asText();
+                    String statusValue = versionStatus.path("status").asText();
+
+                    System.out.println("[DEBUG] Found version: " + versionValue + ", status: " + statusValue);
+
+                    if (version.equals(versionValue) && "DEPLOYED".equals(statusValue)) {
+                        System.out.println("[DEBUG] MATCH FOUND - Deployment is ready!");
+                        return true;
+                    }
+                }
+            }
+        }
+
+        System.out.println("[DEBUG] No matching DEPLOYED status found");
+        return false;
+    }
+
+    /**
+     * Checks if a deployment unit exists (in any state).
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if deployment exists.
+     * @throws Exception If request fails.
+     */
+    private static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET()
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster and waits for it to reach DEPLOYED status.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    private static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        if (resp.statusCode() != 200 && resp.statusCode() != 409) {
+            throw new RuntimeException("Deployment failed: " + resp.statusCode() + "\n" + resp.body());
+        }
+
+        // Wait for deployment to reach DEPLOYED status using polling
+        await()
+                .atMost(DEPLOYMENT_TIMEOUT_SECONDS, SECONDS)
+                .pollInterval(200, MILLISECONDS)
+                .until(() -> isDeployed(unitId, version));
+    }
+
+    /**
+     * Undeploys the given deployment unit from the cluster and waits for it to be removed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @throws Exception If undeployment fails.
+     */
+    public static void undeployUnit(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version))
+                .DELETE()
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        if (resp.statusCode() != 200 && resp.statusCode() != 404) {
+            throw new RuntimeException("Undeploy failed: " + resp.statusCode() + "\n" + resp.body());
+        }
+
+        // Wait for deployment to be removed using polling
+        await()
+                .atMost(DEPLOYMENT_TIMEOUT_SECONDS, SECONDS)
+                .pollInterval(200, MILLISECONDS)
+                .until(() -> !deploymentExists(unitId, version));
+
+        System.out.println("Unit successfully undeployed.");
+    }
+
+    /**
+     * Processes command-line arguments for:
+     * <ul>
+     *   <li><b>runFromIDE</b> – whether the example runs from source</li>
+     *   <li><b>jarPath</b> – path to external JAR when run outside IDE</li>
+     * </ul>
+     *
+     * @param args Command-line arguments (may be null).
+     * @return Map with keys "runFromIDE" and "jarPath".
+     */
+    public static Map<String, Object> processArguments(String[] args) {
+        Map<String, Object> response = new HashMap<>();
+
+        if (args == null) {
+            return response;
+        }
+
+        boolean runFromIDE = true;
+        String jarPath = null;
+
+        for (String arg : args) {
+
+            if (arg.contains("runFromIDE")) {
+                String[] splitArgArr = arg.split("=");
+                if (splitArgArr != null && splitArgArr.length == 2) {

Review Comment:
   `splitArgArr != null` check is redundant — `String.split()` never returns `null`.
   
   Also, `processArguments` returns `Map<String, Object>` which requires unsafe casts at the call site (`(boolean) p.get("runFromIDE")`). Consider returning a typed record/class instead:
   ```java
   record DeploymentArgs(boolean runFromIDE, String jarPath) {}
   ```



##########
examples/java/src/main/java/org/apache/ignite/example/streaming/DistributedComputeWithReceiverExample.java:
##########
@@ -30,98 +35,189 @@
 import org.apache.ignite.catalog.ColumnType;
 import org.apache.ignite.catalog.definitions.TableDefinition;
 import org.apache.ignite.client.IgniteClient;
+import org.apache.ignite.deployment.DeploymentUnit;
+import org.apache.ignite.example.code.deployment.AbstractDeploymentUnitExample;
 import org.apache.ignite.table.DataStreamerReceiver;
 import org.apache.ignite.table.DataStreamerReceiverContext;
 import org.apache.ignite.table.DataStreamerReceiverDescriptor;
 import org.apache.ignite.table.Table;
 import org.apache.ignite.table.Tuple;
 
-/** This example demonstrates how to use the streaming API to simulate a fraud detection process,
- * which typically involves intensive processing of each transaction using ML models.
+/**
+ * This example demonstrates how to use the streaming API to simulate a fraud detection process, which typically involves intensive
+ * processing of each transaction using ML models.
+ *
+ * <p>Find instructions on how to run the example in the {@code README.md}
+ * file located in the {@code examples} directory root.</p>
+ *
+ * <h2>Execution Modes</h2>
+ *
+ * <p>There are two modes of execution:</p>
+ *
+ * <h3>1. Automated : The JAR Deployment for  deployment unit is automated </h3>
+ *
+ * <h4>1.1 With IDE</h4>
+ * <ul>
+ *   <li>
+ *     <b>Run from an IDE</b><br>
+ *     Launch the example directly from the IDE. If the required deployment
+ *     unit is not present, the example automatically builds and deploys the
+ *     necessary JAR.
+ *   </li>
+ * </ul>
+ *
+ * <h3>1.2 Without IDE</h3>
+ * <ul>
+ *   <li>
+ *     <b>Run from the command line</b><br>
+ *     Start the example using a Java command where the classpath includes
+ *     all required dependencies:
+ *
+ *     <pre>{@code
+ * java -cp "{user.home}\\.m2\\repository\\org\\apache\\ignite\\ignite-core\\3.1.0-SNAPSHOT\\
+ * ignite-core-3.1.0-SNAPSHOT.jar{other required jars}"
+ * <example-main-class> runFromIDE=false jarPath="{path-to-examples-jar}"
+ *     }</pre>
+ *
+ *     In this mode, {@code runFromIDE=false} indicates command-line execution,
+ *     and {@code jarPath} must reference the examples JAR used as the
+ *     deployment unit.
+ *   </li>
+ * </ul>
+ *
+ * <h2>2. Manual (with IDE): The JAR deployment for the deployment unit is manual</h2>
+ *
+ * <p>Before running this example, complete the following steps related to
+ * code deployment:</p>
+ *
+ * <ol>
+ *   <li>
+ *     Build the {@code ignite-examples-x.y.z.jar} file:<br>
+ *     {@code ./gradlew :ignite-examples:jar}
+ *   </li>
+ *   <li>
+ *     Deploy the generated JAR as a deployment unit using the CLI:<br>
+ *     <pre>{@code
+ * cluster unit deploy computeExampleUnit \
+ *     --version 1.0.0 \
+ *     --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar
+ *     }</pre>
+ *   </li>
+ * </ol>
  */
 
-public class DistributedComputeWithReceiverExample {
 
-    public static void main(String[] arg) {
+public class DistributedComputeWithReceiverExample extends AbstractDeploymentUnitExample {
+
+    private static final String DEPLOYMENT_UNIT_NAME = "streamerReceiverExampleUnit";
+
+    /** Deployment unit version. */
+    private static final String DEPLOYMENT_UNIT_VERSION = "1.0.0";
+
+    public static void main(String[] arg) throws Exception {
+
+        processDeploymentUnit(arg);
 
         try (IgniteClient client = IgniteClient.builder()
                 .addresses("127.0.0.1:10800")
                 .build()) {
 
-        /* Source data is a list of financial transactions */
-        /* We distribute this processing across the cluster, then gather and return results */
-        List<Tuple> sourceData = IntStream.range(1, 10)
-                .mapToObj(i -> Tuple.create()
-                        .set("txId", i)
-                        .set("txData", "{some-json-data}"))
-                .collect(Collectors.toList());
+            deployIfNotExist(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION, jarPath);
 
-        DataStreamerReceiverDescriptor<Tuple, Void, Tuple> desc = DataStreamerReceiverDescriptor
-                .builder(FraudDetectorReceiver.class)
-                .build();
+            /* Source data is a list of financial transactions */
+            /* We distribute this processing across the cluster, then gather and return results */
+            List<Tuple> sourceData = IntStream.range(1, 10)
+                    .mapToObj(i -> Tuple.create()
+                            .set("txId", i)
+                            .set("txData", "{some-json-data}"))
+                    .collect(Collectors.toList());
 
-        CompletableFuture<Void> streamerFut;
+            DataStreamerReceiverDescriptor<Tuple, Void, Tuple> desc = DataStreamerReceiverDescriptor
+                    .builder(FraudDetectorReceiver.class)
+                    .units(new DeploymentUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION))
+                    .build();
+
+            CompletableFuture<Void> streamerFut;
 
         /* Streaming requires a target table to partition data.
         /* Use a dummy table for this scenario, because we are not going to store any data */
-        TableDefinition txDummyTableDef = TableDefinition.builder("tx_dummy")
-                .columns(column("id", ColumnType.INTEGER))
-                .primaryKey("id")
-                .build();
+            TableDefinition txDummyTableDef = TableDefinition.builder("tx_dummy")
+                    .columns(column("id", ColumnType.INTEGER))
+                    .primaryKey("id")
+                    .build();
+
+            Table dummyTable = client.catalog().createTable(txDummyTableDef);
 
-        Table dummyTable = client.catalog().createTable(txDummyTableDef);
 
-        /* Source data has "txId" field, but target dummy table has "id" column, so keyFunc maps "txId" to "id" */
-        Function<Tuple, Tuple> keyFunc = sourceItem -> Tuple.create().set("id", sourceItem.value("txId"));
+            /* Source data has "txId" field, but target dummy table has "id" column, so keyFunc maps "txId" to "id" */
+            Function<Tuple, Tuple> keyFunc = sourceItem -> Tuple.create().set("id", sourceItem.value("txId"));
 
         /* Payload function is used to extract the payload (data that goes to the receiver) from the source item.
         /* In our case, we want to use the whole source item as the payload */
-        Function<Tuple, Tuple> payloadFunc = Function.identity();
-
-        Flow.Subscriber<Tuple> resultSubscriber = new Flow.Subscriber<>() {
-            @Override
-            public void onSubscribe(Flow.Subscription subscription) {
-                subscription.request(Long.MAX_VALUE);
+            Function<Tuple, Tuple> payloadFunc = Function.identity();
+
+            Flow.Subscriber<Tuple> resultSubscriber = new Flow.Subscriber<>() {
+                @Override
+                public void onSubscribe(Flow.Subscription subscription) {
+                    subscription.request(Long.MAX_VALUE);
+                }
+
+                @Override
+                public void onNext(Tuple item) {
+                    System.out.println("Transaction processed: " + item);
+                }
+
+                @Override
+                public void onError(Throwable throwable) {
+                    System.err.println("Error during streaming: " + throwable.getMessage());
+                }
+
+                @Override
+                public void onComplete() {
+                    System.out.println("Streaming completed.");
+                }
+            };
+
+            try (var publisher = new SubmissionPublisher<Tuple>()) {
+                streamerFut = dummyTable.recordView().streamData(
+                        publisher,
+                        desc,
+                        keyFunc,
+                        payloadFunc,
+                        null, /* Optional Receiver arguments*/
+                        resultSubscriber,
+                        null /* DataStreamer options */
+                );
+
+                for (Tuple item : sourceData) {
+                    publisher.submit(item);
+                }
             }
 
-            @Override
-            public void onNext(Tuple item) {
-                System.out.println("Transaction processed: " + item);
+            streamerFut.join();
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        } catch (Exception e) {
+            throw new RuntimeException(e);
+        } finally {
+
+            System.out.println("Cleaning up resources");
+            //   undeployUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION);
+
+            /* Drop table */
+            System.out.println("\nDropping the table...");
+            try (
+                    Connection conn = getConnection("jdbc:ignite:thin://127.0.0.1:10800/");

Review Comment:
   This example uses raw JDBC (`DriverManager.getConnection("jdbc:ignite:thin://...")`) for table cleanup, while other examples use `client.sql().executeScript()`. The same inconsistency exists in `MultiTableDataStreamerExample`.
   
   Please use a consistent approach across all examples. Since the `IgniteClient` is already available in the `finally` block, prefer `client.sql().executeScript()` to avoid introducing unnecessary JDBC dependency and connection management.



##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeBroadcastExample.java:
##########
@@ -77,55 +124,87 @@ public static void main(String[] args) {
                 .addresses("127.0.0.1:10800")
                 .build()
         ) {
-            //--------------------------------------------------------------------------------------
-            //
-            // Configuring compute job.
-            //
-            //--------------------------------------------------------------------------------------
 
-            System.out.println("\nConfiguring compute job...");
+            try {
+
+                client.sql().executeScript("DROP SCHEMA IF EXISTS CUSTOM_SCHEMA CASCADE;");

Review Comment:
   The scope of this example has been significantly expanded. The original `ComputeBroadcastExample` only demonstrated broadcast job execution. Now it also:
   - Creates 3 schemas/tables (`CUSTOM_SCHEMA.MY_QUALIFIED_TABLE`, `PUBLIC.MY_TABLE`, `PERSON`)
   - Inserts data into `PERSON`
   - Drops tables in the `finally` block
   
   This table setup was previously handled externally (e.g., by the cluster setup or other examples). Embedding DDL here makes the broadcast example harder to understand — readers must parse 20 lines of schema setup before reaching the actual broadcast logic.
   
   Consider keeping table setup separate or at least extracting it to a helper method with a clear comment explaining why it's needed.



##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeAsyncExample.java:
##########
@@ -32,40 +33,89 @@
 import org.apache.ignite.compute.JobExecutionContext;
 import org.apache.ignite.compute.JobTarget;
 import org.apache.ignite.deployment.DeploymentUnit;
+import org.apache.ignite.example.code.deployment.AbstractDeploymentUnitExample;
 
 /**
- * This example demonstrates the usage of the
- * {@link IgniteCompute#executeAsync(JobTarget, JobDescriptor, Object)} API.
+ * This example demonstrates the usage of the {@link IgniteCompute#executeAsync(JobTarget, JobDescriptor, Object)} API.
  *
- * <p>Find instructions on how to run the example in the README.md file located in the "examples" directory root.
  *
- * <p>The following steps related to code deployment should be additionally executed before running the current example:
+ * <p>Find instructions on how to run the example in the {@code README.md}
+ * file located in the {@code examples} directory root.</p>
+ *
+ * <h2>Execution Modes</h2>

Review Comment:
   This ~50-line "Execution Modes" Javadoc block is copy-pasted identically across 14+ files. If the instructions change, all files must be updated manually.
   
   Consider:
   1. Moving the detailed instructions to a single place (e.g., the `README.md` or a shared doc), and referencing it from each example's Javadoc with a one-liner.
   2. Or at minimum, keeping only a brief note in each file and linking to the common instructions.
   
   (This was already flagged by @iartiukhov — extract common documentation to avoid massive duplication.)



##########
examples/java/src/main/java/org/apache/ignite/example/streaming/DistributedComputeWithReceiverExample.java:
##########
@@ -30,98 +35,189 @@
 import org.apache.ignite.catalog.ColumnType;
 import org.apache.ignite.catalog.definitions.TableDefinition;
 import org.apache.ignite.client.IgniteClient;
+import org.apache.ignite.deployment.DeploymentUnit;
+import org.apache.ignite.example.code.deployment.AbstractDeploymentUnitExample;
 import org.apache.ignite.table.DataStreamerReceiver;
 import org.apache.ignite.table.DataStreamerReceiverContext;
 import org.apache.ignite.table.DataStreamerReceiverDescriptor;
 import org.apache.ignite.table.Table;
 import org.apache.ignite.table.Tuple;
 
-/** This example demonstrates how to use the streaming API to simulate a fraud detection process,
- * which typically involves intensive processing of each transaction using ML models.
+/**
+ * This example demonstrates how to use the streaming API to simulate a fraud detection process, which typically involves intensive
+ * processing of each transaction using ML models.
+ *
+ * <p>Find instructions on how to run the example in the {@code README.md}
+ * file located in the {@code examples} directory root.</p>
+ *
+ * <h2>Execution Modes</h2>
+ *
+ * <p>There are two modes of execution:</p>
+ *
+ * <h3>1. Automated : The JAR Deployment for  deployment unit is automated </h3>
+ *
+ * <h4>1.1 With IDE</h4>
+ * <ul>
+ *   <li>
+ *     <b>Run from an IDE</b><br>
+ *     Launch the example directly from the IDE. If the required deployment
+ *     unit is not present, the example automatically builds and deploys the
+ *     necessary JAR.
+ *   </li>
+ * </ul>
+ *
+ * <h3>1.2 Without IDE</h3>
+ * <ul>
+ *   <li>
+ *     <b>Run from the command line</b><br>
+ *     Start the example using a Java command where the classpath includes
+ *     all required dependencies:
+ *
+ *     <pre>{@code
+ * java -cp "{user.home}\\.m2\\repository\\org\\apache\\ignite\\ignite-core\\3.1.0-SNAPSHOT\\
+ * ignite-core-3.1.0-SNAPSHOT.jar{other required jars}"
+ * <example-main-class> runFromIDE=false jarPath="{path-to-examples-jar}"
+ *     }</pre>
+ *
+ *     In this mode, {@code runFromIDE=false} indicates command-line execution,
+ *     and {@code jarPath} must reference the examples JAR used as the
+ *     deployment unit.
+ *   </li>
+ * </ul>
+ *
+ * <h2>2. Manual (with IDE): The JAR deployment for the deployment unit is manual</h2>
+ *
+ * <p>Before running this example, complete the following steps related to
+ * code deployment:</p>
+ *
+ * <ol>
+ *   <li>
+ *     Build the {@code ignite-examples-x.y.z.jar} file:<br>
+ *     {@code ./gradlew :ignite-examples:jar}
+ *   </li>
+ *   <li>
+ *     Deploy the generated JAR as a deployment unit using the CLI:<br>
+ *     <pre>{@code
+ * cluster unit deploy computeExampleUnit \
+ *     --version 1.0.0 \
+ *     --path=$IGNITE_HOME/examples/build/libs/ignite-examples-x.y.z.jar
+ *     }</pre>
+ *   </li>
+ * </ol>
  */
 
-public class DistributedComputeWithReceiverExample {
 
-    public static void main(String[] arg) {
+public class DistributedComputeWithReceiverExample extends AbstractDeploymentUnitExample {
+
+    private static final String DEPLOYMENT_UNIT_NAME = "streamerReceiverExampleUnit";
+
+    /** Deployment unit version. */
+    private static final String DEPLOYMENT_UNIT_VERSION = "1.0.0";
+
+    public static void main(String[] arg) throws Exception {
+
+        processDeploymentUnit(arg);
 
         try (IgniteClient client = IgniteClient.builder()
                 .addresses("127.0.0.1:10800")
                 .build()) {
 
-        /* Source data is a list of financial transactions */
-        /* We distribute this processing across the cluster, then gather and return results */
-        List<Tuple> sourceData = IntStream.range(1, 10)
-                .mapToObj(i -> Tuple.create()
-                        .set("txId", i)
-                        .set("txData", "{some-json-data}"))
-                .collect(Collectors.toList());
+            deployIfNotExist(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION, jarPath);
 
-        DataStreamerReceiverDescriptor<Tuple, Void, Tuple> desc = DataStreamerReceiverDescriptor
-                .builder(FraudDetectorReceiver.class)
-                .build();
+            /* Source data is a list of financial transactions */
+            /* We distribute this processing across the cluster, then gather and return results */
+            List<Tuple> sourceData = IntStream.range(1, 10)
+                    .mapToObj(i -> Tuple.create()
+                            .set("txId", i)
+                            .set("txData", "{some-json-data}"))
+                    .collect(Collectors.toList());
 
-        CompletableFuture<Void> streamerFut;
+            DataStreamerReceiverDescriptor<Tuple, Void, Tuple> desc = DataStreamerReceiverDescriptor
+                    .builder(FraudDetectorReceiver.class)
+                    .units(new DeploymentUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION))
+                    .build();
+
+            CompletableFuture<Void> streamerFut;
 
         /* Streaming requires a target table to partition data.
         /* Use a dummy table for this scenario, because we are not going to store any data */
-        TableDefinition txDummyTableDef = TableDefinition.builder("tx_dummy")
-                .columns(column("id", ColumnType.INTEGER))
-                .primaryKey("id")
-                .build();
+            TableDefinition txDummyTableDef = TableDefinition.builder("tx_dummy")
+                    .columns(column("id", ColumnType.INTEGER))
+                    .primaryKey("id")
+                    .build();
+
+            Table dummyTable = client.catalog().createTable(txDummyTableDef);
 
-        Table dummyTable = client.catalog().createTable(txDummyTableDef);
 
-        /* Source data has "txId" field, but target dummy table has "id" column, so keyFunc maps "txId" to "id" */
-        Function<Tuple, Tuple> keyFunc = sourceItem -> Tuple.create().set("id", sourceItem.value("txId"));
+            /* Source data has "txId" field, but target dummy table has "id" column, so keyFunc maps "txId" to "id" */
+            Function<Tuple, Tuple> keyFunc = sourceItem -> Tuple.create().set("id", sourceItem.value("txId"));
 
         /* Payload function is used to extract the payload (data that goes to the receiver) from the source item.
         /* In our case, we want to use the whole source item as the payload */
-        Function<Tuple, Tuple> payloadFunc = Function.identity();
-
-        Flow.Subscriber<Tuple> resultSubscriber = new Flow.Subscriber<>() {
-            @Override
-            public void onSubscribe(Flow.Subscription subscription) {
-                subscription.request(Long.MAX_VALUE);
+            Function<Tuple, Tuple> payloadFunc = Function.identity();
+
+            Flow.Subscriber<Tuple> resultSubscriber = new Flow.Subscriber<>() {
+                @Override
+                public void onSubscribe(Flow.Subscription subscription) {
+                    subscription.request(Long.MAX_VALUE);
+                }
+
+                @Override
+                public void onNext(Tuple item) {
+                    System.out.println("Transaction processed: " + item);
+                }
+
+                @Override
+                public void onError(Throwable throwable) {
+                    System.err.println("Error during streaming: " + throwable.getMessage());
+                }
+
+                @Override
+                public void onComplete() {
+                    System.out.println("Streaming completed.");
+                }
+            };
+
+            try (var publisher = new SubmissionPublisher<Tuple>()) {
+                streamerFut = dummyTable.recordView().streamData(
+                        publisher,
+                        desc,
+                        keyFunc,
+                        payloadFunc,
+                        null, /* Optional Receiver arguments*/
+                        resultSubscriber,
+                        null /* DataStreamer options */
+                );
+
+                for (Tuple item : sourceData) {
+                    publisher.submit(item);
+                }
             }
 
-            @Override
-            public void onNext(Tuple item) {
-                System.out.println("Transaction processed: " + item);
+            streamerFut.join();
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        } catch (Exception e) {
+            throw new RuntimeException(e);
+        } finally {
+
+            System.out.println("Cleaning up resources");
+            //   undeployUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION);

Review Comment:
   The `undeployUnit` call is commented out, which means the deployment unit is never cleaned up. This is inconsistent with every other example in this PR. Either uncomment it or add a comment explaining why cleanup is skipped here.



##########
examples/java/src/main/java/org/apache/ignite/example/table/QueryExample.java:
##########
@@ -62,7 +62,9 @@ public static void main(String[] args) throws Exception {
     }
 
     /**
-     * Demonstrates querying with an implicit transaction.
+     * Demonstrates querying with an explicit transaction.

Review Comment:
   The Javadoc says "Demonstrates querying with an **explicit** transaction" but the method is `performQueryWithoutTransaction` which uses an **implicit** transaction (`null` is passed as the transaction parameter). The original text ("implicit transaction") was correct.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2642336126


##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,263 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+/**
+ * Utility class for building and deploying Ignite compute units.
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+
+    /**
+     * Builds a JAR file by packaging all compiled classes present in the given directory.
+     *
+     * @param classesDir Directory containing compiled .class files.
+     * @param jarPath Target JAR file path to create.
+     * @throws IOException If building the JAR fails.
+     */
+    public static void buildJar(Path classesDir, Path jarPath) throws IOException {
+        if (!Files.exists(classesDir)) {
+            throw new IllegalArgumentException("Compiled classes not found: " + classesDir);
+        }
+
+        Files.createDirectories(jarPath.getParent());
+
+        try (OutputStream fos = Files.newOutputStream(jarPath);
+                JarOutputStream jar = new JarOutputStream(fos, createManifest())) {
+
+            Files.walk(classesDir).filter(Files::isRegularFile).forEach(path -> {
+                String entry = classesDir.relativize(path).toString().replace("\\", "/");
+                try (InputStream is = Files.newInputStream(path)) {
+                    jar.putNextEntry(new JarEntry(entry));
+                    is.transferTo(jar);
+                    jar.closeEntry();
+                } catch (Exception e) {
+                    throw new RuntimeException(e);
+                }
+            });
+        }
+
+        System.out.println("JAR built: " + jarPath);
+    }
+
+    /**
+     * Creates a simple manifest declaring manifest version.
+     *
+     * @return Manifest object.
+     */
+    private static Manifest createManifest() {
+        Manifest m = new Manifest();
+        m.getMainAttributes().putValue("Manifest-Version", "1.0");
+        return m;
+    }
+
+    /**
+     * Deploys a unit only if it is not already deployed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnitIfNeeded(String unitId, String version, Path jar) throws Exception {
+        if (deploymentExists(unitId, version)) {
+            System.out.println("Deployment unit already active. Skipping deployment.");
+            return;
+        }
+        deployUnit(unitId, version, jar);
+        System.out.println("Deployment completed.");
+    }
+
+    /**
+     * Checks if a deployment unit already exists on the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if active deployment exists.
+     * @throws Exception If request fails.
+     */
+    public static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET().build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        Thread.sleep(500);

Review Comment:
   What a reason for this sleep?



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {
+
+    // Root path of ignite-examples/
+    private static final Path projectRoot =
+            Paths.get("").toAbsolutePath();
+
+    // Compiled class output when running from IDE/CLI
+    private static final Path DEFAULT_CLASSES_DIR =
+            projectRoot.resolve("examples/java/build/classes/java/main");
+
+    // Default JAR output
+    private static final Path DEFAULT_JAR_PATH =
+            Path.of("build/libs/deploymentunit-example-1.0.0.jar");
+
+    protected static String jarPathAsString = "";
+    protected static Path jarPath = DEFAULT_JAR_PATH;
+    protected static boolean runFromIDE = true;
+    // ---------------------------------------------------
+
+    /**
+     * Processes the deployment unit.
+     *
+     * @param args Arguments passed to the deployment process.
+     * @throws IOException if any error occurs.
+     */
+    protected static void processDeploymentUnit(String[] args)
+            throws IOException {
+
+        Map<String, Object> p = DeployComputeUnit.processArguments(args);
+
+        boolean newRunFromIDE = (boolean) p.get("runFromIDE");
+        String newJarPathStr = (String) p.get("jarPath");
+
+        runFromIDE = newRunFromIDE;
+
+        // Use isBlank() instead of trim().isEmpty() to avoid creating a new String
+        if (newJarPathStr != null && !newJarPathStr.isBlank()) {
+            jarPathAsString = newJarPathStr;
+            jarPath = Path.of(newJarPathStr);
+        }
+
+        if (runFromIDE) {
+            DeployComputeUnit.buildJar(

Review Comment:
   Why do you build jar each time on runtime? Could you create a prebuilded jar and put it to resources? Or if you want to have unit code and work with it, then you need to achieve it in another way. Please take a look into `modules/compute` and how we manage units, espacially unit1 and unit2 source sets.



##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,263 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+/**
+ * Utility class for building and deploying Ignite compute units.
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+
+    /**
+     * Builds a JAR file by packaging all compiled classes present in the given directory.
+     *
+     * @param classesDir Directory containing compiled .class files.
+     * @param jarPath Target JAR file path to create.
+     * @throws IOException If building the JAR fails.
+     */
+    public static void buildJar(Path classesDir, Path jarPath) throws IOException {
+        if (!Files.exists(classesDir)) {
+            throw new IllegalArgumentException("Compiled classes not found: " + classesDir);
+        }
+
+        Files.createDirectories(jarPath.getParent());
+
+        try (OutputStream fos = Files.newOutputStream(jarPath);
+                JarOutputStream jar = new JarOutputStream(fos, createManifest())) {
+
+            Files.walk(classesDir).filter(Files::isRegularFile).forEach(path -> {
+                String entry = classesDir.relativize(path).toString().replace("\\", "/");
+                try (InputStream is = Files.newInputStream(path)) {
+                    jar.putNextEntry(new JarEntry(entry));
+                    is.transferTo(jar);
+                    jar.closeEntry();
+                } catch (Exception e) {
+                    throw new RuntimeException(e);
+                }
+            });
+        }
+
+        System.out.println("JAR built: " + jarPath);
+    }
+
+    /**
+     * Creates a simple manifest declaring manifest version.
+     *
+     * @return Manifest object.
+     */
+    private static Manifest createManifest() {
+        Manifest m = new Manifest();
+        m.getMainAttributes().putValue("Manifest-Version", "1.0");
+        return m;
+    }
+
+    /**
+     * Deploys a unit only if it is not already deployed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnitIfNeeded(String unitId, String version, Path jar) throws Exception {
+        if (deploymentExists(unitId, version)) {
+            System.out.println("Deployment unit already active. Skipping deployment.");
+            return;
+        }
+        deployUnit(unitId, version, jar);
+        System.out.println("Deployment completed.");
+    }
+
+    /**
+     * Checks if a deployment unit already exists on the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if active deployment exists.
+     * @throws Exception If request fails.
+     */
+    public static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET().build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        Thread.sleep(500);
+
+        if (resp.statusCode() != 200 && resp.statusCode() != 409) {
+            throw new RuntimeException("Deployment failed: " + resp.statusCode() + "\n" + resp.body());
+        }
+    }
+
+    /**
+     * Undeploys the given deployment unit from the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @throws Exception If undeployment fails.
+     */
+    public static void undeployUnit(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version))
+                .DELETE()
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        if (resp.statusCode() != 200 && resp.statusCode() != 404) {
+            throw new RuntimeException("Undeploy failed: " + resp.statusCode() + "\n" + resp.body());
+        }
+
+        for (int i = 0; i < 10; i++) {
+            if (!deploymentExists(unitId, version)) {
+                System.out.println("Unit successfully undeployed.");
+                return;
+            }
+            Thread.sleep(300);

Review Comment:
   What a reason for this sleep?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2603995928


##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {
+
+    // Root path of ignite-examples/
+    protected static final Path projectRoot =
+            Paths.get("").toAbsolutePath();
+
+    // Compiled class output when running from IDE/CLI
+    protected static final Path DEFAULT_CLASSES_DIR =
+            projectRoot.resolve("examples/java/build/classes/java/main");
+
+    // Default JAR output
+    protected static final Path DEFAULT_JAR_PATH =
+            Path.of("build/libs/deploymentunit-example-1.0.0.jar");
+
+    protected static volatile String jarPathAsString = "";
+    protected static volatile Path jarPath = DEFAULT_JAR_PATH;
+    protected static volatile boolean runFromIDE = true;
+    // ---------------------------------------------------
+
+    /**
+     * Processes the deployment unit.
+     *
+     * @param args Arguments passed to the deployment process.
+     * @throws IOException if any error occurs.
+     */
+    protected static synchronized void processDeploymentUnit(String[] args)

Review Comment:
   The earlier commit failed the PMD sanity check, which is why I initially modified the class to follow the Singleton pattern. After the PR review, I’m revising the implementation again to remove those unnecessary additions while still keeping the sanity checks passing.



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {
+
+    // Root path of ignite-examples/
+    protected static final Path projectRoot =
+            Paths.get("").toAbsolutePath();
+
+    // Compiled class output when running from IDE/CLI
+    protected static final Path DEFAULT_CLASSES_DIR =
+            projectRoot.resolve("examples/java/build/classes/java/main");
+
+    // Default JAR output
+    protected static final Path DEFAULT_JAR_PATH =
+            Path.of("build/libs/deploymentunit-example-1.0.0.jar");
+
+    protected static volatile String jarPathAsString = "";

Review Comment:
   The earlier commit failed the PMD sanity check, which is why I initially modified the class to follow the Singleton pattern. After the PR review, I’m revising the implementation again to remove those unnecessary additions while still keeping the sanity checks passing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "iartiukhov (via GitHub)" <gi...@apache.org>.
iartiukhov commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2647831294


##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,250 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.awaitility.Awaitility.await;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * Utility class for deploying Ignite compute units.
+ * <p>
+ * Note: The deployment unit JAR is now built at compile time via the deploymentUnitJar Gradle task,
+ * not at runtime. This eliminates the need for runtime JAR building.
+ * </p>
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
+    private static final int DEPLOYMENT_TIMEOUT_SECONDS = 30;
+
+
+    /**
+     * Checks if a deployment unit already exists on the cluster with DEPLOYED status.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if deployment exists with DEPLOYED status.
+     * @throws Exception If request fails.
+     */
+    private static boolean isDeployed(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET()
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        if (resp.statusCode() != 200) {
+            return false;
+        }
+
+        // Parse JSON response to check status
+        JsonNode root = OBJECT_MAPPER.readTree(resp.body());
+        JsonNode versionToStatus = root.path("versionToStatus");
+
+        if (versionToStatus.isArray()) {

Review Comment:
   Looks the clause is always `false` because of the error in JSON parsing logic. 
   So the [deployment check](https://github.com/apache/ignite-3/pull/7122/files#diff-5dbd940687ef85bfa02b01b5b57b7392116aad8e26b27c5ae0700ba8a3249de6R148) does not succeed and fails with timeout.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2832464452


##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.code.deployment;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import org.apache.ignite.example.util.DeployComputeUnit;
+
+public class AbstractDeploymentUnitExample {

Review Comment:
   The class still exists as a separate utility even though no example extends it anymore (good — the inheritance anti-pattern is gone). However, as I mentioned in my previous review, this class should be deleted entirely. All its logic — argument parsing, JAR path resolution, and the `processDeploymentUnit()` method — fits naturally into `DeployComputeUnit`, which is already the utility class for deployment operations. Having two utility classes for the same concern (`AbstractDeploymentUnitExample` + `DeployComputeUnit`) is unnecessary indirection.
   
   Please consolidate `processDeploymentUnit()` and the JAR path fields into `DeployComputeUnit`, then delete this class and update all callers to use `DeployComputeUnit` directly.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2589086411


##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/MyJob.java:
##########
@@ -68,4 +67,4 @@ public CompletableFuture<String> executeAsync(JobExecutionContext ctx, String ar
             throw new RuntimeException("Failed to run script", e);
         }
     }
-}
+}

Review Comment:
   Reverted to original.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "IgGusev (via GitHub)" <gi...@apache.org>.
IgGusev commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2597596452


##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeBroadcastExample.java:
##########
@@ -83,8 +137,40 @@ public static void main(String[] args) {
             //
             //--------------------------------------------------------------------------------------
 
+            try (
+                    Connection conn = getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
+                    Statement stmt = conn.createStatement()
+            ) {
+                stmt.execute("CREATE SCHEMA IF NOT EXISTS CUSTOM_SCHEMA");
+                stmt.execute("CREATE TABLE IF NOT EXISTS CUSTOM_SCHEMA.MY_QUALIFIED_TABLE (" +
+                        "ID INT PRIMARY KEY, MESSAGE VARCHAR(255))");
+
+                stmt.execute("CREATE SCHEMA IF NOT EXISTS PUBLIC");
+                stmt.execute("CREATE TABLE IF NOT EXISTS PUBLIC.MY_TABLE (" +
+                        "ID INT PRIMARY KEY, MESSAGE VARCHAR(255))");
+
+                stmt.execute("CREATE TABLE IF NOT EXISTS PERSON (" +
+                        "ID INT PRIMARY KEY, FIRST_NAME VARCHAR(100)," +
+                        "LAST_NAME VARCHAR(100), AGE INT)");
+
+                stmt.execute("INSERT INTO PERSON VALUES " +
+                        "(1, 'John', 'Doe', 36)," +
+                        "(2, 'Jane', 'Smith', 35)," +
+                        "(3, 'Robert', 'Johnson', 25)");
+
+            }
+
             System.out.println("\nConfiguring compute job...");
 
+            // 1) Check if deployment unit already exists

Review Comment:
   Same here



##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeAsyncExample.java:
##########
@@ -86,6 +140,15 @@ public static void main(String[] args) throws ExecutionException, InterruptedExc
 
             System.out.println("\nConfiguring compute job...");
 
+            // 1) Check if deployment unit already exists

Review Comment:
   What is step 2?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2599230756


##########
examples/java/src/main/java/org/apache/ignite/example/table/MapperExample.java:
##########
@@ -37,6 +41,29 @@ public Integer toColumnType(String cityId) {
     }
 
     public static void main(String[] args) throws Exception {
+
+        try (
+                Connection conn = getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
+                Statement stmt = conn.createStatement()
+        ) {
+
+            stmt.executeUpdate("DROP TABLE IF EXISTS Person");
+
+            stmt.executeUpdate(
+                    "CREATE TABLE IF NOT EXISTS Person ("

Review Comment:
   Suggestion received. Reverted unintended file addition - The file is being handled in a separate PR : https://github.com/apache/ignite-3/pull/7103 



##########
examples/java/src/main/java/org/apache/ignite/example/table/MapperExample.java:
##########
@@ -37,6 +41,29 @@ public Integer toColumnType(String cityId) {
     }
 
     public static void main(String[] args) throws Exception {
+
+        try (
+                Connection conn = getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
+                Statement stmt = conn.createStatement()
+        ) {
+
+            stmt.executeUpdate("DROP TABLE IF EXISTS Person");

Review Comment:
   Suggestion received. Reverted unintended file addition - The file is being handled in a separate PR : https://github.com/apache/ignite-3/pull/7103 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2598941862


##########
examples/java/src/main/java/org/apache/ignite/example/table/MapperExample.java:
##########
@@ -37,6 +41,29 @@ public Integer toColumnType(String cityId) {
     }
 
     public static void main(String[] args) throws Exception {
+
+        try (
+                Connection conn = getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
+                Statement stmt = conn.createStatement()
+        ) {
+
+            stmt.executeUpdate("DROP TABLE IF EXISTS Person");

Review Comment:
   I think this is a bad practice, an example must not delete tables that it did not create.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2602434440


##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;

Review Comment:
   Done



##########
examples/java/src/main/java/org/apache/ignite/example/code/deployment/AbstractDeploymentUnitExample.java:
##########
@@ -0,0 +1,57 @@
+package org.apache.ignite.example.code.deployment;

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "iartiukhov (via GitHub)" <gi...@apache.org>.
iartiukhov commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2584099030


##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeJobPriorityExample.java:
##########
@@ -122,13 +187,19 @@ public static void main(String[] args) {
             //--------------------------------------------------------------------------------------
 
             CompletableFuture.allOf(jobFutures.toArray(new CompletableFuture[0])).join();
+        } catch (Exception e) {
+            throw new RuntimeException(e);
+        } finally {
+
+            System.out.println("Cleaning up resources");
+            undeployUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION);
         }
     }
 
     /**
      * High-priority job.
      */
-    private static class HighPriorityJob implements ComputeJob<Integer, String> {
+    public static class HighPriorityJob implements ComputeJob<Integer, String> {

Review Comment:
   What's the reason behind changing the visibility scope? Here and in all other implementations of ComputeJob.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "Pochatkin (via GitHub)" <gi...@apache.org>.
Pochatkin commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2642573933


##########
examples/java/src/main/java/org/apache/ignite/example/util/DeployComputeUnit.java:
##########
@@ -0,0 +1,263 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.example.util;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.jar.JarEntry;
+import java.util.jar.JarOutputStream;
+import java.util.jar.Manifest;
+
+/**
+ * Utility class for building and deploying Ignite compute units.
+ */
+public class DeployComputeUnit {
+
+    private static final String BASE_URL = "http://localhost:10300";
+    private static final HttpClient HTTP = HttpClient.newHttpClient();
+
+    /**
+     * Builds a JAR file by packaging all compiled classes present in the given directory.
+     *
+     * @param classesDir Directory containing compiled .class files.
+     * @param jarPath Target JAR file path to create.
+     * @throws IOException If building the JAR fails.
+     */
+    public static void buildJar(Path classesDir, Path jarPath) throws IOException {
+        if (!Files.exists(classesDir)) {
+            throw new IllegalArgumentException("Compiled classes not found: " + classesDir);
+        }
+
+        Files.createDirectories(jarPath.getParent());
+
+        try (OutputStream fos = Files.newOutputStream(jarPath);
+                JarOutputStream jar = new JarOutputStream(fos, createManifest())) {
+
+            Files.walk(classesDir).filter(Files::isRegularFile).forEach(path -> {
+                String entry = classesDir.relativize(path).toString().replace("\\", "/");
+                try (InputStream is = Files.newInputStream(path)) {
+                    jar.putNextEntry(new JarEntry(entry));
+                    is.transferTo(jar);
+                    jar.closeEntry();
+                } catch (Exception e) {
+                    throw new RuntimeException(e);
+                }
+            });
+        }
+
+        System.out.println("JAR built: " + jarPath);
+    }
+
+    /**
+     * Creates a simple manifest declaring manifest version.
+     *
+     * @return Manifest object.
+     */
+    private static Manifest createManifest() {
+        Manifest m = new Manifest();
+        m.getMainAttributes().putValue("Manifest-Version", "1.0");
+        return m;
+    }
+
+    /**
+     * Deploys a unit only if it is not already deployed.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnitIfNeeded(String unitId, String version, Path jar) throws Exception {
+        if (deploymentExists(unitId, version)) {
+            System.out.println("Deployment unit already active. Skipping deployment.");
+            return;
+        }
+        deployUnit(unitId, version, jar);
+        System.out.println("Deployment completed.");
+    }
+
+    /**
+     * Checks if a deployment unit already exists on the cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @return True if active deployment exists.
+     * @throws Exception If request fails.
+     */
+    public static boolean deploymentExists(String unitId, String version) throws Exception {
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/cluster/units/" + unitId))
+                .GET().build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+        return resp.statusCode() == 200 && resp.body().contains("\"version\":\"" + version + "\"");
+    }
+
+    /**
+     * Deploys a unit to the Ignite cluster.
+     *
+     * @param unitId Deployment unit ID.
+     * @param version Deployment version.
+     * @param jar Path to the JAR file to upload.
+     * @throws Exception If deployment fails.
+     */
+    public static void deployUnit(String unitId, String version, Path jar) throws Exception {
+        String boundary = "igniteBoundary";
+
+        byte[] jarBytes = Files.readAllBytes(jar);
+
+        String start =
+                "--" + boundary + "\r\n" +
+                        "Content-Disposition: form-data; name=\"unitContent\"; filename=\"" + jar.getFileName() + "\"\r\n" +
+                        "Content-Type: application/java-archive\r\n\r\n";
+
+        String end = "\r\n--" + boundary + "--\r\n";
+
+        byte[] startBytes = start.getBytes();
+        byte[] endBytes = end.getBytes();
+
+        byte[] full = new byte[startBytes.length + jarBytes.length + endBytes.length];
+
+        System.arraycopy(startBytes, 0, full, 0, startBytes.length);
+        System.arraycopy(jarBytes, 0, full, startBytes.length, jarBytes.length);
+        System.arraycopy(endBytes, 0, full, startBytes.length + jarBytes.length, endBytes.length);
+
+        HttpRequest req = HttpRequest.newBuilder()
+                .uri(new URI(BASE_URL + "/management/v1/deployment/units/" + unitId + "/" + version + "?deployMode=ALL"))
+                .header("Content-Type", "multipart/form-data; boundary=" + boundary)
+                .POST(HttpRequest.BodyPublishers.ofByteArray(full))
+                .build();
+
+        HttpResponse<String> resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString());
+
+        Thread.sleep(500);

Review Comment:
   And you think that just Thread,sleep is the best way? You don't have any checks that state is DEPLOYED or anything else. Please reimplement this approach. You need to check state periodically, if you want you can use Awaitility lib for example (please take a look into existed tests).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "ashishm07 (via GitHub)" <gi...@apache.org>.
ashishm07 commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2589083368


##########
examples/java/src/main/java/org/apache/ignite/example/compute/ComputeBroadcastExample.java:
##########
@@ -111,21 +195,41 @@ public static void main(String[] args) {
 
             QualifiedName customSchemaTable = QualifiedName.parse("CUSTOM_SCHEMA.MY_QUALIFIED_TABLE");
             client.compute().execute(table(customSchemaTable),
-                    JobDescriptor.builder(HelloMessageJob.class).build(), null
+                    JobDescriptor.builder(HelloMessageJob.class)
+                            .units(new DeploymentUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION))
+                            .build(), null
             );
 
-
             QualifiedName customSchemaTableName = QualifiedName.of("PUBLIC", "MY_TABLE");
             client.compute().execute(table(customSchemaTableName),
-                    JobDescriptor.builder(HelloMessageJob.class).build(), null
+                    JobDescriptor.builder(HelloMessageJob.class)
+                            .units(new DeploymentUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION))
+                            .build(), null
             );
+        } catch (Exception e) {
+            throw new RuntimeException(e);
+        } finally {
+
+            System.out.println("Cleaning up resources");
+            // undeployUnit(DEPLOYMENT_UNIT_NAME, DEPLOYMENT_UNIT_VERSION);

Review Comment:
   Added this to verify that other classes using the same deployment unit function correctly. Uncommented it now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] IGNITE-27212 - Complete automation for compute deployment unit; Minor bug fixes [ignite-3]

Posted by "xtern (via GitHub)" <gi...@apache.org>.
xtern commented on code in PR #7122:
URL: https://github.com/apache/ignite-3/pull/7122#discussion_r2598952107


##########
examples/java/src/main/java/org/apache/ignite/example/table/MapperExample.java:
##########
@@ -37,6 +41,29 @@ public Integer toColumnType(String cityId) {
     }
 
     public static void main(String[] args) throws Exception {
+
+        try (
+                Connection conn = getConnection("jdbc:ignite:thin://127.0.0.1:10800/");
+                Statement stmt = conn.createStatement()
+        ) {
+
+            stmt.executeUpdate("DROP TABLE IF EXISTS Person");
+
+            stmt.executeUpdate(
+                    "CREATE TABLE IF NOT EXISTS Person ("

Review Comment:
   I think we should avoid using IF NOT EXISTS/IF EXISTS if the table already exists - the example shouldn't work because the table schema might be different and we'll get confusing errors.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org