You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Manas Kale <ma...@gmail.com> on 2020/08/31 05:43:20 UTC
Packaging multiple Flink jobs from a single IntelliJ project
Hi,
I have an IntelliJ project that has multiple classes with main() functions.
I want to package this project as a JAR that I can submit to the Flink
cluster and specify the entry class when I start the job. Here are my
questions:
- I am not really familiar with Maven and would appreciate some
pointers/examples. From what I understand, I will need to use some sort of
transformer in the Maven shade plugin to merge all of the classes. *If
this is correct, can I see a small example? *
- Also, I can't get a single main class working:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>flink_summarization</groupId>
<artifactId>flink_summarization</artifactId>
<version>0.1</version>
<packaging>jar</packaging>
<name>Flink Quickstart Job</name>
<url>http://www.myorganization.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<flink.version>1.10.1</flink.version>
<java.version>1.8</java.version>
<scala.binary.version>2.11</scala.binary.version>
<maven.compiler.source>${java.version}</maven.compiler.source>
<maven.compiler.target>${java.version}</maven.compiler.target>
</properties>
<repositories>
<repository>
<id>apache.snapshots</id>
<name>Apache Development Snapshot Repository</name>
<url>https://repository.apache.org/content/repositories/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<dependencies>
<!-- Apache Flink dependencies -->
<!-- These dependencies are provided, because they should not be
packaged into the JAR file. -->
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-jdbc -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-state-processor-api_2.11</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-jdbc_2.11</artifactId>
<version>1.11.0</version>
</dependency>
<!-- Add logging framework, to produce console output when
running in the IDE. -->
<!-- These dependencies are excluded from the application JAR by
default. -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.7</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
<scope>runtime</scope>
</dependency>
<!-- Test dependencies-->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-test-utils_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime_2.11</artifactId>
<version>${flink.version}</version>
<scope>test</scope>
<classifier>tests</classifier>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
<version>${flink.version}</version>
<scope>test</scope>
<classifier>tests</classifier>
</dependency>
<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<!-- use 2.9.1 for Java 7 projects -->
<version>3.16.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<!-- Java Compiler -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<!-- We use the maven-shade plugin to create a fat jar that
contains all necessary dependencies. -->
<!-- Change the value of <mainClass>...</mainClass> if your
program entry point changes. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<!-- author = Manas - dependencyReducedPOM was disabled to
stop maven from entering an infinite loop -->
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<excludes>
<exclude>org.apache.flink:force-shading</exclude>
<exclude>com.google.code.findbugs:jsr305</exclude>
<exclude>org.slf4j:*</exclude>
<exclude>log4j:*</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<!-- Do not copy the signatures in the
META-INF folder.
Otherwise, this might cause
SecurityExceptions when using the JAR. -->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>iu.feature_summarization.basic_features.pre.BasicPreProcessJob</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<!-- This improves the out-of-the-box experience in
Eclipse by resolving some warnings. -->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<versionRange>[3.0.0,)</versionRange>
<goals>
<goal>shade</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore/>
</action>
</pluginExecution>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<versionRange>[3.1,)</versionRange>
<goals>
<goal>testCompile</goal>
<goal>compile</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore/>
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
<!-- This profile helps to make things run out of the box in IntelliJ -->
<!-- Its adds Flink's core classes to the runtime class path. -->
<!-- Otherwise they are missing in IntelliJ, because the dependency
is 'provided' -->
<profiles>
<profile>
<id>add-dependencies-for-IDEA</id>
<activation>
<property>
<name>idea.version</name>
</property>
</activation>
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>${flink.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>compile</scope>
</dependency>
</dependencies>
</profile>
</profiles>
</project>
The fully qualified name within tag <mainClass> is correct and contains the
main() method. However, when I submit this to Flink I get the error:
> ../../flink-1.11.0/bin/flink run target/flink_summarization-0.1.jar
------------------------------------------------------------
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The program's
entry point class '
iu.feature_summarization.basic_features.pre.BasicPreProcessJob' was not
found in the jar file
.
*Am I specifying some <scope> tag incorrectly in my pom file?*
Thank you,
Manas
Re: Packaging multiple Flink jobs from a single IntelliJ project
Posted by Flavio Pompermaier <po...@okkam.it>.
Yes, the recommended way to proceed in your use case is to put all classes
in a single JAR and to specify the main class to run to the flink client.
Best,
Flavio
Re: Packaging multiple Flink jobs from a single IntelliJ project
Posted by Manas Kale <ma...@gmail.com>.
Guess I figured out a solution for the first question as well - I am
packaging multiple main() classes in the same JAR and specifying entrypoint
classes when submitting the JAR. Most of my issues stemmed from an
improperly configured POM file and a mismatch in Flink runtime versions.
I'll assume this is the recommended way to go about doing this, thanks for
reading and have a great day!
On Mon, Aug 31, 2020 at 12:03 PM Manas Kale <ma...@gmail.com> wrote:
> Hi,
> I solved my second issue - I was not following Maven's convention for
> placing source code (I had not placed my source in src/main/java).
> However, I still would like some help with my first question - what is the
> recommended way to set a project with multiple main() classes? At the end,
> I would like to be able to run each main() class as a separate job. Should
> I create a single JAR and specify different entrypoint classes each time or
> should I create separate JARs for each main() class?
>
> On Mon, Aug 31, 2020 at 11:13 AM Manas Kale <ma...@gmail.com> wrote:
>
>> Hi,
>> I have an IntelliJ project that has multiple classes with main()
>> functions. I want to package this project as a JAR that I can submit to the
>> Flink cluster and specify the entry class when I start the job. Here are my
>> questions:
>>
>> - I am not really familiar with Maven and would appreciate some
>> pointers/examples. From what I understand, I will need to use some sort of
>> transformer in the Maven shade plugin to merge all of the classes. *If
>> this is correct, can I see a small example? *
>> - Also, I can't get a single main class working:
>>
>>
>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>> <modelVersion>4.0.0</modelVersion>
>>
>> <groupId>flink_summarization</groupId>
>> <artifactId>flink_summarization</artifactId>
>> <version>0.1</version>
>> <packaging>jar</packaging>
>>
>> <name>Flink Quickstart Job</name>
>> <url>http://www.myorganization.org</url>
>>
>> <properties>
>> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
>> <flink.version>1.10.1</flink.version>
>> <java.version>1.8</java.version>
>> <scala.binary.version>2.11</scala.binary.version>
>> <maven.compiler.source>${java.version}</maven.compiler.source>
>> <maven.compiler.target>${java.version}</maven.compiler.target>
>> </properties>
>>
>> <repositories>
>> <repository>
>> <id>apache.snapshots</id>
>> <name>Apache Development Snapshot Repository</name>
>> <url>https://repository.apache.org/content/repositories/snapshots/</url>
>> <releases>
>> <enabled>false</enabled>
>> </releases>
>> <snapshots>
>> <enabled>true</enabled>
>> </snapshots>
>> </repository>
>> </repositories>
>>
>> <dependencies>
>> <!-- Apache Flink dependencies -->
>> <!-- These dependencies are provided, because they should not be packaged into the JAR file. -->
>> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-jdbc -->
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-java</artifactId>
>> <version>${flink.version}</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
>> <version>${flink.version}</version>
>> <scope>provided</scope>
>> </dependency>
>>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-connector-kafka_2.11</artifactId>
>> <version>${flink.version}</version>
>> </dependency>
>>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-state-processor-api_2.11</artifactId>
>> <version>${flink.version}</version>
>> <scope>provided</scope>
>> </dependency>
>>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-connector-jdbc_2.11</artifactId>
>> <version>1.11.0</version>
>> </dependency>
>>
>> <!-- Add logging framework, to produce console output when running in the IDE. -->
>> <!-- These dependencies are excluded from the application JAR by default. -->
>> <dependency>
>> <groupId>org.slf4j</groupId>
>> <artifactId>slf4j-log4j12</artifactId>
>> <version>1.7.7</version>
>> <scope>runtime</scope>
>> </dependency>
>> <dependency>
>> <groupId>log4j</groupId>
>> <artifactId>log4j</artifactId>
>> <version>1.2.17</version>
>> <scope>runtime</scope>
>> </dependency>
>>
>> <!-- Test dependencies-->
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-test-utils_${scala.binary.version}</artifactId>
>> <version>${flink.version}</version>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-runtime_2.11</artifactId>
>> <version>${flink.version}</version>
>> <scope>test</scope>
>> <classifier>tests</classifier>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-streaming-java_2.11</artifactId>
>> <version>${flink.version}</version>
>> <scope>test</scope>
>> <classifier>tests</classifier>
>> </dependency>
>> <dependency>
>> <groupId>org.assertj</groupId>
>> <artifactId>assertj-core</artifactId>
>> <!-- use 2.9.1 for Java 7 projects -->
>> <version>3.16.1</version>
>> <scope>test</scope>
>> </dependency>
>>
>>
>> </dependencies>
>>
>> <build>
>> <plugins>
>>
>> <!-- Java Compiler -->
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-compiler-plugin</artifactId>
>> <version>3.1</version>
>> <configuration>
>> <source>${java.version}</source>
>> <target>${java.version}</target>
>> </configuration>
>> </plugin>
>>
>> <!-- We use the maven-shade plugin to create a fat jar that contains all necessary dependencies. -->
>> <!-- Change the value of <mainClass>...</mainClass> if your program entry point changes. -->
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-shade-plugin</artifactId>
>> <version>3.0.0</version>
>> <!-- author = Manas - dependencyReducedPOM was disabled to stop maven from entering an infinite loop -->
>> <configuration>
>> <createDependencyReducedPom>false</createDependencyReducedPom>
>> </configuration>
>> <executions>
>> <!-- Run shade goal on package phase -->
>> <execution>
>> <phase>package</phase>
>> <goals>
>> <goal>shade</goal>
>> </goals>
>> <configuration>
>> <artifactSet>
>> <excludes>
>> <exclude>org.apache.flink:force-shading</exclude>
>> <exclude>com.google.code.findbugs:jsr305</exclude>
>> <exclude>org.slf4j:*</exclude>
>> <exclude>log4j:*</exclude>
>> </excludes>
>> </artifactSet>
>> <filters>
>> <filter>
>> <!-- Do not copy the signatures in the META-INF folder.
>> Otherwise, this might cause SecurityExceptions when using the JAR. -->
>> <artifact>*:*</artifact>
>> <excludes>
>> <exclude>META-INF/*.SF</exclude>
>> <exclude>META-INF/*.DSA</exclude>
>> <exclude>META-INF/*.RSA</exclude>
>> </excludes>
>> </filter>
>> </filters>
>> <transformers>
>> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
>> <mainClass>iu.feature_summarization.basic_features.pre.BasicPreProcessJob</mainClass>
>> </transformer>
>>
>> </transformers>
>> </configuration>
>> </execution>
>> </executions>
>> </plugin>
>> </plugins>
>>
>> <pluginManagement>
>> <plugins>
>>
>> <!-- This improves the out-of-the-box experience in Eclipse by resolving some warnings. -->
>> <plugin>
>> <groupId>org.eclipse.m2e</groupId>
>> <artifactId>lifecycle-mapping</artifactId>
>> <version>1.0.0</version>
>> <configuration>
>> <lifecycleMappingMetadata>
>> <pluginExecutions>
>> <pluginExecution>
>> <pluginExecutionFilter>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-shade-plugin</artifactId>
>> <versionRange>[3.0.0,)</versionRange>
>> <goals>
>> <goal>shade</goal>
>> </goals>
>> </pluginExecutionFilter>
>> <action>
>> <ignore/>
>> </action>
>> </pluginExecution>
>> <pluginExecution>
>> <pluginExecutionFilter>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-compiler-plugin</artifactId>
>> <versionRange>[3.1,)</versionRange>
>> <goals>
>> <goal>testCompile</goal>
>> <goal>compile</goal>
>> </goals>
>> </pluginExecutionFilter>
>> <action>
>> <ignore/>
>> </action>
>> </pluginExecution>
>> </pluginExecutions>
>> </lifecycleMappingMetadata>
>> </configuration>
>> </plugin>
>> </plugins>
>> </pluginManagement>
>> </build>
>>
>> <!-- This profile helps to make things run out of the box in IntelliJ -->
>> <!-- Its adds Flink's core classes to the runtime class path. -->
>> <!-- Otherwise they are missing in IntelliJ, because the dependency is 'provided' -->
>> <profiles>
>> <profile>
>> <id>add-dependencies-for-IDEA</id>
>>
>> <activation>
>> <property>
>> <name>idea.version</name>
>> </property>
>> </activation>
>>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-java</artifactId>
>> <version>${flink.version}</version>
>> <scope>compile</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
>> <version>${flink.version}</version>
>> <scope>compile</scope>
>> </dependency>
>> </dependencies>
>> </profile>
>> </profiles>
>>
>> </project>
>>
>>
>> The fully qualified name within tag <mainClass> is correct and contains
>> the main() method. However, when I submit this to Flink I get the error:
>> > ../../flink-1.11.0/bin/flink run target/flink_summarization-0.1.jar
>>
>> ------------------------------------------------------------
>> The program finished with the following exception:
>>
>> org.apache.flink.client.program.ProgramInvocationException: The program's
>> entry point class '
>> iu.feature_summarization.basic_features.pre.BasicPreProcessJob' was not
>> found in the jar file
>> .
>>
>> *Am I specifying some <scope> tag incorrectly in my pom file?*
>>
>>
>> Thank you,
>> Manas
>>
>
Re: Packaging multiple Flink jobs from a single IntelliJ project
Posted by Manas Kale <ma...@gmail.com>.
Hi,
I solved my second issue - I was not following Maven's convention for
placing source code (I had not placed my source in src/main/java).
However, I still would like some help with my first question - what is the
recommended way to set a project with multiple main() classes? At the end,
I would like to be able to run each main() class as a separate job. Should
I create a single JAR and specify different entrypoint classes each time or
should I create separate JARs for each main() class?
On Mon, Aug 31, 2020 at 11:13 AM Manas Kale <ma...@gmail.com> wrote:
> Hi,
> I have an IntelliJ project that has multiple classes with main()
> functions. I want to package this project as a JAR that I can submit to the
> Flink cluster and specify the entry class when I start the job. Here are my
> questions:
>
> - I am not really familiar with Maven and would appreciate some
> pointers/examples. From what I understand, I will need to use some sort of
> transformer in the Maven shade plugin to merge all of the classes. *If
> this is correct, can I see a small example? *
> - Also, I can't get a single main class working:
>
>
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
> <modelVersion>4.0.0</modelVersion>
>
> <groupId>flink_summarization</groupId>
> <artifactId>flink_summarization</artifactId>
> <version>0.1</version>
> <packaging>jar</packaging>
>
> <name>Flink Quickstart Job</name>
> <url>http://www.myorganization.org</url>
>
> <properties>
> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
> <flink.version>1.10.1</flink.version>
> <java.version>1.8</java.version>
> <scala.binary.version>2.11</scala.binary.version>
> <maven.compiler.source>${java.version}</maven.compiler.source>
> <maven.compiler.target>${java.version}</maven.compiler.target>
> </properties>
>
> <repositories>
> <repository>
> <id>apache.snapshots</id>
> <name>Apache Development Snapshot Repository</name>
> <url>https://repository.apache.org/content/repositories/snapshots/</url>
> <releases>
> <enabled>false</enabled>
> </releases>
> <snapshots>
> <enabled>true</enabled>
> </snapshots>
> </repository>
> </repositories>
>
> <dependencies>
> <!-- Apache Flink dependencies -->
> <!-- These dependencies are provided, because they should not be packaged into the JAR file. -->
> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-jdbc -->
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-java</artifactId>
> <version>${flink.version}</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
> <version>${flink.version}</version>
> <scope>provided</scope>
> </dependency>
>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-connector-kafka_2.11</artifactId>
> <version>${flink.version}</version>
> </dependency>
>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-state-processor-api_2.11</artifactId>
> <version>${flink.version}</version>
> <scope>provided</scope>
> </dependency>
>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-connector-jdbc_2.11</artifactId>
> <version>1.11.0</version>
> </dependency>
>
> <!-- Add logging framework, to produce console output when running in the IDE. -->
> <!-- These dependencies are excluded from the application JAR by default. -->
> <dependency>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-log4j12</artifactId>
> <version>1.7.7</version>
> <scope>runtime</scope>
> </dependency>
> <dependency>
> <groupId>log4j</groupId>
> <artifactId>log4j</artifactId>
> <version>1.2.17</version>
> <scope>runtime</scope>
> </dependency>
>
> <!-- Test dependencies-->
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-test-utils_${scala.binary.version}</artifactId>
> <version>${flink.version}</version>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-runtime_2.11</artifactId>
> <version>${flink.version}</version>
> <scope>test</scope>
> <classifier>tests</classifier>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-streaming-java_2.11</artifactId>
> <version>${flink.version}</version>
> <scope>test</scope>
> <classifier>tests</classifier>
> </dependency>
> <dependency>
> <groupId>org.assertj</groupId>
> <artifactId>assertj-core</artifactId>
> <!-- use 2.9.1 for Java 7 projects -->
> <version>3.16.1</version>
> <scope>test</scope>
> </dependency>
>
>
> </dependencies>
>
> <build>
> <plugins>
>
> <!-- Java Compiler -->
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-compiler-plugin</artifactId>
> <version>3.1</version>
> <configuration>
> <source>${java.version}</source>
> <target>${java.version}</target>
> </configuration>
> </plugin>
>
> <!-- We use the maven-shade plugin to create a fat jar that contains all necessary dependencies. -->
> <!-- Change the value of <mainClass>...</mainClass> if your program entry point changes. -->
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-shade-plugin</artifactId>
> <version>3.0.0</version>
> <!-- author = Manas - dependencyReducedPOM was disabled to stop maven from entering an infinite loop -->
> <configuration>
> <createDependencyReducedPom>false</createDependencyReducedPom>
> </configuration>
> <executions>
> <!-- Run shade goal on package phase -->
> <execution>
> <phase>package</phase>
> <goals>
> <goal>shade</goal>
> </goals>
> <configuration>
> <artifactSet>
> <excludes>
> <exclude>org.apache.flink:force-shading</exclude>
> <exclude>com.google.code.findbugs:jsr305</exclude>
> <exclude>org.slf4j:*</exclude>
> <exclude>log4j:*</exclude>
> </excludes>
> </artifactSet>
> <filters>
> <filter>
> <!-- Do not copy the signatures in the META-INF folder.
> Otherwise, this might cause SecurityExceptions when using the JAR. -->
> <artifact>*:*</artifact>
> <excludes>
> <exclude>META-INF/*.SF</exclude>
> <exclude>META-INF/*.DSA</exclude>
> <exclude>META-INF/*.RSA</exclude>
> </excludes>
> </filter>
> </filters>
> <transformers>
> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
> <mainClass>iu.feature_summarization.basic_features.pre.BasicPreProcessJob</mainClass>
> </transformer>
>
> </transformers>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
>
> <pluginManagement>
> <plugins>
>
> <!-- This improves the out-of-the-box experience in Eclipse by resolving some warnings. -->
> <plugin>
> <groupId>org.eclipse.m2e</groupId>
> <artifactId>lifecycle-mapping</artifactId>
> <version>1.0.0</version>
> <configuration>
> <lifecycleMappingMetadata>
> <pluginExecutions>
> <pluginExecution>
> <pluginExecutionFilter>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-shade-plugin</artifactId>
> <versionRange>[3.0.0,)</versionRange>
> <goals>
> <goal>shade</goal>
> </goals>
> </pluginExecutionFilter>
> <action>
> <ignore/>
> </action>
> </pluginExecution>
> <pluginExecution>
> <pluginExecutionFilter>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-compiler-plugin</artifactId>
> <versionRange>[3.1,)</versionRange>
> <goals>
> <goal>testCompile</goal>
> <goal>compile</goal>
> </goals>
> </pluginExecutionFilter>
> <action>
> <ignore/>
> </action>
> </pluginExecution>
> </pluginExecutions>
> </lifecycleMappingMetadata>
> </configuration>
> </plugin>
> </plugins>
> </pluginManagement>
> </build>
>
> <!-- This profile helps to make things run out of the box in IntelliJ -->
> <!-- Its adds Flink's core classes to the runtime class path. -->
> <!-- Otherwise they are missing in IntelliJ, because the dependency is 'provided' -->
> <profiles>
> <profile>
> <id>add-dependencies-for-IDEA</id>
>
> <activation>
> <property>
> <name>idea.version</name>
> </property>
> </activation>
>
> <dependencies>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-java</artifactId>
> <version>${flink.version}</version>
> <scope>compile</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
> <version>${flink.version}</version>
> <scope>compile</scope>
> </dependency>
> </dependencies>
> </profile>
> </profiles>
>
> </project>
>
>
> The fully qualified name within tag <mainClass> is correct and contains
> the main() method. However, when I submit this to Flink I get the error:
> > ../../flink-1.11.0/bin/flink run target/flink_summarization-0.1.jar
>
> ------------------------------------------------------------
> The program finished with the following exception:
>
> org.apache.flink.client.program.ProgramInvocationException: The program's
> entry point class '
> iu.feature_summarization.basic_features.pre.BasicPreProcessJob' was not
> found in the jar file
> .
>
> *Am I specifying some <scope> tag incorrectly in my pom file?*
>
>
> Thank you,
> Manas
>