You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Smile@LETTers" <le...@163.com> on 2021/01/19 10:18:27 UTC
Test failed in
flink-end-to-end-tests/flink-end-to-end-tests-common-kafka
Hi,
I got an error when tried to compile & package Flink (version 1.12 & current master).
It can be reproduced by run 'mvn clean test' under flink-end-to-end-tests/flink-end-to-end-tests-common-kafka.
It seems that a necessary dependency for test scope was missing and some classes can not be found.
After adding the dependency kafka-avro-serializer to the pom of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes well.
And I just wonder that is this a bug or I missed some local setting?
Best regards.
Smile
Error logs attached:
[INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka >--------
[INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading ...
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ flink-end-to-end-tests-common-kafka ---
[INFO] Deleting /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using plugin classloader, includes GMavenPlus classpath.
[INFO] Using Groovy 2.5.12 to perform execute.
includes: org.apache.flink.tests.util.categories.Dummy
excludes:
[INFO]
[INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Highest basedir set to: /Users/smile/Downloads/W/code/flink/apache/master/flink
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 5 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java uses unchecked or unsafe operations.
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 4 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.842 s
[INFO] Finished at: 2021-01-19T18:04:39+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project flink-end-to-end-tests-common-kafka: Compilation failure
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
[ERROR] class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Re:Re: Re: Re: Test failed in
flink-end-to-end-tests/flink-end-to-end-tests-common-kafka
Posted by "Smile@LETTers" <le...@163.com>.
Thanks, Matthias!
I tried your suggestion and it does work.
After installing kafka-avro-serializer with pom I got some more errors about io.confluent:kafka-schema-registry-parent:pom:5.5.2 and io.confluent:rest-utils-parent:pom:5.5.2 and so on. After manually installing all these dependencies with pom, the build succeeded.
---------------------------------------
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-avro-serializer-5.5.2.jar -DpomFile=kafka-avro-serializer-5.5.2.pom
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-schema-serializer -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-schema-serializer-5.5.2.jar -DpomFile=kafka-schema-serializer-5.5.2.pom
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-schema-registry-client -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-schema-registry-client-5.5.2.jar -DpomFile=kafka-schema-registry-client-5.5.2.pom
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-schema-registry-parent -Dversion=5.5.2 -Dpackaging=pom -Dfile=kafka-schema-registry-parent-5.5.2.pom -DpomFile=kafka-schema-registry-parent-5.5.2.pom
mvn install:install-file -DgroupId=io.confluent -DartifactId=common -Dversion=5.5.2 -Dpackaging=pom -Dfile=common-5.5.2.pom -DpomFile=common-5.5.2.pom
mvn install:install-file -DgroupId=io.confluent -DartifactId=common-parent -Dversion=5.5.2 -Dpackaging=pom -Dfile=common-parent-5.5.2.pom -DpomFile=common-parent-5.5.2.pom
mvn install:install-file -DgroupId=io.confluent -DartifactId=common-config -Dversion=5.5.2 -Dpackaging=jar -Dfile=common-config-5.5.2.jar -DpomFile=common-config-5.5.2.pom
mvn install:install-file -DgroupId=io.confluent -DartifactId=common-utils -Dversion=5.5.2 -Dpackaging=jar -Dfile=common-utils-5.5.2.jar -DpomFile=common-utils-5.5.2.pom
At 2021-01-25 16:36:12, "Matthias Pohl" <ma...@ververica.com> wrote:
Hi Smile,
you missed installing the pom provided by mvnrepository.org [1]. Maven will install a basic pom if none is provided [2]. This basic pom file will not include any dependencies. You should be able to fix your problem by running your command above but adding the -DpomFile property with the pom file provided in [1]:
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-avro-serializer-5.5.2.jar -DpomFile=kafka-avro-serializer-5.5.2.pom
[1] https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom
[2] https://maven.apache.org/plugins/maven-install-plugin/install-file-mojo.html#pomFile
On Mon, Jan 25, 2021 at 8:25 AM Smile@LETTers <le...@163.com> wrote:
Hi Matthias,
Sorry for my miss leading. I mean kafka-schema-serializer rather than kafka-avro-serializer.
io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe is in kafka-schema-serializer and kafka-schema-serializer should be a dependency of kafka-avro-serializer according to their pom.xml files(see [1], [2]).
I couldn't resolve a valid kafka-avro-serializer.jar in my mirror so I downloaded it manually from [3] and installed it using:
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-avro-serializer-5.5.2.jar
After that, I tried to build Flink and got the above exceptions. Then I tried to add the dependency of kafka-schema-serializer to flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml (also manually installed it to my local maven repo) and everything went well.
I also tried to remove it from the pom.xml after installing, and the exception came back.
Maybe there was something wrong with the manually-installed kafka-avro-serializer?
[1]. https://mvnrepository.com/artifact/io.confluent/kafka-schema-serializer/usages
[2]. https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom
[3]. https://mvnrepository.com/artifact/io.confluent/kafka-avro-serializer
At 2021-01-22 21:22:51, "Matthias Pohl" <ma...@ververica.com> wrote:
Hi Smile,
Have you used a clean checkout? I second Robert's statement considering that the dependency you're talking about is already part of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml. It also has the correct scope set both in master and release-1.12.
Best,
Matthias
On Fri, Jan 22, 2021 at 10:04 AM Smile@LETTers <le...@163.com> wrote:
Yes, I've tried from both the root directory and the sub module. Neither or them works. And the error messages are the same.
At 2021-01-21 23:22:12, "Robert Metzger" <rm...@apache.org> wrote:
Since our CI system is able to build Flink, I believe it's a local issue.
Are you sure that the build is failing when you build Flink from the root directory (not calling maven from within a maven module?)
On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <le...@163.com> wrote:
Hi,
I got an error when tried to compile & package Flink (version 1.12 & current master).
It can be reproduced by run 'mvn clean test' under flink-end-to-end-tests/flink-end-to-end-tests-common-kafka.
It seems that a necessary dependency for test scope was missing and some classes can not be found.
After adding the dependency kafka-avro-serializer to the pom of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes well.
And I just wonder that is this a bug or I missed some local setting?
Best regards.
Smile
Error logs attached:
[INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka >--------
[INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading ...
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ flink-end-to-end-tests-common-kafka ---
[INFO] Deleting /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using plugin classloader, includes GMavenPlus classpath.
[INFO] Using Groovy 2.5.12 to perform execute.
includes: org.apache.flink.tests.util.categories.Dummy
excludes:
[INFO]
[INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Highest basedir set to: /Users/smile/Downloads/W/code/flink/apache/master/flink
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 5 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java uses unchecked or unsafe operations.
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 4 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.842 s
[INFO] Finished at: 2021-01-19T18:04:39+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project flink-end-to-end-tests-common-kafka: Compilation failure
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
[ERROR] class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Re: Re: Re: Test failed in flink-end-to-end-tests/flink-end-to-end-tests-common-kafka
Posted by Matthias Pohl <ma...@ververica.com>.
Hi Smile,
you missed installing the pom provided by mvnrepository.org [1]. Maven will
install a basic pom if none is provided [2]. This basic pom file will not
include any dependencies. You should be able to fix your problem by running
your command above but adding the -DpomFile property with the pom file
provided in [1]:
mvn install:install-file -DgroupId=io.confluent
-DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar
-Dfile=kafka-avro-serializer-5.5.2.jar
-DpomFile=kafka-avro-serializer-5.5.2.pom
[1]
https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom
[2]
https://maven.apache.org/plugins/maven-install-plugin/install-file-mojo.html#pomFile
On Mon, Jan 25, 2021 at 8:25 AM Smile@LETTers <le...@163.com> wrote:
> Hi Matthias,
> Sorry for my miss leading. I mean kafka-schema-serializer rather than
> kafka-avro-serializer.
>
> io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe is in
> kafka-schema-serializer and kafka-schema-serializer should be a dependency
> of kafka-avro-serializer according to their pom.xml files(see [1], [2]).
> I couldn't resolve a valid kafka-avro-serializer.jar in my mirror so I
> downloaded it manually from [3] and installed it using:
> mvn install:install-file -DgroupId=io.confluent
> -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar
> -Dfile=kafka-avro-serializer-5.5.2.jar
> After that, I tried to build Flink and got the above exceptions. Then I
> tried to add the dependency of kafka-schema-serializer to
> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml (also
> manually installed it to my local maven repo) and everything went well.
> I also tried to remove it from the pom.xml after installing, and the
> exception came back.
> Maybe there was something wrong with the manually-installed
> kafka-avro-serializer?
>
>
> [1].
> https://mvnrepository.com/artifact/io.confluent/kafka-schema-serializer/usages
> [2].
> https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom
> [3]. https://mvnrepository.com/artifact/io.confluent/kafka-avro-serializer
>
> At 2021-01-22 21:22:51, "Matthias Pohl" <ma...@ververica.com> wrote:
>
> Hi Smile,
> Have you used a clean checkout? I second Robert's statement considering
> that the dependency you're talking about is already part
> of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml. It
> also has the correct scope set both in master and release-1.12.
>
> Best,
> Matthias
>
> On Fri, Jan 22, 2021 at 10:04 AM Smile@LETTers <le...@163.com>
> wrote:
>
>> Yes, I've tried from both the root directory and the sub module. Neither
>> or them works. And the error messages are the same.
>>
>> At 2021-01-21 23:22:12, "Robert Metzger" <rm...@apache.org> wrote:
>>
>> Since our CI system is able to build Flink, I believe it's a local issue.
>>
>> Are you sure that the build is failing when you build Flink from the root
>> directory (not calling maven from within a maven module?)
>>
>> On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <le...@163.com>
>> wrote:
>>
>>> Hi,
>>> I got an error when tried to compile & package Flink (version 1.12 &
>>> current master).
>>> It can be reproduced by run 'mvn clean test' under
>>> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka.
>>>
>>> It seems that a necessary dependency for test scope was missing and some
>>> classes can not be found.
>>> After adding the dependency kafka-avro-serializer to the pom of
>>> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes
>>> well.
>>>
>>> And I just wonder that is this a bug or I missed some local setting?
>>>
>>> Best regards.
>>> Smile
>>>
>>> ------------------------------
>>> Error logs attached:
>>>
>>>
>>> [INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka
>>> >--------
>>> [INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT
>>> [INFO] --------------------------------[ jar
>>> ]---------------------------------
>>> Downloading ...
>>> [INFO]
>>> [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO] Deleting
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target
>>> [INFO]
>>> [INFO] --- maven-checkstyle-plugin:2.17:check (validate) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce
>>> (enforce-maven-version) @ flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml)
>>> @ flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO] Using plugin classloader, includes GMavenPlus classpath.
>>> [INFO] Using Groovy 2.5.12 to perform execute.
>>> includes: org.apache.flink.tests.util.categories.Dummy
>>> excludes:
>>> [INFO]
>>> [INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO] Highest basedir set to:
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink
>>> [INFO]
>>> [INFO] --- maven-remote-resources-plugin:1.5:process
>>> (process-resource-bundles) @ flink-end-to-end-tests-common-kafka ---
>>> [INFO]
>>> [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>> [INFO] skip non existing resourceDirectory
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources
>>> [INFO] Copying 3 resources
>>> [INFO]
>>> [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @
>>> flink-end-to-end-tests-common-kafka ---
>>> [INFO] Compiling 5 source files to
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes
>>> [INFO]
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java:
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java
>>> uses unchecked or unsafe operations.
>>> [INFO]
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java:
>>> Recompile with -Xlint:unchecked for details.
>>> [INFO]
>>> [INFO] --- maven-resources-plugin:3.1.0:testResources
>>> (default-testResources) @ flink-end-to-end-tests-common-kafka ---
>>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>> [INFO] Copying 2 resources
>>> [INFO] Copying 3 resources
>>> [INFO]
>>> [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile)
>>> @ flink-end-to-end-tests-common-kafka ---
>>> [INFO] Compiling 4 source files to
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes
>>> [INFO] -------------------------------------------------------------
>>> [ERROR] COMPILATION ERROR :
>>> [INFO] -------------------------------------------------------------
>>> [ERROR]
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20]
>>> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
>>> class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
>>> not found
>>> [INFO] 1 error
>>> [INFO] -------------------------------------------------------------
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] BUILD FAILURE
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Total time: 29.842 s
>>> [INFO] Finished at: 2021-01-19T18:04:39+08:00
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] Failed to execute goal
>>> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
>>> (default-testCompile) on project flink-end-to-end-tests-common-kafka:
>>> Compilation failure
>>> [ERROR]
>>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20]
>>> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
>>> [ERROR] class file for
>>> io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
>>> [ERROR]
>>> [ERROR] -> [Help 1]
>>> [ERROR]
>>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>>> -e switch.
>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>> [ERROR]
>>> [ERROR] For more information about the errors and possible solutions,
>>> please read the following articles:
>>> [ERROR] [Help 1]
>>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>>>
>>
>
Re:Re: Re: Test failed in
flink-end-to-end-tests/flink-end-to-end-tests-common-kafka
Posted by "Smile@LETTers" <le...@163.com>.
Hi Matthias,
Sorry for my miss leading. I mean kafka-schema-serializer rather than kafka-avro-serializer.
io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe is in kafka-schema-serializer and kafka-schema-serializer should be a dependency of kafka-avro-serializer according to their pom.xml files(see [1], [2]).
I couldn't resolve a valid kafka-avro-serializer.jar in my mirror so I downloaded it manually from [3] and installed it using:
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-avro-serializer-5.5.2.jar
After that, I tried to build Flink and got the above exceptions. Then I tried to add the dependency of kafka-schema-serializer to flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml (also manually installed it to my local maven repo) and everything went well.
I also tried to remove it from the pom.xml after installing, and the exception came back.
Maybe there was something wrong with the manually-installed kafka-avro-serializer?
[1]. https://mvnrepository.com/artifact/io.confluent/kafka-schema-serializer/usages
[2]. https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom
[3]. https://mvnrepository.com/artifact/io.confluent/kafka-avro-serializer
At 2021-01-22 21:22:51, "Matthias Pohl" <ma...@ververica.com> wrote:
Hi Smile,
Have you used a clean checkout? I second Robert's statement considering that the dependency you're talking about is already part of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml. It also has the correct scope set both in master and release-1.12.
Best,
Matthias
On Fri, Jan 22, 2021 at 10:04 AM Smile@LETTers <le...@163.com> wrote:
Yes, I've tried from both the root directory and the sub module. Neither or them works. And the error messages are the same.
At 2021-01-21 23:22:12, "Robert Metzger" <rm...@apache.org> wrote:
Since our CI system is able to build Flink, I believe it's a local issue.
Are you sure that the build is failing when you build Flink from the root directory (not calling maven from within a maven module?)
On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <le...@163.com> wrote:
Hi,
I got an error when tried to compile & package Flink (version 1.12 & current master).
It can be reproduced by run 'mvn clean test' under flink-end-to-end-tests/flink-end-to-end-tests-common-kafka.
It seems that a necessary dependency for test scope was missing and some classes can not be found.
After adding the dependency kafka-avro-serializer to the pom of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes well.
And I just wonder that is this a bug or I missed some local setting?
Best regards.
Smile
Error logs attached:
[INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka >--------
[INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading ...
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ flink-end-to-end-tests-common-kafka ---
[INFO] Deleting /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using plugin classloader, includes GMavenPlus classpath.
[INFO] Using Groovy 2.5.12 to perform execute.
includes: org.apache.flink.tests.util.categories.Dummy
excludes:
[INFO]
[INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Highest basedir set to: /Users/smile/Downloads/W/code/flink/apache/master/flink
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 5 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java uses unchecked or unsafe operations.
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 4 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.842 s
[INFO] Finished at: 2021-01-19T18:04:39+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project flink-end-to-end-tests-common-kafka: Compilation failure
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
[ERROR] class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Re: Re: Test failed in flink-end-to-end-tests/flink-end-to-end-tests-common-kafka
Posted by Matthias Pohl <ma...@ververica.com>.
Hi Smile,
Have you used a clean checkout? I second Robert's statement considering
that the dependency you're talking about is already part
of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml. It
also has the correct scope set both in master and release-1.12.
Best,
Matthias
On Fri, Jan 22, 2021 at 10:04 AM Smile@LETTers <le...@163.com>
wrote:
> Yes, I've tried from both the root directory and the sub module. Neither
> or them works. And the error messages are the same.
>
> At 2021-01-21 23:22:12, "Robert Metzger" <rm...@apache.org> wrote:
>
> Since our CI system is able to build Flink, I believe it's a local issue.
>
> Are you sure that the build is failing when you build Flink from the root
> directory (not calling maven from within a maven module?)
>
> On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <le...@163.com>
> wrote:
>
>> Hi,
>> I got an error when tried to compile & package Flink (version 1.12 &
>> current master).
>> It can be reproduced by run 'mvn clean test' under
>> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka.
>>
>> It seems that a necessary dependency for test scope was missing and some
>> classes can not be found.
>> After adding the dependency kafka-avro-serializer to the pom of
>> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes
>> well.
>>
>> And I just wonder that is this a bug or I missed some local setting?
>>
>> Best regards.
>> Smile
>>
>> ------------------------------
>> Error logs attached:
>>
>>
>> [INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka
>> >--------
>> [INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT
>> [INFO] --------------------------------[ jar
>> ]---------------------------------
>> Downloading ...
>> [INFO]
>> [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO] Deleting
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target
>> [INFO]
>> [INFO] --- maven-checkstyle-plugin:2.17:check (validate) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version)
>> @ flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml)
>> @ flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO] Using plugin classloader, includes GMavenPlus classpath.
>> [INFO] Using Groovy 2.5.12 to perform execute.
>> includes: org.apache.flink.tests.util.categories.Dummy
>> excludes:
>> [INFO]
>> [INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO] Highest basedir set to:
>> /Users/smile/Downloads/W/code/flink/apache/master/flink
>> [INFO]
>> [INFO] --- maven-remote-resources-plugin:1.5:process
>> (process-resource-bundles) @ flink-end-to-end-tests-common-kafka ---
>> [INFO]
>> [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] skip non existing resourceDirectory
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources
>> [INFO] Copying 3 resources
>> [INFO]
>> [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @
>> flink-end-to-end-tests-common-kafka ---
>> [INFO] Compiling 5 source files to
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes
>> [INFO]
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java:
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java
>> uses unchecked or unsafe operations.
>> [INFO]
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java:
>> Recompile with -Xlint:unchecked for details.
>> [INFO]
>> [INFO] --- maven-resources-plugin:3.1.0:testResources
>> (default-testResources) @ flink-end-to-end-tests-common-kafka ---
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] Copying 2 resources
>> [INFO] Copying 3 resources
>> [INFO]
>> [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile)
>> @ flink-end-to-end-tests-common-kafka ---
>> [INFO] Compiling 4 source files to
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes
>> [INFO] -------------------------------------------------------------
>> [ERROR] COMPILATION ERROR :
>> [INFO] -------------------------------------------------------------
>> [ERROR]
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20]
>> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
>> class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
>> not found
>> [INFO] 1 error
>> [INFO] -------------------------------------------------------------
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 29.842 s
>> [INFO] Finished at: 2021-01-19T18:04:39+08:00
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
>> (default-testCompile) on project flink-end-to-end-tests-common-kafka:
>> Compilation failure
>> [ERROR]
>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20]
>> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
>> [ERROR] class file for
>> io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
>> [ERROR]
>> [ERROR] -> [Help 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>> -e switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>>
>
Re:Re: Test failed in
flink-end-to-end-tests/flink-end-to-end-tests-common-kafka
Posted by "Smile@LETTers" <le...@163.com>.
Yes, I've tried from both the root directory and the sub module. Neither or them works. And the error messages are the same.
At 2021-01-21 23:22:12, "Robert Metzger" <rm...@apache.org> wrote:
Since our CI system is able to build Flink, I believe it's a local issue.
Are you sure that the build is failing when you build Flink from the root directory (not calling maven from within a maven module?)
On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <le...@163.com> wrote:
Hi,
I got an error when tried to compile & package Flink (version 1.12 & current master).
It can be reproduced by run 'mvn clean test' under flink-end-to-end-tests/flink-end-to-end-tests-common-kafka.
It seems that a necessary dependency for test scope was missing and some classes can not be found.
After adding the dependency kafka-avro-serializer to the pom of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes well.
And I just wonder that is this a bug or I missed some local setting?
Best regards.
Smile
Error logs attached:
[INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka >--------
[INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading ...
[INFO]
[INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ flink-end-to-end-tests-common-kafka ---
[INFO] Deleting /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using plugin classloader, includes GMavenPlus classpath.
[INFO] Using Groovy 2.5.12 to perform execute.
includes: org.apache.flink.tests.util.categories.Dummy
excludes:
[INFO]
[INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @ flink-end-to-end-tests-common-kafka ---
[INFO] Highest basedir set to: /Users/smile/Downloads/W/code/flink/apache/master/flink
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ flink-end-to-end-tests-common-kafka ---
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 5 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java uses unchecked or unsafe operations.
[INFO] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ flink-end-to-end-tests-common-kafka ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ flink-end-to-end-tests-common-kafka ---
[INFO] Compiling 4 source files to /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.842 s
[INFO] Finished at: 2021-01-19T18:04:39+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project flink-end-to-end-tests-common-kafka: Compilation failure
[ERROR] /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
[ERROR] class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
Re: Test failed in flink-end-to-end-tests/flink-end-to-end-tests-common-kafka
Posted by Robert Metzger <rm...@apache.org>.
Since our CI system is able to build Flink, I believe it's a local issue.
Are you sure that the build is failing when you build Flink from the root
directory (not calling maven from within a maven module?)
On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <le...@163.com>
wrote:
> Hi,
> I got an error when tried to compile & package Flink (version 1.12 &
> current master).
> It can be reproduced by run 'mvn clean test' under
> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka.
>
> It seems that a necessary dependency for test scope was missing and some
> classes can not be found.
> After adding the dependency kafka-avro-serializer to the pom of
> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes
> well.
>
> And I just wonder that is this a bug or I missed some local setting?
>
> Best regards.
> Smile
>
> ------------------------------
> Error logs attached:
>
>
> [INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka
> >--------
> [INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT
> [INFO] --------------------------------[ jar
> ]---------------------------------
> Downloading ...
> [INFO]
> [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @
> flink-end-to-end-tests-common-kafka ---
> [INFO] Deleting
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.17:check (validate) @
> flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @
> flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven-version)
> @ flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @
> flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml) @
> flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @
> flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @
> flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @
> flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @
> flink-end-to-end-tests-common-kafka ---
> [INFO] Using plugin classloader, includes GMavenPlus classpath.
> [INFO] Using Groovy 2.5.12 to perform execute.
> includes: org.apache.flink.tests.util.categories.Dummy
> excludes:
> [INFO]
> [INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @
> flink-end-to-end-tests-common-kafka ---
> [INFO] Highest basedir set to:
> /Users/smile/Downloads/W/code/flink/apache/master/flink
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.5:process
> (process-resource-bundles) @ flink-end-to-end-tests-common-kafka ---
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @
> flink-end-to-end-tests-common-kafka ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @
> flink-end-to-end-tests-common-kafka ---
> [INFO] Compiling 5 source files to
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes
> [INFO]
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java:
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java
> uses unchecked or unsafe operations.
> [INFO]
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java:
> Recompile with -Xlint:unchecked for details.
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:testResources
> (default-testResources) @ flink-end-to-end-tests-common-kafka ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 2 resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @
> flink-end-to-end-tests-common-kafka ---
> [INFO] Compiling 4 source files to
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes
> [INFO] -------------------------------------------------------------
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20]
> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
> class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
> not found
> [INFO] 1 error
> [INFO] -------------------------------------------------------------
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 29.842 s
> [INFO] Finished at: 2021-01-19T18:04:39+08:00
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile
> (default-testCompile) on project flink-end-to-end-tests-common-kafka:
> Compilation failure
> [ERROR]
> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20]
> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe
> [ERROR] class file for
> io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found
> [ERROR]
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>
>