You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Debski <a....@avsystem.com> on 2017/07/07 08:11:56 UTC
elastic-search connector for 1.3.1 not working for local
environemnt
Today I added flink-connector-ealsticsearch to my project and when I started
flink engine using createLocalEnvironmentWithWebUI method I got following
exception
xception in thread "main" java.lang.IllegalArgumentException
at org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown
Source)
at org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown
Source)
at
org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)
at
org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)
at
org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)
at
org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:670)
at
org.apache.flink.streaming.api.scala.DataStream.clean(DataStream.scala:1061)
at
org.apache.flink.streaming.api.scala.DataStream.map(DataStream.scala:564)
at Test$.main(Test.scala:61)
at Test.main(Test.scala)
I found out that someone had similar problem in the past:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/asm-IllegalArgumentException-with-1-0-0-td5411.html
and problem seems to be with shaded asm dependency in Flink jars.
I've found out that there will be some improvements soon to this situation:
https://issues.apache.org/jira/browse/FLINK-6529 but currently it seems that
ES connector is not very usable during development.
What surprises me is that it seems that asm dependency shaded in ES
connector is from one of ASM 4.X.X releases because of contents of
ClassVisitor constructor:
public ClassVisitor(int var1, ClassVisitor var2) {
if(var1 != 262144) {
throw new IllegalArgumentException();
} else {
this.api = var1;
this.cv = var2;
}
}
and in 5.X.X versions it looks like
public ClassVisitor(int var1, ClassVisitor var2) {
if(var1 != 262144 && var1 != 327680) {
throw new IllegalArgumentException();
} else {
this.api = var1;
this.cv = var2;
}
}
Those magic numbers are in fact constants from ASM called ASM4 (262144) and
ASM5 (327680).
I grepped through GitHub repo and there are no traces of ASM 4.X.X anywhere
so I suspect that there is something wrong with published ES connector jar.
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/elastic-search-connector-for-1-3-1-not-working-for-local-environemnt-tp14140.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: elastic-search connector for 1.3.1 not working for local
environemnt
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
Thanks a lot for figuring this out! I opened a Jira issue: https://issues.apache.org/jira/browse/FLINK-7133 <https://issues.apache.org/jira/browse/FLINK-7133>
Best,
Aljoscha
> On 7. Jul 2017, at 10:53, Debski <a....@avsystem.com> wrote:
>
> I've found out the source of the problem when I build flink locally.
> elastic-search base depends on (by default) ES version 1.7.1 that depends on
> asm 4.1 and that version is shaded to elasticsearch-base-jar. I tried to set
> elasticsearch.version property in Maven to 5.1.2 (the same as elasticsearch5
> connector) but then elasticsearch-base does not compile:
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
> (default-testCompile) on project flink-connector-elasticsearch-base_2.11:
> Compilation failure
> [ERROR]
> /home/adebski/Downloads/flink-release-1.3.1/flink-connectors/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java:[491,92]
> no suitable constructor found for
> BulkItemResponse(int,java.lang.String,org.elasticsearch.action.ActionResponse)
> [ERROR] constructor
> org.elasticsearch.action.bulk.BulkItemResponse.BulkItemResponse(int,java.lang.String,org.elasticsearch.action.DocWriteResponse)
> is not applicable
> [ERROR] (argument mismatch; org.elasticsearch.action.ActionResponse cannot
> be converted to org.elasticsearch.action.DocWriteResponse)
> [ERROR] constructor
> org.elasticsearch.action.bulk.BulkItemResponse.BulkItemResponse(int,java.lang.String,org.elasticsearch.action.bulk.BulkItemResponse.Failure)
> is not applicable
> [ERROR] (argument mismatch; org.elasticsearch.action.ActionResponse cannot
> be converted to org.elasticsearch.action.bulk.BulkItemResponse.Failure)
>
>
>
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/elastic-search-connector-for-1-3-1-not-working-for-local-environemnt-tp14140p14141.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: elastic-search connector for 1.3.1 not working for local
environemnt
Posted by Debski <a....@avsystem.com>.
I've found out the source of the problem when I build flink locally.
elastic-search base depends on (by default) ES version 1.7.1 that depends on
asm 4.1 and that version is shaded to elasticsearch-base-jar. I tried to set
elasticsearch.version property in Maven to 5.1.2 (the same as elasticsearch5
connector) but then elasticsearch-base does not compile:
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
(default-testCompile) on project flink-connector-elasticsearch-base_2.11:
Compilation failure
[ERROR]
/home/adebski/Downloads/flink-release-1.3.1/flink-connectors/flink-connector-elasticsearch-base/src/test/java/org/apache/flink/streaming/connectors/elasticsearch/ElasticsearchSinkBaseTest.java:[491,92]
no suitable constructor found for
BulkItemResponse(int,java.lang.String,org.elasticsearch.action.ActionResponse)
[ERROR] constructor
org.elasticsearch.action.bulk.BulkItemResponse.BulkItemResponse(int,java.lang.String,org.elasticsearch.action.DocWriteResponse)
is not applicable
[ERROR] (argument mismatch; org.elasticsearch.action.ActionResponse cannot
be converted to org.elasticsearch.action.DocWriteResponse)
[ERROR] constructor
org.elasticsearch.action.bulk.BulkItemResponse.BulkItemResponse(int,java.lang.String,org.elasticsearch.action.bulk.BulkItemResponse.Failure)
is not applicable
[ERROR] (argument mismatch; org.elasticsearch.action.ActionResponse cannot
be converted to org.elasticsearch.action.bulk.BulkItemResponse.Failure)
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/elastic-search-connector-for-1-3-1-not-working-for-local-environemnt-tp14140p14141.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.