You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by 程骥 <ji...@qq.com> on 2017/07/26 01:39:48 UTC
回复： How can I set charset for flink sql？
OK，thanks for remind me.
My sql like this(contain a Chinese word):
SELECT
'HIGH' AS LEVEL,
'Firewall uplink bandwidth exception:greater than 10000' AS content,
`system.process.username`,
`system.process.memory.rss.bytes`
FROM
test
WHERE
`system.process.username` LIKE '%高危%'
AND
`system.process.memory.rss.bytes` > 10000
Get exception when I submit the job to cluster.
Caused by: org.apache.calcite.runtime.CalciteException: Failed to encode '%高危%' in character set 'ISO-8859-1'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_45]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) ~[na:1.8.0_45]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) ~[na:1.8.0_45]
at java.lang.reflect.Constructor.newInstance(Unknown Source) ~[na:1.8.0_45]
at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.util.NlsString.<init>(NlsString.java:81) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:864) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.rex.RexBuilder.makeCharLiteral(RexBuilder.java:1051) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertLiteral(SqlNodeToRexConverterImpl.java:117) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4408) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3787) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql.SqlLiteral.accept(SqlLiteral.java:427) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4321) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.StandardConvertletTable.convertExpressionList(StandardConvertletTable.java:968) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:944) ~[flink-table_2.11-1.3.1.jar:1.3.1]
at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(StandardConvertletTable.java:928) ~[flink-table_2.11-1.3.1.jar:1.3.1]
... 50 common frames omitted
Is there anyone tell me how to deal with it,thanks!
------------------ 原始邮件 ------------------
发件人: "Nico Kruber";<ni...@data-artisans.com>;
发送时间: 2017年7月25日(星期二) 晚上11:48
收件人: "user"<us...@flink.apache.org>;
抄送: "程骥"<ji...@qq.com>;
主题: Re: How can I set charset for flink sql？
Please, for the sake of making your email searchable, do not post stack traces
as screenshots but rather text into your email.
On Tuesday, 25 July 2017 12:18:56 CEST 程骥 wrote:
> My sql like this(contain a Chinese word)
>
> Get exception when I submit the job to cluster.
>
>
>
> Is there anyone tell me how to deal with it,thanks!
Re: How can I set charset for flink sql？
Posted by Fabian Hueske <fh...@gmail.com>.
As Timo proposed, I would implement a Scalar user-defined function which
returns a boolean and use that instead of LIKE.
Have a look here [1].
Best, Fabian
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/table/udfs.html#scalar-functions
2017-07-26 3:47 GMT+02:00 Ted Yu <yu...@gmail.com>:
> Logged CALCITE-1903 for this bug.
>
> FYI
>
> On Tue, Jul 25, 2017 at 6:39 PM, 程骥 <ji...@qq.com> wrote:
>
>> OK，thanks for remind me.
>>
>> My sql like this(contain a Chinese word):
>>
>> SELECT
>> 'HIGH' AS LEVEL,
>> 'Firewall uplink bandwidth exception:greater than 10000' AS content,
>> `system.process.username`,
>> `system.process.memory.rss.bytes`
>> FROM
>> test
>> WHERE
>> `system.process.username` LIKE '%高危%'
>> AND
>> `system.process.memory.rss.bytes` > 10000
>>
>> Get exception when I submit the job to cluster.
>>
>> Caused by: org.apache.calcite.runtime.CalciteException: Failed to encode
>> '%高危%' in character set 'ISO-8859-1'
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> ~[na:1.8.0_45]
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
>> ~[na:1.8.0_45]
>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
>> Source) ~[na:1.8.0_45]
>> at java.lang.reflect.Constructor.newInstance(Unknown Source)
>> ~[na:1.8.0_45]
>> at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.util.NlsString.<init>(NlsString.java:81)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:864)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.rex.RexBuilder.makeCharLiteral(RexBuilder.java:1051)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convert
>> Literal(SqlNodeToRexConverterImpl.java:117)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4408)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3787)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.sql.SqlLiteral.accept(SqlLiteral.java:427)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.conv
>> ertExpression(SqlToRelConverter.java:4321) ~[flink-table_2.11-1.3.1.jar:1
>> .3.1]
>> at org.apache.calcite.sql2rel.StandardConvertletTable.convertEx
>> pressionList(StandardConvertletTable.java:968)
>> ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCa
>> ll(StandardConvertletTable.java:944) ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCa
>> ll(StandardConvertletTable.java:928) ~[flink-table_2.11-1.3.1.jar:1.3.1]
>> ... 50 common frames omitted
>>
>> Is there anyone tell me how to deal with it,thanks!
>>
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Nico Kruber";<ni...@data-artisans.com>;
>> *发送时间:* 2017年7月25日(星期二) 晚上11:48
>> *收件人:* "user"<us...@flink.apache.org>;
>> *抄送:* "程骥"<ji...@qq.com>;
>> *主题:* Re: How can I set charset for flink sql？
>>
>> Please, for the sake of making your email searchable, do not post stack
>> traces
>> as screenshots but rather text into your email.
>>
>> On Tuesday, 25 July 2017 12:18:56 CEST 程骥 wrote:
>> > My sql like this(contain a Chinese word)
>> >
>> > Get exception when I submit the job to cluster.
>> >
>> >
>> >
>> > Is there anyone tell me how to deal with it,thanks!
>>
>>
>>
>
Re: How can I set charset for flink sql？
Posted by Ted Yu <yu...@gmail.com>.
Logged CALCITE-1903 for this bug.
FYI
On Tue, Jul 25, 2017 at 6:39 PM, 程骥 <ji...@qq.com> wrote:
> OK，thanks for remind me.
>
> My sql like this(contain a Chinese word):
>
> SELECT
> 'HIGH' AS LEVEL,
> 'Firewall uplink bandwidth exception:greater than 10000' AS content,
> `system.process.username`,
> `system.process.memory.rss.bytes`
> FROM
> test
> WHERE
> `system.process.username` LIKE '%高危%'
> AND
> `system.process.memory.rss.bytes` > 10000
>
> Get exception when I submit the job to cluster.
>
> Caused by: org.apache.calcite.runtime.CalciteException: Failed to encode
> '%高危%' in character set 'ISO-8859-1'
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> ~[na:1.8.0_45]
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
> ~[na:1.8.0_45]
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
> Source) ~[na:1.8.0_45]
> at java.lang.reflect.Constructor.newInstance(Unknown Source)
> ~[na:1.8.0_45]
> at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.util.NlsString.<init>(NlsString.java:81)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.rex.RexBuilder.makeLiteral(RexBuilder.java:864)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.rex.RexBuilder.makeCharLiteral(RexBuilder.java:1051)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql2rel.SqlNodeToRexConverterImpl.convertLiteral(
> SqlNodeToRexConverterImpl.java:117) ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.
> visit(SqlToRelConverter.java:4408) ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.
> visit(SqlToRelConverter.java:3787) ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql.SqlLiteral.accept(SqlLiteral.java:427)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.
> convertExpression(SqlToRelConverter.java:4321)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql2rel.StandardConvertletTable.
> convertExpressionList(StandardConvertletTable.java:968)
> ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(
> StandardConvertletTable.java:944) ~[flink-table_2.11-1.3.1.jar:1.3.1]
> at org.apache.calcite.sql2rel.StandardConvertletTable.convertCall(
> StandardConvertletTable.java:928) ~[flink-table_2.11-1.3.1.jar:1.3.1]
> ... 50 common frames omitted
>
> Is there anyone tell me how to deal with it,thanks!
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Nico Kruber";<ni...@data-artisans.com>;
> *发送时间:* 2017年7月25日(星期二) 晚上11:48
> *收件人:* "user"<us...@flink.apache.org>;
> *抄送:* "程骥"<ji...@qq.com>;
> *主题:* Re: How can I set charset for flink sql？
>
> Please, for the sake of making your email searchable, do not post stack
> traces
> as screenshots but rather text into your email.
>
> On Tuesday, 25 July 2017 12:18:56 CEST 程骥 wrote:
> > My sql like this(contain a Chinese word)
> >
> > Get exception when I submit the job to cluster.
> >
> >
> >
> > Is there anyone tell me how to deal with it,thanks!
>
>
>