You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "luoyuxia (Jira)" <ji...@apache.org> on 2022/04/08 12:29:00 UTC
[jira] [Updated] (FLINK-27145) The code generated for agg function accepts empty parameters can't be compiled
[ https://issues.apache.org/jira/browse/FLINK-27145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
luoyuxia updated FLINK-27145:
-----------------------------
Description:
In batch mode, when I try to call Hive's `count` function(GenericUDAFCount) using the following sql:
{code:java}
select count(*) from src;
{code}
It'll throw the exception "Unexpected token ")" in primary" while compling the generated code.
It happens in the following generated code:
{code:java}
function_org$apache$flink$table$planner$utils$xxx.accumulate(
acc$7,); {code}
The reason is the following code in method `AggCodeGenHelper#genAccumulateFlatAggregateBuffer`
{code:java}
s"""
|$externalAccTypeTerm $externalAccTerm = $externalAccCode;
|${functionIdentifiers(function)}.accumulate(
| $externalAccTerm,
| ${operandTerms.mkString(", ")}); // but operandTerms can be empty list
|$aggBufferName = ${genToInternalConverter(ctx, externalAccType)(externalAccTerm)};
|${aggBufferExpr.nullTerm} = false;
""".stripMargin {code}
In this case, for count(*), it'll regards as an agg function with empty parameter in Hive.
So, the exception happens.
Although the agg function with empty parameter is rare, it may happens. We should make the code more rubst.
was:
In batch mode, when I try to call Hive's `count` function(GenericUDAFCount) using the following sql:
{code:java}
select count(*) from src;
{code}
It'll throw the exception "Unexpected token ")" in primary" while compling the generated code.
It happens in the following generated code:
{code:java}
function_org$apache$flink$table$planner$utils$xxx.accumulate(
acc$7,); {code}
The reason is the following code in method `AggCodeGenHelper#genAccumulateFlatAggregateBuffer`
{code:java}
s"""
|$externalAccTypeTerm $externalAccTerm = $externalAccCode;
|${functionIdentifiers(function)}.accumulate(
| $externalAccTerm,
| ${operandTerms.mkString(", ")}); // but operandTerms can be empty list
|$aggBufferName = ${genToInternalConverter(ctx, externalAccType)(externalAccTerm)};
|${aggBufferExpr.nullTerm} = false;
""".stripMargin {code}
In this case, for `count(*)`, it'll regards as an agg function with empty parameter in Hive.
So, the exception happens.
Although the agg function with empty parameter is rare, it may happens. We should make the code more rubst.
> The code generated for agg function accepts empty parameters can't be compiled
> ------------------------------------------------------------------------------
>
> Key: FLINK-27145
> URL: https://issues.apache.org/jira/browse/FLINK-27145
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Reporter: luoyuxia
> Priority: Major
> Fix For: 1.16.0
>
>
> In batch mode, when I try to call Hive's `count` function(GenericUDAFCount) using the following sql:
> {code:java}
> select count(*) from src;
> {code}
> It'll throw the exception "Unexpected token ")" in primary" while compling the generated code.
> It happens in the following generated code:
> {code:java}
> function_org$apache$flink$table$planner$utils$xxx.accumulate(
> acc$7,); {code}
>
> The reason is the following code in method `AggCodeGenHelper#genAccumulateFlatAggregateBuffer`
> {code:java}
> s"""
> |$externalAccTypeTerm $externalAccTerm = $externalAccCode;
> |${functionIdentifiers(function)}.accumulate(
> | $externalAccTerm,
> | ${operandTerms.mkString(", ")}); // but operandTerms can be empty list
> |$aggBufferName = ${genToInternalConverter(ctx, externalAccType)(externalAccTerm)};
> |${aggBufferExpr.nullTerm} = false;
> """.stripMargin {code}
> In this case, for count(*), it'll regards as an agg function with empty parameter in Hive.
> So, the exception happens.
> Although the agg function with empty parameter is rare, it may happens. We should make the code more rubst.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)