You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Dong-iL, Kim" <ki...@gmail.com> on 2016/08/01 08:18:18 UTC
Scala Table API with Java POJO
I’ve create a program using table API and get an exception like this.
org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
There is an error not in java program, but in scala program.
how can I use java POJO with scala Table API.
Re: Scala Table API with Java POJO
Posted by Timo Walther <tw...@apache.org>.
Ok, then I think I have no better solution than use the Table API of the
upcoming 1.1 release. The Table API has been completely rewritten and
the POJO support is now much better. Maybe you could try the recent 1.1
RC1 release.
Am 01/08/16 um 11:11 schrieb Dong-iL, Kim:
> I\u2019ve tried like this, but not work.
>
> dataSet.as(\u2018id as \u2018id, \u2018amount as \u2018amount)
>
> dataSet.as(\u2018id, \u2018amount)
>
> dataSet.as(\u201cid, amount\u201d)
>
> thanks.
>
>> On Aug 1, 2016, at 6:03 PM, Timo Walther <tw...@apache.org> wrote:
>>
>> I think you need to use ".as()" instead of "toTable()" to supply the field order.
>>
>> Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:
>>> Hi Timo.
>>> I\u2019m using scala API.
>>> There is no error with java API.
>>> my code snippet is this.
>>>
>>> dataSet.toTable
>>> .groupBy(\u201cid")
>>> .select(\u2018id, \u2018amount.sum as \u2018amount)
>>> .where(\u2018amount > 0)
>>> .toDataSet[TestPojo]
>>> .print()
>>>
>>> Thanks.
>>>
>>>> On Aug 1, 2016, at 5:50 PM, Timo Walther <tw...@apache.org> wrote:
>>>>
>>>> Hi Kim,
>>>>
>>>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>>>
>>>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>>>
>>>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>>>
>>>> Timo
>>>>
>>>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>>>> my flink ver is 1.0.3.
>>>>> thanks.
>>>>>
>>>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>>>>>>
>>>>>> I\u2019ve create a program using table API and get an exception like this.
>>>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<\u2026.> is not deterministic.
>>>>>> There is an error not in java program, but in scala program.
>>>>>> how can I use java POJO with scala Table API.
>>>>>>
>>>> --
>>>> Freundliche Gr��e / Kind Regards
>>>>
>>>> Timo Walther
>>>>
>>>> Follow me: @twalthr
>>>> https://www.linkedin.com/in/twalthr
>>>>
>>
>> --
>> Freundliche Gr��e / Kind Regards
>>
>> Timo Walther
>>
>> Follow me: @twalthr
>> https://www.linkedin.com/in/twalthr
>>
--
Freundliche Gr��e / Kind Regards
Timo Walther
Follow me: @twalthr
https://www.linkedin.com/in/twalthr
Re: Scala Table API with Java POJO
Posted by "Dong-iL, Kim" <ki...@gmail.com>.
in org.apache.flink.api.table.plan.PlanTranslator.
val inputType = set.getType().asInstanceOf[CompositeType[A]]
if (!inputType.hasDeterministicFieldOrder && checkDeterministicFields) {
throw new ExpressionException(s"You cannot rename fields upon Table creation: " +
s"Field order of input type $inputType is not deterministic." )
}
when A is a PojoType, hasDeterministicFieldOrder always returns false.
what shall I do using Pojo.
Thanks.
> On Aug 1, 2016, at 6:11 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>
> I’ve tried like this, but not work.
>
> dataSet.as(‘id as ‘id, ‘amount as ‘amount)
>
> dataSet.as(‘id, ‘amount)
>
> dataSet.as(“id, amount”)
>
> thanks.
>
>> On Aug 1, 2016, at 6:03 PM, Timo Walther <tw...@apache.org> wrote:
>>
>> I think you need to use ".as()" instead of "toTable()" to supply the field order.
>>
>> Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:
>>> Hi Timo.
>>> I’m using scala API.
>>> There is no error with java API.
>>> my code snippet is this.
>>>
>>> dataSet.toTable
>>> .groupBy(“id")
>>> .select(‘id, ‘amount.sum as ‘amount)
>>> .where(‘amount > 0)
>>> .toDataSet[TestPojo]
>>> .print()
>>>
>>> Thanks.
>>>
>>>> On Aug 1, 2016, at 5:50 PM, Timo Walther <tw...@apache.org> wrote:
>>>>
>>>> Hi Kim,
>>>>
>>>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>>>
>>>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>>>
>>>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>>>
>>>> Timo
>>>>
>>>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>>>> my flink ver is 1.0.3.
>>>>> thanks.
>>>>>
>>>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>>>>>>
>>>>>> I’ve create a program using table API and get an exception like this.
>>>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>>>>> There is an error not in java program, but in scala program.
>>>>>> how can I use java POJO with scala Table API.
>>>>>>
>>>>
>>>> --
>>>> Freundliche Grüße / Kind Regards
>>>>
>>>> Timo Walther
>>>>
>>>> Follow me: @twalthr
>>>> https://www.linkedin.com/in/twalthr
>>>>
>>
>>
>> --
>> Freundliche Grüße / Kind Regards
>>
>> Timo Walther
>>
>> Follow me: @twalthr
>> https://www.linkedin.com/in/twalthr
>>
>
Re: Scala Table API with Java POJO
Posted by "Dong-iL, Kim" <ki...@gmail.com>.
I’ve tried like this, but not work.
dataSet.as(‘id as ‘id, ‘amount as ‘amount)
dataSet.as(‘id, ‘amount)
dataSet.as(“id, amount”)
thanks.
> On Aug 1, 2016, at 6:03 PM, Timo Walther <tw...@apache.org> wrote:
>
> I think you need to use ".as()" instead of "toTable()" to supply the field order.
>
> Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:
>> Hi Timo.
>> I’m using scala API.
>> There is no error with java API.
>> my code snippet is this.
>>
>> dataSet.toTable
>> .groupBy(“id")
>> .select(‘id, ‘amount.sum as ‘amount)
>> .where(‘amount > 0)
>> .toDataSet[TestPojo]
>> .print()
>>
>> Thanks.
>>
>>> On Aug 1, 2016, at 5:50 PM, Timo Walther <tw...@apache.org> wrote:
>>>
>>> Hi Kim,
>>>
>>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>>
>>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>>
>>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>>
>>> Timo
>>>
>>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>>> my flink ver is 1.0.3.
>>>> thanks.
>>>>
>>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>>>>>
>>>>> I’ve create a program using table API and get an exception like this.
>>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>>>> There is an error not in java program, but in scala program.
>>>>> how can I use java POJO with scala Table API.
>>>>>
>>>
>>> --
>>> Freundliche Grüße / Kind Regards
>>>
>>> Timo Walther
>>>
>>> Follow me: @twalthr
>>> https://www.linkedin.com/in/twalthr
>>>
>
>
> --
> Freundliche Grüße / Kind Regards
>
> Timo Walther
>
> Follow me: @twalthr
> https://www.linkedin.com/in/twalthr
>
Re: Scala Table API with Java POJO
Posted by Timo Walther <tw...@apache.org>.
I think you need to use ".as()" instead of "toTable()" to supply the
field order.
Am 01/08/16 um 10:56 schrieb Dong-iL, Kim:
> Hi Timo.
> I\u2019m using scala API.
> There is no error with java API.
> my code snippet is this.
>
> dataSet.toTable
> .groupBy(\u201cid")
> .select(\u2018id, \u2018amount.sum as \u2018amount)
> .where(\u2018amount > 0)
> .toDataSet[TestPojo]
> .print()
>
> Thanks.
>
>> On Aug 1, 2016, at 5:50 PM, Timo Walther <tw...@apache.org> wrote:
>>
>> Hi Kim,
>>
>> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>>
>> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>>
>> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>>
>> Timo
>>
>> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>>> my flink ver is 1.0.3.
>>> thanks.
>>>
>>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>>>>
>>>> I\u2019ve create a program using table API and get an exception like this.
>>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<\u2026.> is not deterministic.
>>>> There is an error not in java program, but in scala program.
>>>> how can I use java POJO with scala Table API.
>>>>
>>
>> --
>> Freundliche Gr��e / Kind Regards
>>
>> Timo Walther
>>
>> Follow me: @twalthr
>> https://www.linkedin.com/in/twalthr
>>
--
Freundliche Gr��e / Kind Regards
Timo Walther
Follow me: @twalthr
https://www.linkedin.com/in/twalthr
Re: Scala Table API with Java POJO
Posted by "Dong-iL, Kim" <ki...@gmail.com>.
Hi Timo.
I’m using scala API.
There is no error with java API.
my code snippet is this.
dataSet.toTable
.groupBy(“id")
.select(‘id, ‘amount.sum as ‘amount)
.where(‘amount > 0)
.toDataSet[TestPojo]
.print()
Thanks.
> On Aug 1, 2016, at 5:50 PM, Timo Walther <tw...@apache.org> wrote:
>
> Hi Kim,
>
> as the exception says: POJOs have no deterministic field order. You have to specify the order during the DataSet to Table conversion:
>
> Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a, pojoField2 as b");
>
> I hope that helps. Otherwise it would help if you could supply a code snippet of your program.
>
> Timo
>
> Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
>> my flink ver is 1.0.3.
>> thanks.
>>
>>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>>>
>>> I’ve create a program using table API and get an exception like this.
>>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
>>> There is an error not in java program, but in scala program.
>>> how can I use java POJO with scala Table API.
>>>
>
>
> --
> Freundliche Grüße / Kind Regards
>
> Timo Walther
>
> Follow me: @twalthr
> https://www.linkedin.com/in/twalthr
>
Re: Scala Table API with Java POJO
Posted by Timo Walther <tw...@apache.org>.
Hi Kim,
as the exception says: POJOs have no deterministic field order. You have
to specify the order during the DataSet to Table conversion:
Table table = tableEnv.fromDataSet(pojoDataSet, "pojoField as a,
pojoField2 as b");
I hope that helps. Otherwise it would help if you could supply a code
snippet of your program.
Timo
Am 01/08/16 um 10:19 schrieb Dong-iL, Kim:
> my flink ver is 1.0.3.
> thanks.
>
>> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>>
>> I\u2019ve create a program using table API and get an exception like this.
>> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<\u2026.> is not deterministic.
>> There is an error not in java program, but in scala program.
>> how can I use java POJO with scala Table API.
>>
--
Freundliche Gr��e / Kind Regards
Timo Walther
Follow me: @twalthr
https://www.linkedin.com/in/twalthr
Re: Scala Table API with Java POJO
Posted by "Dong-iL, Kim" <ki...@gmail.com>.
my flink ver is 1.0.3.
thanks.
> On Aug 1, 2016, at 5:18 PM, Dong-iL, Kim <ki...@gmail.com> wrote:
>
> I’ve create a program using table API and get an exception like this.
> org.apache.flink.api.table.ExpressionException: You cannot rename fields upon Table creation: Field order of input type PojoType<….> is not deterministic.
> There is an error not in java program, but in scala program.
> how can I use java POJO with scala Table API.
>