You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Hongyuhong <ho...@huawei.com> on 2017/01/13 08:54:12 UTC
the attribute order in sql 'select * from...'
Hi,
I'm now using streaming sql, And I have the sql like
select * FROM OrderA where user > 2
the OrderA has 3 attr (user, product, amount)
and I expect the result is as the order like input, but it has been sorted by attr name,
and I found the order has already been sorted when call addSource,
What is the purpose of doing so?cause it's a little not meet our requirements.
Thanks very much.
public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
super(typeClass);
checkArgument(Modifier.isPublic(typeClass.getModifiers()),
"POJO %s is not public", typeClass);
this.fields = fields.toArray(new PojoField[fields.size()]);
Arrays.sort(this.fields, new Comparator<PojoField>() {
@Override
public int compare(PojoField o1, PojoField o2) {
return o1.getField().getName().compareTo(o2.getField().getName());
}
});
Best,
Yuhong
Re: the attribute order in sql 'select * from...'
Posted by Timo Walther <tw...@apache.org>.
Hi Yuhong,
as a solution you can specify the order of your Pojo fields when
converting from DataStream to Table.
Table table = tableEnv
.fromDataSet(env.fromCollection(data), "department AS a, " +
"age AS b, " +
"salary AS c, " +
"name AS d")
.select("a, b, c, d");
Timo
Am 13/01/17 um 10:22 schrieb Fabian Hueske:
> Hi Yuhong,
>
> I assume that OrderA is a table of POJO objects and you are expecting
> the order of the attribute to be as the order in which the fields of
> the POJO are defined in the source code.
>
> Flink accepts fields which are either public members or accessible via
> a getter and setter.
> This makes it difficult to automatically define an order, esp. if some
> fields use getter and setter or public fields. Would the order depend
> on the field (which might not exist in case of getter/setter) or
> setter or getter methods (which might also not exist).
> I'm also not sure if it is possible to extract the line number of a
> method or field via reflection.
>
> Best, Fabian
>
>
>
> 2017-01-13 9:54 GMT+01:00 Hongyuhong <hongyuhong@huawei.com
> <ma...@huawei.com>>:
>
> Hi,
>
> I\u2019m now using streaming sql, And I have the sql like
>
> select * FROM OrderA where user > 2
>
> the OrderA has 3 attr (user, product, amount)
>
> and I expect the result is as the order like input, but it has
> been sorted by attr name,
>
> and I found the order has already been sorted when call addSource,
>
> What is the purpose of doing so?cause it\u2019s a little not meet our
> requirements.
>
> Thanks very much.
>
> public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
>
> super(typeClass);
>
> checkArgument(Modifier.isPublic(typeClass.getModifiers()),
>
> "POJO %s is not public", typeClass);
>
> this.fields = fields.toArray(new PojoField[fields.size()]);
>
> Arrays.sort(this.fields, new Comparator<PojoField>() {
>
> @Override
>
> public int compare(PojoField o1, PojoField o2) {
>
> return o1.getField().getName().compareTo(o2.getField().getName());
>
> }
>
> });
>
> Best,
>
> Yuhong
>
>
Re: the attribute order in sql 'select * from...'
Posted by Timo Walther <tw...@apache.org>.
Especially, as it might also change the serialized binary format.
Am 13/01/17 um 11:24 schrieb Fabian Hueske:
> I think the sorting is done for consistency reasons, i.e., that all
> PojoTypeInfos for the same class behave the same.
> Since this code is used in many parts of Flink and many jobs (DataSet,
> DataStream, etc.) I would be very careful to change the default
> behavior here.
>
> Maybe we can add a constructor that does not sort the fields.
>
> 2017-01-13 10:46 GMT+01:00 Hongyuhong <hongyuhong@huawei.com
> <ma...@huawei.com>>:
>
> Hi Fabian,
>
> Yes, OrderA is a table of POJO.
>
> But what I consume is that in construct func PojoTypeInfo
>
> The input param (fields)\u2019s order is right, it \u2018s change after the
> sort operation, and I\u2019m wonder if the sort operation can be removed?
>
> public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
>
> super(typeClass);
>
> checkArgument(Modifier.isPublic(typeClass.getModifiers()),"POJO
> %s is not public", typeClass);
>
> this.fields = fields.toArray(new PojoField[fields.size()]);
>
> *Arrays.sort(this.fields, new Comparator<PojoField>() {*
>
> * @Override*
>
> * public int compare(PojoField o1, PojoField o2) {*
>
> * return o1.getField().getName().compareTo(o2.getField().getName());*
>
> * }*
>
> });
>
> *\u53d1\u4ef6\u4eba:*Fabian Hueske [mailto:fhueske@gmail.com
> <ma...@gmail.com>]
> *\u53d1\u9001\u65f6\u95f4:*2017\u5e741\u670813\u65e517:23
> *\u6536\u4ef6\u4eba:*user@flink.apache.org <ma...@flink.apache.org>
> *\u4e3b\u9898:*Re: the attribute order in sql 'select * from...'
>
> Hi Yuhong,
>
> I assume that OrderA is a table of POJO objects and you are
> expecting the order of the attribute to be as the order in which
> the fields of the POJO are defined in the source code.
>
> Flink accepts fields which are either public members or accessible
> via a getter and setter.
>
> This makes it difficult to automatically define an order, esp. if
> some fields use getter and setter or public fields. Would the
> order depend on the field (which might not exist in case of
> getter/setter) or setter or getter methods (which might also not
> exist).
>
> I'm also not sure if it is possible to extract the line number of
> a method or field via reflection.
>
> Best, Fabian
>
> 2017-01-13 9:54 GMT+01:00 Hongyuhong <hongyuhong@huawei.com
> <ma...@huawei.com>>:
>
> Hi,
>
> I\u2019m now using streaming sql, And I have the sql like
>
> select * FROM OrderA where user > 2
>
> the OrderA has 3 attr (user, product, amount)
>
> and I expect the result is as the order like input, but it has
> been sorted by attr name,
>
> and I found the order has already been sorted when call addSource,
>
> What is the purpose of doing so?cause it\u2019s a little not meet our
> requirements.
>
> Thanks very much.
>
> public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
>
> super(typeClass);
>
> checkArgument(Modifier.isPublic(typeClass.getModifiers()),
>
> "POJO %s is not public", typeClass);
>
> this.fields = fields.toArray(new PojoField[fields.size()]);
>
> Arrays.sort(this.fields, new Comparator<PojoField>() {
>
> @Override
>
> public int compare(PojoField o1, PojoField o2) {
>
> return o1.getField().getName().compareTo(o2.getField().getName());
>
> }
>
> });
>
> Best,
>
> Yuhong
>
>
Re: the attribute order in sql 'select * from...'
Posted by Fabian Hueske <fh...@gmail.com>.
I think the sorting is done for consistency reasons, i.e., that all
PojoTypeInfos for the same class behave the same.
Since this code is used in many parts of Flink and many jobs (DataSet,
DataStream, etc.) I would be very careful to change the default behavior
here.
Maybe we can add a constructor that does not sort the fields.
2017-01-13 10:46 GMT+01:00 Hongyuhong <ho...@huawei.com>:
> Hi Fabian,
>
>
>
> Yes, OrderA is a table of POJO.
>
> But what I consume is that in construct func PojoTypeInfo
>
> The input param (fields)’s order is right, it ‘s change after the sort
> operation, and I’m wonder if the sort operation can be removed?
>
>
>
> public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
>
> super(typeClass);
>
> checkArgument(Modifier.isPublic(typeClass.getModifiers()),"POJO %s is
> not public", typeClass);
>
> this.fields = fields.toArray(new PojoField[fields.size()]);
>
> *Arrays.sort(this.fields, new Comparator<PojoField>() {*
>
> * @Override*
>
> * public int compare(PojoField o1, PojoField o2) {*
>
> * return o1.getField().getName().compareTo(o2.getField().getName());*
>
> * }*
>
> });
>
>
>
>
>
> *发件人:* Fabian Hueske [mailto:fhueske@gmail.com]
> *发送时间:* 2017年1月13日 17:23
> *收件人:* user@flink.apache.org
> *主题:* Re: the attribute order in sql 'select * from...'
>
>
>
> Hi Yuhong,
>
> I assume that OrderA is a table of POJO objects and you are expecting the
> order of the attribute to be as the order in which the fields of the POJO
> are defined in the source code.
>
> Flink accepts fields which are either public members or accessible via a
> getter and setter.
>
> This makes it difficult to automatically define an order, esp. if some
> fields use getter and setter or public fields. Would the order depend on
> the field (which might not exist in case of getter/setter) or setter or
> getter methods (which might also not exist).
>
> I'm also not sure if it is possible to extract the line number of a method
> or field via reflection.
>
> Best, Fabian
>
>
>
>
>
> 2017-01-13 9:54 GMT+01:00 Hongyuhong <ho...@huawei.com>:
>
> Hi,
>
> I’m now using streaming sql, And I have the sql like
>
> select * FROM OrderA where user > 2
>
> the OrderA has 3 attr (user, product, amount)
>
> and I expect the result is as the order like input, but it has been sorted
> by attr name,
>
> and I found the order has already been sorted when call addSource,
>
> What is the purpose of doing so?cause it’s a little not meet our
> requirements.
>
>
>
> Thanks very much.
>
>
>
>
>
> public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
>
> super(typeClass);
>
>
>
> checkArgument(Modifier.isPublic(typeClass.
> getModifiers()),
>
> "POJO %s is not public", typeClass);
>
>
>
> this.fields = fields.toArray(new
> PojoField[fields.size()]);
>
>
>
> Arrays.sort(this.fields, new Comparator<PojoField>() {
>
> @Override
>
> public int compare(PojoField o1, PojoField o2)
> {
>
> return o1.getField().getName().
> compareTo(o2.getField().getName());
>
> }
>
> });
>
>
>
>
>
> Best,
>
> Yuhong
>
>
>
Re: the attribute order in sql 'select * from...'
Posted by Hongyuhong <ho...@huawei.com>.
Hi Fabian,
Yes, OrderA is a table of POJO.
But what I consume is that in construct func PojoTypeInfo
The input param (fields)’s order is right, it ‘s change after the sort operation, and I’m wonder if the sort operation can be removed?
public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
super(typeClass);
checkArgument(Modifier.isPublic(typeClass.getModifiers()),"POJO %s is not public", typeClass);
this.fields = fields.toArray(new PojoField[fields.size()]);
Arrays.sort(this.fields, new Comparator<PojoField>() {
@Override
public int compare(PojoField o1, PojoField o2) {
return o1.getField().getName().compareTo(o2.getField().getName());
}
});
发件人: Fabian Hueske [mailto:fhueske@gmail.com]
发送时间: 2017年1月13日 17:23
收件人: user@flink.apache.org
主题: Re: the attribute order in sql 'select * from...'
Hi Yuhong,
I assume that OrderA is a table of POJO objects and you are expecting the order of the attribute to be as the order in which the fields of the POJO are defined in the source code.
Flink accepts fields which are either public members or accessible via a getter and setter.
This makes it difficult to automatically define an order, esp. if some fields use getter and setter or public fields. Would the order depend on the field (which might not exist in case of getter/setter) or setter or getter methods (which might also not exist).
I'm also not sure if it is possible to extract the line number of a method or field via reflection.
Best, Fabian
2017-01-13 9:54 GMT+01:00 Hongyuhong <ho...@huawei.com>>:
Hi,
I’m now using streaming sql, And I have the sql like
select * FROM OrderA where user > 2
the OrderA has 3 attr (user, product, amount)
and I expect the result is as the order like input, but it has been sorted by attr name,
and I found the order has already been sorted when call addSource,
What is the purpose of doing so?cause it’s a little not meet our requirements.
Thanks very much.
public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
super(typeClass);
checkArgument(Modifier.isPublic(typeClass.getModifiers()),
"POJO %s is not public", typeClass);
this.fields = fields.toArray(new PojoField[fields.size()]);
Arrays.sort(this.fields, new Comparator<PojoField>() {
@Override
public int compare(PojoField o1, PojoField o2) {
return o1.getField().getName().compareTo(o2.getField().getName());
}
});
Best,
Yuhong
Re: the attribute order in sql 'select * from...'
Posted by Fabian Hueske <fh...@gmail.com>.
Hi Yuhong,
I assume that OrderA is a table of POJO objects and you are expecting the
order of the attribute to be as the order in which the fields of the POJO
are defined in the source code.
Flink accepts fields which are either public members or accessible via a
getter and setter.
This makes it difficult to automatically define an order, esp. if some
fields use getter and setter or public fields. Would the order depend on
the field (which might not exist in case of getter/setter) or setter or
getter methods (which might also not exist).
I'm also not sure if it is possible to extract the line number of a method
or field via reflection.
Best, Fabian
2017-01-13 9:54 GMT+01:00 Hongyuhong <ho...@huawei.com>:
> Hi,
>
> I’m now using streaming sql, And I have the sql like
>
> select * FROM OrderA where user > 2
>
> the OrderA has 3 attr (user, product, amount)
>
> and I expect the result is as the order like input, but it has been sorted
> by attr name,
>
> and I found the order has already been sorted when call addSource,
>
> What is the purpose of doing so?cause it’s a little not meet our
> requirements.
>
>
>
> Thanks very much.
>
>
>
>
>
> public PojoTypeInfo(Class<T> typeClass, List<PojoField> fields) {
>
> super(typeClass);
>
>
>
> checkArgument(Modifier.isPublic(typeClass.
> getModifiers()),
>
> "POJO %s is not public", typeClass);
>
>
>
> this.fields = fields.toArray(new
> PojoField[fields.size()]);
>
>
>
> Arrays.sort(this.fields, new Comparator<PojoField>() {
>
> @Override
>
> public int compare(PojoField o1, PojoField o2)
> {
>
> return o1.getField().getName().
> compareTo(o2.getField().getName());
>
> }
>
> });
>
>
>
>
>
> Best,
>
> Yuhong
>