You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by NicoK <gi...@git.apache.org> on 2017/01/17 17:24:01 UTC
[GitHub] flink pull request #3142: [FLINK-5527][query] querying a non-existing key do...
GitHub user NicoK opened a pull request:
https://github.com/apache/flink/pull/3142
[FLINK-5527][query] querying a non-existing key does not return the default value
Querying for a non-existing key for a state that has a default value set currently results in an UnknownKeyOrNamespace exception when the MemoryStateBackend or FsStateBackend is used. It should return the default value instead just like the RocksDBStateBackend.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NicoK/flink flink-5527
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3142.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3142
----
commit 0d7c5f3ebf3a721b679cf830b124ffd4d646c8f6
Author: Nico Kruber <ni...@data-artisans.com>
Date: 2017-01-17T13:26:16Z
[FLINK-5527][query] querying a non-existing key does not return the default value
Querying for a non-existing key for a state that has a default value set currently results in an UnknownKeyOrNamespace exception when the MemoryStateBackend or FsStateBackend is used. It should return the default value instead just like the RocksDBStateBackend.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3142
In a separate change, we actually deprecated the default value in states. It does not behave very well at runtime in many cases and we thought that it would be simpler to let the user code handle default values.
Can queryable state not simply return null as well? I am also not sure if the assumption can be made that external queries want the same default value as the state updating code paths (which frequently use the default value only as a shortcut for initializing the state).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3142: [FLINK-5527][query] querying a non-existing key do...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK closed the pull request at:
https://github.com/apache/flink/pull/3142
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/3142
I saw that deprecation but nonetheless the default value is exposed which is why a consistent behaviour is needed.
Since the state descriptor says "that is the value if nothing is set", I would find it more consistent if that was true for both use cases of "state", i.e. inside an operator and as queryable state. It would make things more complicated for users if it were only true for the operator use, I guess.
If the default value is not used, however, it is set to `null` internally and so `null` is returned anyway only at the penalty of this one check that the actual value is `null`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the issue:
https://github.com/apache/flink/pull/3142
I agree with both of your points, but I'm slightly leaning towards the side of consistent behaviour since this behaviour is not special for queryable state per se. It's just implementing the current contract of the state interfaces, which still allow for a default value. Once we remove that from the state interfaces, queryable state should also not expose it anymore, since queryable state is registered within the program.
In the end I'm fine with both approaches and it's up to you both to decide. :wink:
PS Note that `null` is not returned in case of a query to a non-existent value for a key/namespace, but the `UnknownKeyOrNamespace` exception fails the queries `Future` instance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on the issue:
https://github.com/apache/flink/pull/3142
Ok, let's not introduce the (now deprecated) default values in the queryable state API.
I'll create a new Jira and PR for removing that part from the RocksDB back-end and consistently return `null` for now (which translates to the `UnknownKeyOrNamespace` exception).
Later we can decide to return `null` instead, if desired. Although a query for non-existent state could be considered a failure from the point of the queryable state API.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3142: [FLINK-5527][query] querying a non-existing key does not ...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3142
The default value so far referred to access to the state object from within a program.
I would avoid extending this definition to queryable state given that we are deprecating it.
If we clearly spell the behavior out, I see no problem that queryable state access does not use the default value.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---