You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Mirza Aliev (Jira)" <ji...@apache.org> on 2023/04/11 18:38:00 UTC
[jira] [Updated] (IGNITE-18954) Design a language for parsing of a filter expression
[ https://issues.apache.org/jira/browse/IGNITE-18954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mirza Aliev updated IGNITE-18954:
---------------------------------
Description:
{*}Motivation{*}:
We need to parse and be able to use filters, that user set for the distribution zone
*Definition of done:*
* Filters are parsed and it is possible to filter arbitrary set of attributes
* Validator is implemented, so we can validate a string representation of a filter expression
*Implementation details:*
Language must allow to perform simple operations, like
* “ “ – default empty filter, means that all nodes match
* () expressions with parentheses
* key1 = “A” && key2 = “B”, () && ()
* key1 = “A” || key2 = “B”, () || ()
* key1 != "A"
Attributes representation must look like key -> value pairs, like
{noformat}
(region = EU || region = US) && (storage != HDD || storage = SSD)
{noformat}
After some investigation, I propose to not implement our own language for the filters and attributes, but reuse JSON functionality.
For nodes, user just pass JSON representation of attributes through the {{ignite-config.json}}:
{code:java}
{
"network": {
"port": 3344
},
"rest": {
"port": 10300
},
"nodeAttributes": {
"region": "EU",
"storage": "SSD",
"dataRegion" : 10
}
}
{code}
For parsing, we can use https://github.com/json-path/JsonPath, which is already used in ignite {{sql-engine}} module, so we don't need to bring a new dependancy.
{{JsonPath}} has a wide functionality to work with JSON, we are interested in filtering queries. See all list of available functionality here https://github.com/json-path/JsonPath#functions. Example of usages you can find here https://rows.com/docs/filtering-with-jsonpath
In general, it will allow us to make all operations, that we have listed before.
Lets provide some examples with our node attributes:
{{['nodeName'][?(@.['region'] == 'EU')]}} -- will give all nodeName, that have region equals to 'EU'
{{['nodeName'].[?(@.['dataRegion'] > 10 && @.[storage] != 'HDD')]}} -- will give all nodeNames, that has dataRegion > 10 and storage not equal to HDD
UPD:
was:
{*}Motivation{*}:
We need to parse and be able to use filters, that user set for the distribution zone
*Definition of done:*
* Filters are parsed and it is possible to filter arbitrary set of attributes
* Validator is implemented, so we can validate a string representation of a filter expression
*Implementation details:*
Language must allow to perform simple operations, like
* “ “ – default empty filter, means that all nodes match
* () expressions with parentheses
* key1 = “A” && key2 = “B”, () && ()
* key1 = “A” || key2 = “B”, () || ()
* key1 != "A"
Attributes representation must look like key -> value pairs, like
{noformat}
(region = EU || region = US) && (storage != HDD || storage = SSD)
{noformat}
After some investigation, I propose to not implement our own language for the filters and attributes, but reuse JSON functionality.
For nodes, user just pass JSON representation of attributes through the {{ignite-config.json}}:
{code:java}
{
"network": {
"port": 3344
},
"rest": {
"port": 10300
},
"nodeAttributes": {
"nodeName": "node1"
"region": "EU",
"storage": "SSD",
"dataRegion" : 10
}
}
{code}
For parsing, we can use https://github.com/json-path/JsonPath, which is already used in ignite {{sql-engine}} module, so we don't need to bring a new dependancy.
{{JsonPath}} has a wide functionality to work with JSON, we are interested in filtering queries. See all list of available functionality here https://github.com/json-path/JsonPath#functions. Example of usages you can find here https://rows.com/docs/filtering-with-jsonpath
In general, it will allow us to make all operations, that we have listed before.
Lets provide some examples with our node attributes:
{{['nodeName'][?(@.['region'] == 'EU')]}} -- will give all nodeName, that have region equals to 'EU'
{{['nodeName'].[?(@.['dataRegion'] > 10 && @.[storage] != 'HDD')]}} -- will give all nodeNames, that has dataRegion > 10 and storage not equal to HDD
> Design a language for parsing of a filter expression
> ----------------------------------------------------
>
> Key: IGNITE-18954
> URL: https://issues.apache.org/jira/browse/IGNITE-18954
> Project: Ignite
> Issue Type: Improvement
> Reporter: Mirza Aliev
> Assignee: Mirza Aliev
> Priority: Major
> Labels: ignite-3
>
> {*}Motivation{*}:
> We need to parse and be able to use filters, that user set for the distribution zone
> *Definition of done:*
>
> * Filters are parsed and it is possible to filter arbitrary set of attributes
> * Validator is implemented, so we can validate a string representation of a filter expression
> *Implementation details:*
> Language must allow to perform simple operations, like
> * “ “ – default empty filter, means that all nodes match
> * () expressions with parentheses
> * key1 = “A” && key2 = “B”, () && ()
> * key1 = “A” || key2 = “B”, () || ()
> * key1 != "A"
> Attributes representation must look like key -> value pairs, like
> {noformat}
> (region = EU || region = US) && (storage != HDD || storage = SSD)
> {noformat}
> After some investigation, I propose to not implement our own language for the filters and attributes, but reuse JSON functionality.
> For nodes, user just pass JSON representation of attributes through the {{ignite-config.json}}:
> {code:java}
> {
> "network": {
> "port": 3344
> },
> "rest": {
> "port": 10300
> },
> "nodeAttributes": {
> "region": "EU",
> "storage": "SSD",
> "dataRegion" : 10
> }
> }
> {code}
> For parsing, we can use https://github.com/json-path/JsonPath, which is already used in ignite {{sql-engine}} module, so we don't need to bring a new dependancy.
> {{JsonPath}} has a wide functionality to work with JSON, we are interested in filtering queries. See all list of available functionality here https://github.com/json-path/JsonPath#functions. Example of usages you can find here https://rows.com/docs/filtering-with-jsonpath
> In general, it will allow us to make all operations, that we have listed before.
> Lets provide some examples with our node attributes:
> {{['nodeName'][?(@.['region'] == 'EU')]}} -- will give all nodeName, that have region equals to 'EU'
> {{['nodeName'].[?(@.['dataRegion'] > 10 && @.[storage] != 'HDD')]}} -- will give all nodeNames, that has dataRegion > 10 and storage not equal to HDD
> UPD:
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)