[jira] [Updated] (GRIFFIN-311) keep null value columns optionally

["XIAOYU YU (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GRIFFIN-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

XIAOYU YU updated GRIFFIN-311:
------------------------------
    Description: 
Hi all, 
 Our team are facing a confused issue that griffin's output could not keep the null value columns. 
 Our spark version is 2.2. For the following table xxx.xxxx and rule: 
|name|age|date|
|NULL|12|2018-11-11|
|sizhe|NULL|2018-11-11|

rule: 
{code:java}
"rule" : "select name, age from xxx.xxxx", 
"out" : {"type" : "metric", "flatten" : "array"} 
"dsl.type" : "spark-sql", "out.dataframe.name" : "payloads"{code}
Griffin result would be as follows, which ignore the null value and null line.
{code:java}
 [{"age": 12}, {"name": "sizhe"}]  {code}
It would confuse some users, when they found some columns and lines missed in result and the expected result for them should be:
{code:java}
 [{"name": null, "age": 12}, {"name": "sizhe", "age": null}]  {code}
 
 And for sql: 
{code:java}
"rule" : "select name, age from xxx.xxxx where date = '2019-01-01'",  {code}
Griffin result would be []. But the expected result for them should be
{code:java}
[{"name": null, "age": null}]{code}
, which contains a null line.
  
 Could griffin add a new feature to keep the null value columns optionally to make these users happy?

  was:
Hi all, 
Our team are facing a confused issue that griffin's output could not keep the null value columns. 
Our spark version is 2.2. For the following table xxx.xxxx and rule: 
|name|age|date|
|NULL|12|2018-11-11|
|sizhe|NULL|2018-11-11|
rule:     __     
{{}}
{code:java}
"rule" : "select name, age from xxx.xxxx", 
"out" : {"type" : "metric", "flatten" : "array"} 
"dsl.type" : "spark-sql", "out.dataframe.name" : "payloads"{code}
{{}}
 
Griffin result would be as follows, which ignore the null value and null line.

{code:java}
[{"age":12}, {"name": "sizhe"}]{code}
It would confuse some users, when they found some columns and lines missed in result and the expected result for them should be:
{{}}
{code:java}
 [{"name": null, "age": 12}, {"name": "sizhe", "age": null}]  {code}
 
And for sql: 
{{}}
{code:java}
"rule" : "select name, age from xxx.xxxx where date = '2019-01-01'",  {code}
Griffin result would be []. But the expected result for them should be
{code:java}
[{"name": null, "age": null}]{code}
, which contains a null line.
 
Could griffin add a new feature to keep the null value columns optionally to make these users happy?


> keep null value columns optionally
> ----------------------------------
>
>                 Key: GRIFFIN-311
>                 URL: https://issues.apache.org/jira/browse/GRIFFIN-311
>             Project: Griffin
>          Issue Type: New Feature
>            Reporter: XIAOYU YU
>            Priority: Major
>
> Hi all, 
>  Our team are facing a confused issue that griffin's output could not keep the null value columns. 
>  Our spark version is 2.2. For the following table xxx.xxxx and rule: 
> |name|age|date|
> |NULL|12|2018-11-11|
> |sizhe|NULL|2018-11-11|
> rule: 
> {code:java}
> "rule" : "select name, age from xxx.xxxx", 
> "out" : {"type" : "metric", "flatten" : "array"} 
> "dsl.type" : "spark-sql", "out.dataframe.name" : "payloads"{code}
> Griffin result would be as follows, which ignore the null value and null line.
> {code:java}
>  [{"age": 12}, {"name": "sizhe"}]  {code}
> It would confuse some users, when they found some columns and lines missed in result and the expected result for them should be:
> {code:java}
>  [{"name": null, "age": 12}, {"name": "sizhe", "age": null}]  {code}
>  
>  And for sql: 
> {code:java}
> "rule" : "select name, age from xxx.xxxx where date = '2019-01-01'",  {code}
> Griffin result would be []. But the expected result for them should be
> {code:java}
> [{"name": null, "age": null}]{code}
> , which contains a null line.
>   
>  Could griffin add a new feature to keep the null value columns optionally to make these users happy?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)