Question regarding sql with druid and __time

[Don Bowman <do...@agilicus.com>]

I'm using druid 0.13 w/ superset (master).
If i add the druid cluster using its 'native' and then also add it as 'sql'
i can experiment w/ the variance in the interface.

If i use a 'lbig number with trendline' chart, in the 'native' interface,
it works.
But in the sql interface it does not discover there is a timestamp column
(__time) so it doesn't work.

I don't think(?) that i'm supposed to add time as a 'dimension' in druid
(its implicit).

in the sql labs interface it works, e.g. I do:
select * from "l3-agent" order by __time limit 5;

and this returns my columns including a __time one.

But in the chart for 'big number with trendline' there is 'no results
found' for time column.
(t says Datetime column not provided as part table configuration and is
required by this type of chart)

Am I using this incorrectly? Or its a bug?

Re: Question regarding sql with druid and __time

[Maxime Beauchemin <ma...@gmail.com>]

Can you open a Github issue with the stacktrace you get when saving the
datasource configuration? Also we should add some logic to detect the
`__time` column and flag it as temporal automatically.

As a workaround, note that there are 2 ways to edit the datasource, the old
(Sources->Tables-> edit) and the new forms (Explore -> Click on Datasource
-> Modal).

Max

On Fri, Feb 8, 2019 at 5:05 AM Don Bowman <do...@agilicus.com> wrote:

> the __time column shows 'long' and is temporal is false.
> setting that 'true' and saving (it doesn't actually save),
> *but* if i set 'main datetime column' to __time manually, it works.
>
> Otherwise I get "Datetime column not provided as part table configuration
> and is required by this type of chart"
>
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.7/site-packages/superset/viz.py", line 410, in get_df_payload
>     df = self.get_df(query_obj)
>   File "/usr/local/lib/python3.7/site-packages/superset/viz.py", line 213, in get_df
>     self.results = self.datasource.query(query_obj)
>   File "/usr/local/lib/python3.7/site-packages/superset/connectors/sqla/models.py", line 799, in query
>     sql = self.get_query_str(query_obj)
>   File "/usr/local/lib/python3.7/site-packages/superset/connectors/sqla/models.py", line 473, in get_query_str
>     qry = self.get_sqla_query(**query_obj)
>   File "/usr/local/lib/python3.7/site-packages/superset/connectors/sqla/models.py", line 579, in get_sqla_query
>     'Datetime column not provided as part table configuration '
> Exception: Datetime column not provided as part table configuration and is required by this type of chart
>
>
> On Fri, 8 Feb 2019 at 02:31, Maxime Beauchemin <ma...@gmail.com>
> wrote:
>
>> Is the "Is temporal" checkbox checked for the column "__time" in the
>> "Datasource Editor"?
>>
>> [image: Screen Shot 2019-02-07 at 11.30.09 PM.png]
>>
>> On Thu, Feb 7, 2019 at 3:22 PM Don Bowman <do...@agilicus.com> wrote:
>>
>>> This is the sql labs interface showing the schema. it shows the __time
>>> as timestamp. but the charts that are timeseries don't work. I guess
>>> there's a bug, i'll look into it. Perhaps in pydruid?
>>> [image: image.png]
>>>
>>> On Wed, 6 Feb 2019 at 20:41, Don Bowman <do...@agilicus.com> wrote:
>>>
>>>> its from master.
>>>>
>>>>
>>>>
>>>> On Wed, 6 Feb 2019 at 20:26, Maxime Beauchemin <
>>>> maximebeauchemin@gmail.com> wrote:
>>>>
>>>>> Which version of Superset are you using?
>>>>>
>>>>> On Tue, Feb 5, 2019 at 7:38 PM Don Bowman <do...@agilicus.com> wrote:
>>>>>
>>>>> > I'm using druid 0.13 w/ superset (master).
>>>>> > If i add the druid cluster using its 'native' and then also add it
>>>>> as 'sql'
>>>>> > i can experiment w/ the variance in the interface.
>>>>> >
>>>>> > If i use a 'lbig number with trendline' chart, in the 'native'
>>>>> interface,
>>>>> > it works.
>>>>> > But in the sql interface it does not discover there is a timestamp
>>>>> column
>>>>> > (__time) so it doesn't work.
>>>>> >
>>>>> > I don't think(?) that i'm supposed to add time as a 'dimension' in
>>>>> druid
>>>>> > (its implicit).
>>>>> >
>>>>> > in the sql labs interface it works, e.g. I do:
>>>>> > select * from "l3-agent" order by __time limit 5;
>>>>> >
>>>>> > and this returns my columns including a __time one.
>>>>> >
>>>>> > But in the chart for 'big number with trendline' there is 'no results
>>>>> > found' for time column.
>>>>> > (t says Datetime column not provided as part table configuration and
>>>>> is
>>>>> > required by this type of chart)
>>>>> >
>>>>> > Am I using this incorrectly? Or its a bug?
>>>>> >
>>>>>
>>>>

Re: Question regarding sql with druid and __time

[Don Bowman <do...@agilicus.com>]

the __time column shows 'long' and is temporal is false.
setting that 'true' and saving (it doesn't actually save),
*but* if i set 'main datetime column' to __time manually, it works.

Otherwise I get "Datetime column not provided as part table configuration
and is required by this type of chart"

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/superset/viz.py", line
410, in get_df_payload
    df = self.get_df(query_obj)
  File "/usr/local/lib/python3.7/site-packages/superset/viz.py", line
213, in get_df
    self.results = self.datasource.query(query_obj)
  File "/usr/local/lib/python3.7/site-packages/superset/connectors/sqla/models.py",
line 799, in query
    sql = self.get_query_str(query_obj)
  File "/usr/local/lib/python3.7/site-packages/superset/connectors/sqla/models.py",
line 473, in get_query_str
    qry = self.get_sqla_query(**query_obj)
  File "/usr/local/lib/python3.7/site-packages/superset/connectors/sqla/models.py",
line 579, in get_sqla_query
    'Datetime column not provided as part table configuration '
Exception: Datetime column not provided as part table configuration
and is required by this type of chart


On Fri, 8 Feb 2019 at 02:31, Maxime Beauchemin <ma...@gmail.com>
wrote:

> Is the "Is temporal" checkbox checked for the column "__time" in the
> "Datasource Editor"?
>
> [image: Screen Shot 2019-02-07 at 11.30.09 PM.png]
>
> On Thu, Feb 7, 2019 at 3:22 PM Don Bowman <do...@agilicus.com> wrote:
>
>> This is the sql labs interface showing the schema. it shows the __time as
>> timestamp. but the charts that are timeseries don't work. I guess there's a
>> bug, i'll look into it. Perhaps in pydruid?
>> [image: image.png]
>>
>> On Wed, 6 Feb 2019 at 20:41, Don Bowman <do...@agilicus.com> wrote:
>>
>>> its from master.
>>>
>>>
>>>
>>> On Wed, 6 Feb 2019 at 20:26, Maxime Beauchemin <
>>> maximebeauchemin@gmail.com> wrote:
>>>
>>>> Which version of Superset are you using?
>>>>
>>>> On Tue, Feb 5, 2019 at 7:38 PM Don Bowman <do...@agilicus.com> wrote:
>>>>
>>>> > I'm using druid 0.13 w/ superset (master).
>>>> > If i add the druid cluster using its 'native' and then also add it as
>>>> 'sql'
>>>> > i can experiment w/ the variance in the interface.
>>>> >
>>>> > If i use a 'lbig number with trendline' chart, in the 'native'
>>>> interface,
>>>> > it works.
>>>> > But in the sql interface it does not discover there is a timestamp
>>>> column
>>>> > (__time) so it doesn't work.
>>>> >
>>>> > I don't think(?) that i'm supposed to add time as a 'dimension' in
>>>> druid
>>>> > (its implicit).
>>>> >
>>>> > in the sql labs interface it works, e.g. I do:
>>>> > select * from "l3-agent" order by __time limit 5;
>>>> >
>>>> > and this returns my columns including a __time one.
>>>> >
>>>> > But in the chart for 'big number with trendline' there is 'no results
>>>> > found' for time column.
>>>> > (t says Datetime column not provided as part table configuration and
>>>> is
>>>> > required by this type of chart)
>>>> >
>>>> > Am I using this incorrectly? Or its a bug?
>>>> >
>>>>
>>>

Re: Question regarding sql with druid and __time

[Maxime Beauchemin <ma...@gmail.com>]

Is the "Is temporal" checkbox checked for the column "__time" in the
"Datasource Editor"?

[image: Screen Shot 2019-02-07 at 11.30.09 PM.png]

On Thu, Feb 7, 2019 at 3:22 PM Don Bowman <do...@agilicus.com> wrote:

> This is the sql labs interface showing the schema. it shows the __time as
> timestamp. but the charts that are timeseries don't work. I guess there's a
> bug, i'll look into it. Perhaps in pydruid?
> [image: image.png]
>
> On Wed, 6 Feb 2019 at 20:41, Don Bowman <do...@agilicus.com> wrote:
>
>> its from master.
>>
>>
>>
>> On Wed, 6 Feb 2019 at 20:26, Maxime Beauchemin <
>> maximebeauchemin@gmail.com> wrote:
>>
>>> Which version of Superset are you using?
>>>
>>> On Tue, Feb 5, 2019 at 7:38 PM Don Bowman <do...@agilicus.com> wrote:
>>>
>>> > I'm using druid 0.13 w/ superset (master).
>>> > If i add the druid cluster using its 'native' and then also add it as
>>> 'sql'
>>> > i can experiment w/ the variance in the interface.
>>> >
>>> > If i use a 'lbig number with trendline' chart, in the 'native'
>>> interface,
>>> > it works.
>>> > But in the sql interface it does not discover there is a timestamp
>>> column
>>> > (__time) so it doesn't work.
>>> >
>>> > I don't think(?) that i'm supposed to add time as a 'dimension' in
>>> druid
>>> > (its implicit).
>>> >
>>> > in the sql labs interface it works, e.g. I do:
>>> > select * from "l3-agent" order by __time limit 5;
>>> >
>>> > and this returns my columns including a __time one.
>>> >
>>> > But in the chart for 'big number with trendline' there is 'no results
>>> > found' for time column.
>>> > (t says Datetime column not provided as part table configuration and is
>>> > required by this type of chart)
>>> >
>>> > Am I using this incorrectly? Or its a bug?
>>> >
>>>
>>

Re: Question regarding sql with druid and __time

[Don Bowman <do...@agilicus.com>]

This is the sql labs interface showing the schema. it shows the __time as
timestamp. but the charts that are timeseries don't work. I guess there's a
bug, i'll look into it. Perhaps in pydruid?
[image: image.png]

On Wed, 6 Feb 2019 at 20:41, Don Bowman <do...@agilicus.com> wrote:

> its from master.
>
>
>
> On Wed, 6 Feb 2019 at 20:26, Maxime Beauchemin <ma...@gmail.com>
> wrote:
>
>> Which version of Superset are you using?
>>
>> On Tue, Feb 5, 2019 at 7:38 PM Don Bowman <do...@agilicus.com> wrote:
>>
>> > I'm using druid 0.13 w/ superset (master).
>> > If i add the druid cluster using its 'native' and then also add it as
>> 'sql'
>> > i can experiment w/ the variance in the interface.
>> >
>> > If i use a 'lbig number with trendline' chart, in the 'native'
>> interface,
>> > it works.
>> > But in the sql interface it does not discover there is a timestamp
>> column
>> > (__time) so it doesn't work.
>> >
>> > I don't think(?) that i'm supposed to add time as a 'dimension' in druid
>> > (its implicit).
>> >
>> > in the sql labs interface it works, e.g. I do:
>> > select * from "l3-agent" order by __time limit 5;
>> >
>> > and this returns my columns including a __time one.
>> >
>> > But in the chart for 'big number with trendline' there is 'no results
>> > found' for time column.
>> > (t says Datetime column not provided as part table configuration and is
>> > required by this type of chart)
>> >
>> > Am I using this incorrectly? Or its a bug?
>> >
>>
>

Re: Question regarding sql with druid and __time

[Don Bowman <do...@agilicus.com>]

its from master.



On Wed, 6 Feb 2019 at 20:26, Maxime Beauchemin <ma...@gmail.com>
wrote:

> Which version of Superset are you using?
>
> On Tue, Feb 5, 2019 at 7:38 PM Don Bowman <do...@agilicus.com> wrote:
>
> > I'm using druid 0.13 w/ superset (master).
> > If i add the druid cluster using its 'native' and then also add it as
> 'sql'
> > i can experiment w/ the variance in the interface.
> >
> > If i use a 'lbig number with trendline' chart, in the 'native' interface,
> > it works.
> > But in the sql interface it does not discover there is a timestamp column
> > (__time) so it doesn't work.
> >
> > I don't think(?) that i'm supposed to add time as a 'dimension' in druid
> > (its implicit).
> >
> > in the sql labs interface it works, e.g. I do:
> > select * from "l3-agent" order by __time limit 5;
> >
> > and this returns my columns including a __time one.
> >
> > But in the chart for 'big number with trendline' there is 'no results
> > found' for time column.
> > (t says Datetime column not provided as part table configuration and is
> > required by this type of chart)
> >
> > Am I using this incorrectly? Or its a bug?
> >
>

Re: Question regarding sql with druid and __time

[Maxime Beauchemin <ma...@gmail.com>]

Which version of Superset are you using?

On Tue, Feb 5, 2019 at 7:38 PM Don Bowman <do...@agilicus.com> wrote:

> I'm using druid 0.13 w/ superset (master).
> If i add the druid cluster using its 'native' and then also add it as 'sql'
> i can experiment w/ the variance in the interface.
>
> If i use a 'lbig number with trendline' chart, in the 'native' interface,
> it works.
> But in the sql interface it does not discover there is a timestamp column
> (__time) so it doesn't work.
>
> I don't think(?) that i'm supposed to add time as a 'dimension' in druid
> (its implicit).
>
> in the sql labs interface it works, e.g. I do:
> select * from "l3-agent" order by __time limit 5;
>
> and this returns my columns including a __time one.
>
> But in the chart for 'big number with trendline' there is 'no results
> found' for time column.
> (t says Datetime column not provided as part table configuration and is
> required by this type of chart)
>
> Am I using this incorrectly? Or its a bug?
>