[GitHub] [incubator-superset] john-bodley commented on issue #6360: [SIP-15] Transparent and Consistent Time Intervals

[GitBox <gi...@apache.org>]

john-bodley commented on issue #6360: [SIP-15] Transparent and Consistent Time Intervals
URL: https://github.com/apache/incubator-superset/issues/6360#issuecomment-494474516
 
 
   Here's some findings from a survey which was given to a group of data scientists at Airbnb. The results are based on 46 responses.
   
   **Q1. Imagine you had data for the entire month of January (encoded as dates, e.g. 2019-01-01). Which of the following intervals make most sense?**
   
   - `[start, end]`: e.g. `[2019-01-01, 2019-01-31]` **89.1%**
   - `[start, end)`: e.g. `[2019-01-01, 2019-02-01)` **10.9%**
   - `(start, end]`: e.g. `(2018-12-31, 2019-01-31]` **0%**
   - `(start, end)`: e.g. `(2019-12-31, 2019-02-01)` **0%**
   
   **Q2. Imagine you had data for the entire month on January (encoded as times, e.g 2019-01-01 01:23:45.678). Which of the following intervals make the most sense?**
   
   - `[start, end]`: .e.g. `[2019-01-01 00:00:00.000, 2019-01-31 23:59:59.999]` **60.9%**
   - `[start, end)`: e.g. `[2019-01-01 00:00:00.000, 2019-02-01 00:00:00.000)` **37%**
   - `(start, end]`: e.g. `(2018-12-31 23:59:59.999, 2019-01-31 23:59:59.999]` **0%**
   - `(start, end)`: e.g. `(2018-12-31 23:59:59.999, 2019-02-01 00:00:00.000)` **2.1%**
   
   **Q3. Are you aware of any discrepancies with how Superset filters dates or times for SQL databases (Hive, Presto, etc.)?**
   
   - Yes **21.7%**
   - No **78.3%**
   
   **Q4. Could you please outline which discrepancies you are aware of?**
   
   - Sometimes when I put my start date in, Superset returns the next date (like it starts with an open interval).
   - Sometimes, I set the date range for a chart to be e.g. Jan 1 forward, but the chart ends up not including Jan 1, so I have to change the start date of the chart to dec 31...
   - I am generally aware of issues with date filtering and take a trial and error approach until my visualization contains the intended time range.
   - Druid and presto do not handle end dates the same way. In some cases the end date is included, but not in others.
   - The date filters on Superset are pegged to the end of the day (because timestamps are added in the underlying query) while selecting a date (without timestamps) in Hive or Presto will be inclusive of the full day's activities.
   - For relative times ("Last Quarter"), the utility in my mind would be looking at (literally) last quarter (Q1), rather than the last ~90 days which is I think what superset does now.
   - Granularity period are truncated when selecting exact in ds selector. For instance selecting 2018-01-01 to 2019-01-01 and plotting a line graph per month, would return different result for Jan 2019 than doing the same graph with period 2018-01-01 to 2019-02-01.
   - End date not always included in the interval selected.
   
   **Q5. Are you aware of any discrepancies with how Superset filters dates or times for Druid datasources?**
   
   - Yes **10.9%**
   - No **89.1%**
   
   **Q6. Could you please outline which discrepancies you are aware of?**
   
   - The default time filters are not super intuitive.
   - There is a difference in using "Last Saturday" as an end date criteria. When you run a normal query, the end date is inclusive (it will include last Saturday). with druid, the end date is a "less than" so you actually have to set your date to "Last Sunday".
   - Different handling on date/timeseries range between charts.
   
   **Q7. Currently Superset handles all temporal data as time. Should Superset support both date and time types?**
   
   - Yes **58.7%**
   - No **0%**
   - Unsure **41.3%**
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org