Generating data samples from result sets of queries

[Jaideep Dhok <ja...@inmobi.com>]

Hi all,

One of our requirements is to generate sample values of measures and
dimensions so that users can get a feel of the result _before_ they submit
a query.

Could we get this data from result sets? From the cube rewriter module we
can figure which columns will be present in the result set, excluding
projected columns.

Once query is finished, **result formatter* *in Lens goes through the
entire result set to format data properly. We can piggyback on this process
and generate samples of the queried columns.

An advantage of this approach is that we don't need separate scans of data
just to generate samples. A limitation is that this data would become
available only after few queries have run.

Thoughts?

Thanks,
-
Jaideep Dhok

-- 
_____________________________________________________________
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

RE: Generating data samples from result sets of queries

[Srikanth Sundarrajan <sr...@hotmail.com>]

My 2 cents FWIW. 

Not sure getting sample values for a dimension or measure is a problem lens should be concerned about. It has optical value in a query builder. If and when we do build one within lens, then it should be a concern for that component and shouldn't belong within the lens server.

Regards
Srikanth Sundarrajan

> Date: Thu, 5 Mar 2015 21:10:10 +0530
> Subject: Generating data samples from result sets of queries
> From: jaideep.dhok@inmobi.com
> To: dev@lens.incubator.apache.org
> 
> Hi all,
> 
> One of our requirements is to generate sample values of measures and
> dimensions so that users can get a feel of the result _before_ they submit
> a query.
> 
> Could we get this data from result sets? From the cube rewriter module we
> can figure which columns will be present in the result set, excluding
> projected columns.
> 
> Once query is finished, **result formatter* *in Lens goes through the
> entire result set to format data properly. We can piggyback on this process
> and generate samples of the queried columns.
> 
> An advantage of this approach is that we don't need separate scans of data
> just to generate samples. A limitation is that this data would become
> available only after few queries have run.
> 
> Thoughts?
> 
> Thanks,
> -
> Jaideep Dhok
> 
> -- 
> _____________________________________________________________
> The information contained in this communication is intended solely for the 
> use of the individual or entity to whom it is addressed and others 
> authorized to receive it. It may contain confidential or legally privileged 
> information. If you are not the intended recipient you are hereby notified 
> that any disclosure, copying, distribution or taking any action in reliance 
> on the contents of this information is strictly prohibited and may be 
> unlawful. If you have received this communication in error, please notify 
> us immediately by responding to this email and then delete it from your 
> system. The firm is neither liable for the proper and complete transmission 
> of the information contained in this communication nor for any delay in its 
> receipt.