Drill 1.0 schema changes

[Jim Bates <jb...@maprtech.com>]

I was doing some basic counts on 1.0 and I noticed that drill is doing a
schema change and I want to know if I can prevent that and only do it if I
want it done.

On the dill sandbox I did a count of a simple field and it worked fine:
select COUNT(orders) from hive.orders;

on anything with a nested JSON it throws an error
select COUNT(personal) from maprdb.customers limit 1;

java.lang.RuntimeException: java.sql.SQLException: UNSUPPORTED_OPERATION
ERROR: Streaming aggregate does not support schema changes

What is the best way to control / mitigate this?

Re: Drill 1.0 schema changes

[Andries Engelbrecht <ae...@maprtech.com>]

Seems like you are trying to do a count on a column family in MapR-DB or HBase.

Try using a specific column rather than a column family.

select count(c.personal.name) from maprdb.customers c;

Please keep in mind that Drill has to manage schema on the fly, so it can’t assume that the schema will stay the same, thus operations like count(*) or in this case count on a column family is very tricky to manage. For most reliable operation, it is best to select a specific column.

With greater flexibility comes responsibility to understand certain operations better in this environment.

—Andries



On May 19, 2015, at 8:35 AM, Jim Bates <jb...@maprtech.com> wrote:

> I was doing some basic counts on 1.0 and I noticed that drill is doing a
> schema change and I want to know if I can prevent that and only do it if I
> want it done.
> 
> On the dill sandbox I did a count of a simple field and it worked fine:
> select COUNT(orders) from hive.orders;
> 
> on anything with a nested JSON it throws an error
> select COUNT(personal) from maprdb.customers limit 1;
> 
> java.lang.RuntimeException: java.sql.SQLException: UNSUPPORTED_OPERATION
> ERROR: Streaming aggregate does not support schema changes
> 
> What is the best way to control / mitigate this?