You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Suresh Kumar Sethuramaswamy <ro...@gmail.com> on 2015/04/17 17:46:04 UTC

Hive Stats shows numRows=0 when data imported through Sqoop

Hi All,

I'm using CDH 5.3.2 and i use sqoop to load data into hive tables.  last
statement of the sqoop import shows stats log which always shows
[numFiles=<some numbers>,,numRows=0], am i missing any conffiguration here?

Though I have set MySQL as stats db, auto stats doesn't seem to be working,
but when I run analyze compute statistics command , numRows reflects with
Correct count .

My concern is with the automatic stats collection



Regards
Suresh

Re: Hive Stats shows numRows=0 when data imported through Sqoop

Posted by Suresh Kumar Sethuramaswamy <ro...@gmail.com>.
Hi Mich

Yes MySQL is my meta store db, I have set
1. Hive.stats.dbclause
2.hive.stats.jdbcdriver
3.hive.stats.dbconnectionstring

The above configure for stats

On Friday, April 17, 2015, Mich Talebzadeh <mi...@peridale.co.uk> wrote:

> Hi Suresh,
>
>
>
> I guess you are also using MySQL as your Hive metastore?
>
>
>
> What configuration have you set for stats collection?
>
>
>
> HTH
>
>
>
> Mich Talebzadeh
>
>
>
> http://talebzadehmich.wordpress.com
>
>
>
> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase** ASE
> 15", **ISBN 978-0-9563693-0-7*.
>
> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN
> 978-0-9759693-0-4*
>
> *Publications due shortly:*
>
> *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and
> Coherence Cache*
>
> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume
> one out shortly
>
>
>
> NOTE: The information in this email is proprietary and confidential. This
> message is for the designated recipient only, if you are not the intended
> recipient, you should destroy it immediately. Any information in this
> message shall not be understood as given or endorsed by Peridale Ltd, its
> subsidiaries or their employees, unless expressly so stated. It is the
> responsibility of the recipient to ensure that this email is virus free,
> therefore neither Peridale Ltd, its subsidiaries nor their employees accept
> any responsibility.
>
>
>
> *From:* Suresh Kumar Sethuramaswamy [mailto:rockssk@gmail.com
> <javascript:_e(%7B%7D,'cvml','rockssk@gmail.com');>]
> *Sent:* 17 April 2015 16:46
> *To:* user@hive.apache.org
> <javascript:_e(%7B%7D,'cvml','user@hive.apache.org');>
> *Subject:* Hive Stats shows numRows=0 when data imported through Sqoop
>
>
>
>
>
> Hi All,
>
>
>
> I'm using CDH 5.3.2 and i use sqoop to load data into hive tables.  last
> statement of the sqoop import shows stats log which always shows
> [numFiles=<some numbers>,,numRows=0], am i missing any conffiguration here?
>
>
>
> Though I have set MySQL as stats db, auto stats doesn't seem to be
> working, but when I run analyze compute statistics command , numRows
> reflects with Correct count .
>
>
>
> My concern is with the automatic stats collection
>
>
>
>
>
>
>
> Regards
>
> Suresh
>
>
>

RE: Hive Stats shows numRows=0 when data imported through Sqoop

Posted by Mich Talebzadeh <mi...@peridale.co.uk>.
Hi Suresh,

 

I guess you are also using MySQL as your Hive metastore?

 

What configuration have you set for stats collection?

 

HTH

 

Mich Talebzadeh

 

http://talebzadehmich.wordpress.com

 

Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7. 

co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4

Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly

 

NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.

 

From: Suresh Kumar Sethuramaswamy [mailto:rockssk@gmail.com] 
Sent: 17 April 2015 16:46
To: user@hive.apache.org
Subject: Hive Stats shows numRows=0 when data imported through Sqoop

 

 

Hi All,

 

I'm using CDH 5.3.2 and i use sqoop to load data into hive tables.  last statement of the sqoop import shows stats log which always shows [numFiles=<some numbers>,,numRows=0], am i missing any conffiguration here?

 

Though I have set MySQL as stats db, auto stats doesn't seem to be working, but when I run analyze compute statistics command , numRows reflects with Correct count . 

 

My concern is with the automatic stats collection 

 

 

 

Regards

Suresh