You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lens.apache.org by Tao Yan <ty...@linkedin.com> on 2016/08/03 04:12:17 UTC
Exception While Purging Query
Hi Lens Developers,
I run the example queries in the '20 minutes demo' page for several times,
and a query failed many time before it succeeded:
lens-shell>query execute cube select product_id, store_sales from sales
where time_range_in(order_time, '2015-04-11-00', '2015-04-13-00')
Then, I found the Lens server keeps printing the following error log:
03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34] [QueryPurger]
WARN org.apache.lens.server.query.LensServerDAO - Re insert happening in
purge: [Ljava.lang.StackTraceElement;@354862c
03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34] [QueryPurger]
WARN *org.apache.lens.server.query.QueryExecutionServiceImpl - Exception
while purging query 27be8f4d-9d9a-43da-9c10-710ef0734ab3*
java.sql.SQLException: Found different value pre-existing in DB while
trying to insert finished query.* Old =* FinishedLensQuery(
*handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*, userQuery=cube select
product_id, store_sales from sales where time_range_in(order_time,
'2015-04-11-00', '2015-04-13-00'), submitter=anonymous,
submissionTime=1470184239496, startTime=1470184243802,
endTime=1470184289999, result=null, status=FAILED, metadata=null,
rows=null, fileSize=null, errorMessage=Stage-0(MOVE):Stage-0: has failed! ,
driverStartTime=1470184239666, driverEndTime=1470184289989, queryName=,
driverName=hive/hive1, selectedDriver=null)
*New =* FinishedLensQuery(*handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*,
userQuery=cube select product_id, store_sales from sales where
time_range_in(order_time, '2015-04-11-00', '2015-04-13-00'),
submitter=anonymous, submissionTime=1470184239496, startTime=1470184243802,
endTime=1470184608172, result=null, status=FAILED, metadata=null,
rows=null, fileSize=null, errorMessage=Error getting query status,
driverStartTime=1470184239666, driverEndTime=0, queryName=,
driverName=hive/hive1, selectedDriver=hive/hive1)
at
org.apache.lens.server.query.LensServerDAO.insertFinishedQuery(LensServerDAO.java:133)
~[classes/:na]
at
org.apache.lens.server.query.QueryExecutionServiceImpl$QueryPurger.run(QueryExecutionServiceImpl.java:1007)
~[classes/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_31]
It looks that the new query has the same handle as the old query. So, Lens
servers complains about it. It seems other queries are not impacted, but
this Warning messages are really annoying and impacting regular logs.
Thanks,
--
*Tao Yan*
Software Engineer
Data Analytics Infrastructure Tools and Services
206.250.5345
tyan@linkedin.com
https://www.linkedin.com/in/taousc
Re: Exception While Purging Query
Posted by Puneet Gupta <pu...@inmobi.com>.
Hi Tao
1. You can see the location of persistence directory in lens-site.xml
(lens.server.persist.location). Default location is
file:///tmp/lensserver. You can delete the entire directory contents or
just delete "query.final" to clear query service state alone.
2. The DB url is also in lens-site.xml (lens.server.db.jdbc.url). Default
url is jdbc:hsqldb:/tmp/lensserver/queries.db. The table would be
finished_queries. You should be able to delete row based on "handle" filter.
Thanks,
Puneet Gupta
On Wed, Aug 3, 2016 at 10:30 PM, Tao Yan <ty...@linkedin.com> wrote:
> Hi Amareshwari,
>
> 1. How do I clear lens persist state?
> 2. Which DB and which table stores the query handles?
>
> Thanks,
>
> On Tue, Aug 2, 2016 at 10:37 PM, amareshwarisr . <am...@gmail.com>
> wrote:
>
>> Tao,
>>
>> Seems the issue is not with new query having same handle. But the other
>> values seem to differ - esp. driverEndTime.
>>
>> Couple of options for you not to see the exception any more :
>> 1. Clear lens server persist state and restart. or
>> 2. Remove the entry from DB corresponding to the problematic query handle.
>>
>> Can you raise a bug with the above exception to get it fixed properly?
>>
>> Thanks
>> Amareshwari
>>
>> On Wed, Aug 3, 2016 at 9:42 AM, Tao Yan <ty...@linkedin.com> wrote:
>>
>>> Hi Lens Developers,
>>>
>>> I run the example queries in the '20 minutes demo' page for several
>>> times, and a query failed many time before it succeeded:
>>>
>>> lens-shell>query execute cube select product_id, store_sales from sales
>>> where time_range_in(order_time, '2015-04-11-00', '2015-04-13-00')
>>>
>>> Then, I found the Lens server keeps printing the following error log:
>>> 03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34]
>>> [QueryPurger] WARN org.apache.lens.server.query.LensServerDAO - Re insert
>>> happening in purge: [Ljava.lang.StackTraceElement;@354862c
>>> 03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34]
>>> [QueryPurger] WARN *org.apache.lens.server.query.QueryExecutionServiceImpl
>>> - Exception while purging query 27be8f4d-9d9a-43da-9c10-710ef0734ab3*
>>> java.sql.SQLException: Found different value pre-existing in DB while
>>> trying to insert finished query.* Old =* FinishedLensQuery(
>>> *handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*, userQuery=cube select
>>> product_id, store_sales from sales where time_range_in(order_time,
>>> '2015-04-11-00', '2015-04-13-00'), submitter=anonymous,
>>> submissionTime=1470184239496, startTime=1470184243802,
>>> endTime=1470184289999, result=null, status=FAILED, metadata=null,
>>> rows=null, fileSize=null, errorMessage=Stage-0(MOVE):Stage-0: has failed! ,
>>> driverStartTime=1470184239666, driverEndTime=1470184289989, queryName=,
>>> driverName=hive/hive1, selectedDriver=null)
>>> *New =* FinishedLensQuery(*handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*,
>>> userQuery=cube select product_id, store_sales from sales where
>>> time_range_in(order_time, '2015-04-11-00', '2015-04-13-00'),
>>> submitter=anonymous, submissionTime=1470184239496, startTime=1470184243802,
>>> endTime=1470184608172, result=null, status=FAILED, metadata=null,
>>> rows=null, fileSize=null, errorMessage=Error getting query status,
>>> driverStartTime=1470184239666, driverEndTime=0, queryName=,
>>> driverName=hive/hive1, selectedDriver=hive/hive1)
>>> at
>>> org.apache.lens.server.query.LensServerDAO.insertFinishedQuery(LensServerDAO.java:133)
>>> ~[classes/:na]
>>> at
>>> org.apache.lens.server.query.QueryExecutionServiceImpl$QueryPurger.run(QueryExecutionServiceImpl.java:1007)
>>> ~[classes/:na]
>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_31]
>>>
>>>
>>> It looks that the new query has the same handle as the old query. So,
>>> Lens servers complains about it. It seems other queries are not impacted,
>>> but this Warning messages are really annoying and impacting regular logs.
>>>
>>> Thanks,
>>> --
>>>
>>> *Tao Yan*
>>> Software Engineer
>>> Data Analytics Infrastructure Tools and Services
>>>
>>>
>>>
>>> 206.250.5345
>>> tyan@linkedin.com
>>> https://www.linkedin.com/in/taousc
>>>
>>
>>
>
>
> --
>
> *Tao Yan*
> Software Engineer
> Data Analytics Infrastructure Tools and Services
>
>
>
> 206.250.5345
> tyan@linkedin.com
> https://www.linkedin.com/in/taousc
>
--
_____________________________________________________________
The information contained in this communication is intended solely for the
use of the individual or entity to whom it is addressed and others
authorized to receive it. It may contain confidential or legally privileged
information. If you are not the intended recipient you are hereby notified
that any disclosure, copying, distribution or taking any action in reliance
on the contents of this information is strictly prohibited and may be
unlawful. If you have received this communication in error, please notify
us immediately by responding to this email and then delete it from your
system. The firm is neither liable for the proper and complete transmission
of the information contained in this communication nor for any delay in its
receipt.
Re: Exception While Purging Query
Posted by Tao Yan <ty...@linkedin.com>.
Hi Amareshwari,
1. How do I clear lens persist state?
2. Which DB and which table stores the query handles?
Thanks,
On Tue, Aug 2, 2016 at 10:37 PM, amareshwarisr . <am...@gmail.com>
wrote:
> Tao,
>
> Seems the issue is not with new query having same handle. But the other
> values seem to differ - esp. driverEndTime.
>
> Couple of options for you not to see the exception any more :
> 1. Clear lens server persist state and restart. or
> 2. Remove the entry from DB corresponding to the problematic query handle.
>
> Can you raise a bug with the above exception to get it fixed properly?
>
> Thanks
> Amareshwari
>
> On Wed, Aug 3, 2016 at 9:42 AM, Tao Yan <ty...@linkedin.com> wrote:
>
>> Hi Lens Developers,
>>
>> I run the example queries in the '20 minutes demo' page for several
>> times, and a query failed many time before it succeeded:
>>
>> lens-shell>query execute cube select product_id, store_sales from sales
>> where time_range_in(order_time, '2015-04-11-00', '2015-04-13-00')
>>
>> Then, I found the Lens server keeps printing the following error log:
>> 03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34] [QueryPurger]
>> WARN org.apache.lens.server.query.LensServerDAO - Re insert happening in
>> purge: [Ljava.lang.StackTraceElement;@354862c
>> 03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34] [QueryPurger]
>> WARN *org.apache.lens.server.query.QueryExecutionServiceImpl -
>> Exception while purging query 27be8f4d-9d9a-43da-9c10-710ef0734ab3*
>> java.sql.SQLException: Found different value pre-existing in DB while
>> trying to insert finished query.* Old =* FinishedLensQuery(
>> *handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*, userQuery=cube select
>> product_id, store_sales from sales where time_range_in(order_time,
>> '2015-04-11-00', '2015-04-13-00'), submitter=anonymous,
>> submissionTime=1470184239496, startTime=1470184243802,
>> endTime=1470184289999, result=null, status=FAILED, metadata=null,
>> rows=null, fileSize=null, errorMessage=Stage-0(MOVE):Stage-0: has failed! ,
>> driverStartTime=1470184239666, driverEndTime=1470184289989, queryName=,
>> driverName=hive/hive1, selectedDriver=null)
>> *New =* FinishedLensQuery(*handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*,
>> userQuery=cube select product_id, store_sales from sales where
>> time_range_in(order_time, '2015-04-11-00', '2015-04-13-00'),
>> submitter=anonymous, submissionTime=1470184239496, startTime=1470184243802,
>> endTime=1470184608172, result=null, status=FAILED, metadata=null,
>> rows=null, fileSize=null, errorMessage=Error getting query status,
>> driverStartTime=1470184239666, driverEndTime=0, queryName=,
>> driverName=hive/hive1, selectedDriver=hive/hive1)
>> at
>> org.apache.lens.server.query.LensServerDAO.insertFinishedQuery(LensServerDAO.java:133)
>> ~[classes/:na]
>> at
>> org.apache.lens.server.query.QueryExecutionServiceImpl$QueryPurger.run(QueryExecutionServiceImpl.java:1007)
>> ~[classes/:na]
>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_31]
>>
>>
>> It looks that the new query has the same handle as the old query. So,
>> Lens servers complains about it. It seems other queries are not impacted,
>> but this Warning messages are really annoying and impacting regular logs.
>>
>> Thanks,
>> --
>>
>> *Tao Yan*
>> Software Engineer
>> Data Analytics Infrastructure Tools and Services
>>
>>
>>
>> 206.250.5345
>> tyan@linkedin.com
>> https://www.linkedin.com/in/taousc
>>
>
>
--
*Tao Yan*
Software Engineer
Data Analytics Infrastructure Tools and Services
206.250.5345
tyan@linkedin.com
https://www.linkedin.com/in/taousc
Re: Exception While Purging Query
Posted by "amareshwarisr ." <am...@gmail.com>.
Tao,
Seems the issue is not with new query having same handle. But the other
values seem to differ - esp. driverEndTime.
Couple of options for you not to see the exception any more :
1. Clear lens server persist state and restart. or
2. Remove the entry from DB corresponding to the problematic query handle.
Can you raise a bug with the above exception to get it fixed properly?
Thanks
Amareshwari
On Wed, Aug 3, 2016 at 9:42 AM, Tao Yan <ty...@linkedin.com> wrote:
> Hi Lens Developers,
>
> I run the example queries in the '20 minutes demo' page for several times,
> and a query failed many time before it succeeded:
>
> lens-shell>query execute cube select product_id, store_sales from sales
> where time_range_in(order_time, '2015-04-11-00', '2015-04-13-00')
>
> Then, I found the Lens server keeps printing the following error log:
> 03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34] [QueryPurger]
> WARN org.apache.lens.server.query.LensServerDAO - Re insert happening in
> purge: [Ljava.lang.StackTraceElement;@354862c
> 03 Aug 2016 03:59:04 [b9bd9e01-d2e9-448e-9045-d329fb86be34] [QueryPurger]
> WARN *org.apache.lens.server.query.QueryExecutionServiceImpl - Exception
> while purging query 27be8f4d-9d9a-43da-9c10-710ef0734ab3*
> java.sql.SQLException: Found different value pre-existing in DB while
> trying to insert finished query.* Old =* FinishedLensQuery(
> *handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*, userQuery=cube select
> product_id, store_sales from sales where time_range_in(order_time,
> '2015-04-11-00', '2015-04-13-00'), submitter=anonymous,
> submissionTime=1470184239496, startTime=1470184243802,
> endTime=1470184289999, result=null, status=FAILED, metadata=null,
> rows=null, fileSize=null, errorMessage=Stage-0(MOVE):Stage-0: has failed! ,
> driverStartTime=1470184239666, driverEndTime=1470184289989, queryName=,
> driverName=hive/hive1, selectedDriver=null)
> *New =* FinishedLensQuery(*handle=27be8f4d-9d9a-43da-9c10-710ef0734ab3*,
> userQuery=cube select product_id, store_sales from sales where
> time_range_in(order_time, '2015-04-11-00', '2015-04-13-00'),
> submitter=anonymous, submissionTime=1470184239496, startTime=1470184243802,
> endTime=1470184608172, result=null, status=FAILED, metadata=null,
> rows=null, fileSize=null, errorMessage=Error getting query status,
> driverStartTime=1470184239666, driverEndTime=0, queryName=,
> driverName=hive/hive1, selectedDriver=hive/hive1)
> at
> org.apache.lens.server.query.LensServerDAO.insertFinishedQuery(LensServerDAO.java:133)
> ~[classes/:na]
> at
> org.apache.lens.server.query.QueryExecutionServiceImpl$QueryPurger.run(QueryExecutionServiceImpl.java:1007)
> ~[classes/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_31]
>
>
> It looks that the new query has the same handle as the old query. So, Lens
> servers complains about it. It seems other queries are not impacted, but
> this Warning messages are really annoying and impacting regular logs.
>
> Thanks,
> --
>
> *Tao Yan*
> Software Engineer
> Data Analytics Infrastructure Tools and Services
>
>
>
> 206.250.5345
> tyan@linkedin.com
> https://www.linkedin.com/in/taousc
>