You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Sanjiv Kumar <sa...@gmail.com> on 2017/01/03 14:31:10 UTC
while running long query drill stopped
Hello
I am using drill latest version (1.9) in embedded mode in Ubuntu
system.
My query is :-
If i run query having less number of records, drill
working fine.
But If i run long query having (8124042 records), drill
stopped without throwing any error.
NOTE:- My Ubuntu Server memory is 4GB. Is this cause problem? If not then
please tell me the solution.
Re: while running long query drill stopped
Posted by Paul Rogers <pr...@maprtech.com>.
Hi Sanjiv,
Drill should have written a log file. Does the log file contain an entry for the exception? Perhaps post the first five or so lines from that log entry (the whole stack trace can be pretty long.)
Was this a fatal exception from Drill or did the JVM itself die? (If the JVM dies, it often prints a log file that tells you it was a JVM issue.)
The memory limit impact rather depends on the kind of query you ran. Different bits of Drill have different memory demands. Please tell us a bit more about your query and the “shape” of your data: number of columns, width of the larger columns, etc.
With that info we may be able to suggest settings to work around the issue.
- Paul
> On Jan 3, 2017, at 6:31 AM, Sanjiv Kumar <sa...@gmail.com> wrote:
>
> Hello
> I am using drill latest version (1.9) in embedded mode in Ubuntu
> system.
> My query is :-
> If i run query having less number of records, drill
> working fine.
> But If i run long query having (8124042 records), drill
> stopped without throwing any error.
>
> NOTE:- My Ubuntu Server memory is 4GB. Is this cause problem? If not then
> please tell me the solution.
Re: while running long query drill stopped
Posted by Paul Rogers <pr...@maprtech.com>.
Hi Sanjiv,
Looks like you are using the JDBC storage plugin to access SQLServer. I’m not very familiar with this particular component. Anyone else have experience they can share with Sanjiv?
- Paul
> On Jan 4, 2017, at 2:29 AM, Sanjiv Kumar <sa...@gmail.com> wrote:
>
> Hi Paul
> My Tables have 14 columns and some column have less width and
> some column have more width, and total number of rows are 8124042.
> And My Log is:--->
>
> 2017-01-04 08:05:23,992 [2793533c-3fc5-5aed-dd29-691b47d4b0cb:foreman]
> INFO o.a.drill.exec.work.foreman.Foreman -
> Query text for query id 2793533c-3fc5-5aed-dd29-691b47d4b0cb:
> select * from <pluginName>.<databasesName>.<schemaName>.<tableName>
> 2017-01-04 08:05:24,012 [2793533c-3fc5-5aed-dd29-691b47d4b0cb:foreman]
> WARN o.a.d.e.store.jdbc.JdbcStoragePlugin - Failure while attempting to
> load JDBC schema.
> com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset
> at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1668)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1655)
> ~[sqljdbc41.jar:na]
> at com.microsoft.sqlserver.jdbc.TDSChannel.write(IOBuffer.java:1805)
> ~[sqljdbc41.jar:na]
> at com.microsoft.sqlserver.jdbc.TDSWriter.flush(IOBuffer.java:3581)
> ~[sqljdbc41.jar:na]
> at com.microsoft.sqlserver.jdbc.TDSWriter.writePacket(IOBuffer.java:3482)
> ~[sqljdbc41.jar:na]
> at com.microsoft.sqlserver.jdbc.TDSWriter.endMessage(IOBuffer.java:3062)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:6120)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:797)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:696)
> ~[sqljdbc41.jar:na]
> at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1716)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQueryInternal(SQLServerStatement.java:631)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getResultSetFromInternalQueries(SQLServerDatabaseMetaData.java:228)
> ~[sqljdbc41.jar:na]
> at
> com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getCatalogs(SQLServerDatabaseMetaData.java:382)
> ~[sqljdbc41.jar:na]
> at
> org.apache.commons.dbcp.DelegatingDatabaseMetaData.getCatalogs(DelegatingDatabaseMetaData.java:190)
> ~[commons-dbcp-1.4.jar:1.4]
> at
> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin$JdbcCatalogSchema.<init>(JdbcStoragePlugin.java:309)
> ~[drill-jdbc-storage-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.registerSchemas(JdbcStoragePlugin.java:430)
> [drill-jdbc-storage-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:61)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:155)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:145)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.ops.QueryContext.getNewDefaultSchema(QueryContext.java:131)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:62)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1008)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
>
> On Tue, Jan 3, 2017 at 8:01 PM, Sanjiv Kumar <sa...@gmail.com> wrote:
>
>> Hello
>> I am using drill latest version (1.9) in embedded mode in Ubuntu
>> system.
>> My query is :-
>> If i run query having less number of records, drill
>> working fine.
>> But If i run long query having (8124042 records), drill
>> stopped without throwing any error.
>>
>> NOTE:- My Ubuntu Server memory is 4GB. Is this cause problem? If not then
>> please tell me the solution.
>>
>>
>>
>
>
> --
> Thanks & Regards.
> Sanjiv Kumar.
Re: while running long query drill stopped
Posted by Sanjiv Kumar <sa...@gmail.com>.
Hi Paul
My Tables have 14 columns and some column have less width and
some column have more width, and total number of rows are 8124042.
And My Log is:--->
2017-01-04 08:05:23,992 [2793533c-3fc5-5aed-dd29-691b47d4b0cb:foreman]
INFO o.a.drill.exec.work.foreman.Foreman -
Query text for query id 2793533c-3fc5-5aed-dd29-691b47d4b0cb:
select * from <pluginName>.<databasesName>.<schemaName>.<tableName>
2017-01-04 08:05:24,012 [2793533c-3fc5-5aed-dd29-691b47d4b0cb:foreman]
WARN o.a.d.e.store.jdbc.JdbcStoragePlugin - Failure while attempting to
load JDBC schema.
com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1668)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1655)
~[sqljdbc41.jar:na]
at com.microsoft.sqlserver.jdbc.TDSChannel.write(IOBuffer.java:1805)
~[sqljdbc41.jar:na]
at com.microsoft.sqlserver.jdbc.TDSWriter.flush(IOBuffer.java:3581)
~[sqljdbc41.jar:na]
at com.microsoft.sqlserver.jdbc.TDSWriter.writePacket(IOBuffer.java:3482)
~[sqljdbc41.jar:na]
at com.microsoft.sqlserver.jdbc.TDSWriter.endMessage(IOBuffer.java:3062)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:6120)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:797)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:696)
~[sqljdbc41.jar:na]
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1716)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQueryInternal(SQLServerStatement.java:631)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getResultSetFromInternalQueries(SQLServerDatabaseMetaData.java:228)
~[sqljdbc41.jar:na]
at
com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getCatalogs(SQLServerDatabaseMetaData.java:382)
~[sqljdbc41.jar:na]
at
org.apache.commons.dbcp.DelegatingDatabaseMetaData.getCatalogs(DelegatingDatabaseMetaData.java:190)
~[commons-dbcp-1.4.jar:1.4]
at
org.apache.drill.exec.store.jdbc.JdbcStoragePlugin$JdbcCatalogSchema.<init>(JdbcStoragePlugin.java:309)
~[drill-jdbc-storage-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.registerSchemas(JdbcStoragePlugin.java:430)
[drill-jdbc-storage-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl$DrillSchemaFactory.registerSchemas(StoragePluginRegistryImpl.java:365)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:72)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.SchemaTreeProvider.createRootSchema(SchemaTreeProvider.java:61)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:155)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.ops.QueryContext.getRootSchema(QueryContext.java:145)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.ops.QueryContext.getNewDefaultSchema(QueryContext.java:131)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:62)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1008)
[drill-java-exec-1.9.0.jar:1.9.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:264)
[drill-java-exec-1.9.0.jar:1.9.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_111]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
On Tue, Jan 3, 2017 at 8:01 PM, Sanjiv Kumar <sa...@gmail.com> wrote:
> Hello
> I am using drill latest version (1.9) in embedded mode in Ubuntu
> system.
> My query is :-
> If i run query having less number of records, drill
> working fine.
> But If i run long query having (8124042 records), drill
> stopped without throwing any error.
>
> NOTE:- My Ubuntu Server memory is 4GB. Is this cause problem? If not then
> please tell me the solution.
>
>
>
--
Thanks & Regards.
Sanjiv Kumar.