You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Madhu Borkar <ba...@gmail.com> on 2013/11/19 07:16:10 UTC
How to load data in Drill
I have devlopment MapR cluster. I have installed Apache Drill using
binaries. I could run queries with sample-data.
I decided to load test data using pig. Here is my code
---------------------------------
REGISTER file:/home/mapr/lib/parquet-pig-bundle-1.2.5.jar
data = LOAD '$DATA'
USING PigStorage(',')
as (
user: chararray,
street: chararray,
state: chararray,
city: chararray,
type: chararray
);
A = LIMIT data 3;
dump A;
fs -rmr /mapr-hive/parquet/data;
STORE data INTO '/mapr-hive/parquet/data' using parquet.pig.ParquetStorer();
----------------------
Pig script writes to output. I moved data under apache drill directory.
select * from "real-data/users.parquet";
but when I try to query this data I get rpc error?
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: "04ac52e4-b49e-45b8-82c5-cf480154eefd"
endpoint {
address: "Node.etouch.net"
user_port: 31010
bit_port: 32011
}
error_type: 0
message: "Failure while running fragment. < NullPointerException"
]
Please, provide me help how to load data in Apache Drill.?
Is there MapR configuration needed for Drill?
Regards
Re: How to load data in Drill
Posted by Madhu Borkar <ba...@gmail.com>.
Hi Ben,
apache-drill-1.0.0-m1 is drill version,I installed binary.
I have attached log herewith.
Is my pig script looks okay to generate parquet fiel?
Thanks
-Madhu
On Tue, Nov 19, 2013 at 9:34 AM, Ben Becker <bb...@maprtech.com> wrote:
> Hi Madhu,
>
> Could you report which version you're using (e.g. 1.0.0-m1.rc3), and attach
> the logs when this happens? You can use Lilith to view the logs if you
> like (see instructions at the bottom of this
> page<
> https://cwiki.apache.org/confluence/display/DRILL/How+To+Run+Drill+with+Sample+Data#
> >
> ).
>
> Thanks,
> Ben
>
>
> On Mon, Nov 18, 2013 at 10:16 PM, Madhu Borkar <backmeupone@gmail.com
> >wrote:
>
> > I have devlopment MapR cluster. I have installed Apache Drill using
> > binaries. I could run queries with sample-data.
> >
> > I decided to load test data using pig. Here is my code
> >
> > ---------------------------------
> >
> > REGISTER file:/home/mapr/lib/parquet-pig-bundle-1.2.5.jar
> >
> > data = LOAD '$DATA'
> >
> > USING PigStorage(',')
> >
> > as (
> >
> > user: chararray,
> >
> > street: chararray,
> >
> > state: chararray,
> >
> > city: chararray,
> >
> > type: chararray
> >
> > );
> >
> >
> >
> > A = LIMIT data 3;
> >
> > dump A;
> >
> > fs -rmr /mapr-hive/parquet/data;
> >
> > STORE data INTO '/mapr-hive/parquet/data' using
> > parquet.pig.ParquetStorer();
> >
> > ----------------------
> >
> > Pig script writes to output. I moved data under apache drill directory.
> >
> >
> >
> > select * from "real-data/users.parquet";
> >
> >
> >
> > but when I try to query this data I get rpc error?
> >
> > Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure
> while
> > running query.[error_id: "04ac52e4-b49e-45b8-82c5-cf480154eefd"
> >
> > endpoint {
> >
> > address: "Node.etouch.net"
> >
> > user_port: 31010
> >
> > bit_port: 32011
> >
> > }
> >
> > error_type: 0
> >
> > message: "Failure while running fragment. < NullPointerException"
> >
> > ]
> >
> >
> >
> > Please, provide me help how to load data in Apache Drill.?
> >
> > Is there MapR configuration needed for Drill?
> >
> > Regards
> >
>
Re: How to load data in Drill
Posted by Tom Seddon <mr...@gmail.com>.
Hi,
I've done the same thing, converting one of the AMP Labs test data files
into parquet using Pig and then attempting to query via sqlline. I'm
querying the file in HDFS, (which I have tested successfully with the
provided sample files). Here is the error:
0: jdbc:drill:schema=parquet> select * from
"/user/tom/ranking-parquet-new/part-m-00000.parquet";
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: "da7ce1f7-b0e9-4720-bf10-333bf668869d"
endpoint {
address: "localhost"
user_port: 31010
bit_port: 32011
}
error_type: 0
message: "Failure while running fragment. < NullPointerException"
]
java.lang.RuntimeException: org.apache.drill.exec.rpc.RpcException: Remote
failure while running query.[error_id:
"da7ce1f7-b0e9-4720-bf10-333bf668869d"
endpoint {
address: "localhost"
user_port: 31010
bit_port: 32011
}
error_type: 0
message: "Failure while running fragment. < NullPointerException"
]
at
org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:44)
at
net.hydromatic.optiq.runtime.ObjectEnumeratorCursor.next(ObjectEnumeratorCursor.java:44)
at net.hydromatic.optiq.jdbc.OptiqResultSet.next(OptiqResultSet.java:162)
at sqlline.SqlLine$BufferedRows.<init>(SqlLine.java:2499)
at sqlline.SqlLine.print(SqlLine.java:1886)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3835)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3738)
at sqlline.SqlLine.dispatch(SqlLine.java:882)
at sqlline.SqlLine.begin(SqlLine.java:717)
at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:460)
at sqlline.SqlLine.main(SqlLine.java:443)
Caused by: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: "da7ce1f7-b0e9-4720-bf10-333bf668869d"
endpoint {
address: "localhost"
user_port: 31010
bit_port: 32011
}
error_type: 0
message: "Failure while running fragment. < NullPointerException"
]
at
org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)
at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79)
at
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48)
at
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33)
at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142)
at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
at
io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)
at
io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at
io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)
at
io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173)
at
io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)
at
io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
at java.lang.Thread.run(Thread.java:724)
0: jdbc:drill:schema=parquet>
On 22 November 2013 02:12, Madhu Borkar <ba...@gmail.com> wrote:
> Hi Ben,
> I don't have the same sqlline script that you have mentioned. This I found
> under bin
> bin/sqlline
>
>
>
> #!/bin/bash
>
> ARGS=();
> while [ $# -gt 0 ]
> do
> case "$1" in
> -q) shift;
> QUERY=$1;;
> -e) shift;
> QUERY=$1;;
> -f) shift;
> FILE=$1;;
> *) ARGS+=($1);;
> esac
> shift
> done
>
> bin=`dirname "${BASH_SOURCE-$0}"`
> bin=`cd "$bin">/dev/null; pwd`
>
> . "$bin"/drill-config.sh
>
> if [ -z $JAVA_HOME ]
> then
> JAVA=`which java`
> else
> JAVA=`find -L $JAVA_HOME -name java | head -n 1`
> fi
>
> if [ -e $JAVA ]; then
> echo ""
> else
> echo "Java not found."
> exit 1
> fi
>
> $JAVA -version 2>&1 | grep "version" | egrep -e "1.7" > /dev/null
> if [ $? -ne 0 ]; then
> echo "Java 1.7 is required to run Apache Drill."
> exit 1
> fi
>
> # get log directory
> if [ "$DRILL_LOG_DIR" = "" ]; then
> export DRILL_LOG_DIR=/var/log/drill
> fi
>
>
> CP=$DRILL_HOME/jars/*:$CP
> CP=$DRILL_HOME/lib/*:$CP
> CP=$DRILL_HOME/contrib/*:$CP
> CP=$HADOOP_CLASSPATH:$CP
>
> CP=$DRILL_CONF_DIR:$CP
>
> DRILL_SHELL_JAVA_OPTS="$DRILL_SHELL_JAVA_OPTS
> -Dlog.path=$DRILL_LOG_DIR/sqlline.log"
>
> if [ -n "$QUERY" ]
> then
> echo $QUERY | exec $JAVA $DRILL_SHELL_JAVA_OPTS $DRILL_JAVA_OPTS -cp $CP
> sqlline.SqlLine "${ARGS[@]}"
> elif [ -n "$FILE" ]
> then
> exec $JAVA $DRILL_SHELL_JAVA_OPTS $DRILL_JAVA_OPTS -cp $CP
> sqlline.SqlLine "${ARGS[@]}" --run=$FILE
> else
> exec $JAVA $DRILL_SHELL_JAVA_OPTS $DRILL_JAVA_OPTS -cp $CP
> sqlline.SqlLine "${ARGS[@]}"
> fi
> -----------------------
> Is this because I installed from binary?
> Thanks
> -Madhu
>
> On Thu, Nov 21, 2013 at 9:50 AM, Ben Becker <bb...@maprtech.com> wrote:
>
> > Hi Madhu,
> >
> > Could you attach the complete logs during this run? The easiest way is
> > probably to edit the sqlline shell script and remove the '
> > -Dlogback.configurationFile=...' argument like so:
> >
> > -exec java -Xmx2G
> > -Dlogback.configurationFile=./sqlparser/src/test/resources/logback.xml
> -cp
> > ./:"$(cat .classpath)" sqlline.SqlLine --verbose=true "$@"
> > *+exec java -Xmx2G -cp ./:"$(cat .classpath)" sqlline.SqlLine
> > --verbose=true "$@"*
> >
> > This will cause all log output to be directed to the console.
> > Alternatively, you may want to log to a file or another facility by
> > editing the aforementioned logback.xml file. One last note -- although
> > unlikely, parts of the dataset may be included in the logs, so be sure to
> > redact any sensitive information from the logs before posting.
> >
> > Thanks,
> > Ben
> >
> >
> >
> > On Tue, Nov 19, 2013 at 6:36 PM, Madhu Borkar <ba...@gmail.com>
> > wrote:
> >
> > > Ben,
> > > Here is log snippet using lilith..
> > > <?xml version='1.0' encoding='UTF-8'?>
> > > <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "
> > > http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> > > <html xmlns='http://www.w3.org/1999/xhtml
> > '><head><title>Title</title><link
> > > href='detailsView.css' rel='stylesheet'
> > > type='text/css'/></head><body><table><tr
> > > class='even'><th>Message</th><td><pre>Exception during
> > > query</pre></td></tr><tr class='odd'><th>Throwable</th><td
> > > class='throwableContainer'><div class='throwable'><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>org.apache.drill.exec.rpc.RpcException</a><pre><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>Remote
> > > failure while running query.[error_id:
> > > "b9ce54dd-085c-435d-b12f-e9896c60e864"
> > > endpoint {
> > > address: "Node-0.etouch.net"
> > > user_port: 31010
> > > bit_port: 32011
> > > }
> > > error_type: 0
> > > message: "Failure while running fragment. < NullPointerException"
> > > ]</a></pre><hr/><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>at org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79)'>at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> > >
> > >
> >
> href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> > >
> > >
> >
> href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) ~[netty-codec-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) ~[netty-codec-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173)'>at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173) ~[netty-codec-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)'>at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100)'>at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497)'>at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465)'>at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359)'>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)'>at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101) ~[netty-common-4.0.7.Final.jar:na]</a><a
> > >
> > >
> >
> href='ste://java.lang.Thread.run(Thread.java:744)'>at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]</a></div></td></tr><tr
> > > class='even'><th>Level</th><td>ERROR</td></tr><tr
> > >
> > >
> >
> class='odd'><th>Logger</th><td>org.apache.drill.sql.client.full.ResultEnumerator</td></tr><tr
> > > class='even'><th>Thread</th><td>main (id=1)</td></tr><tr
> > > class='odd'><th>Timestamp</th><td>2013-11-19
> > 18:26:42.023-0800</td></tr><tr
> > > class='even'><th>Call Location</th><td><a
> > >
> > >
> >
> href='ste://org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)'>at org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)</a></td></tr><tr
> > >
> > >
> >
> class='odd'><th>Logger Context</th><td><table><tr><th>Name</th><td>default</td></tr><tr><th>Birthtime</th><td>2013-11-19
> > >
> > >
> >
> 18:26:06.371-0800</td></tr><tr><th>Properties</th><td><table><tr><th>Key</th><th>Value</th></tr><tr><td>HOSTNAME</td><td>
> > > Node-0.etouch.net</td></tr></table></td></tr></table></td></tr><tr
> > >
> > >
> >
> class='even'><th>Source ID</th><td>127.0.0.1-20131119T182606709</td></tr><tr
> > >
> class='odd'><th>Local ID</th><td>71</td></tr></table></body></html>
> > >
> > >
> > > On Tue, Nov 19, 2013 at 9:34 AM, Ben Becker <bb...@maprtech.com>
> > wrote:
> > >
> > > > Hi Madhu,
> > > >
> > > > Could you report which version you're using (e.g. 1.0.0-m1.rc3), and
> > > attach
> > > > the logs when this happens? You can use Lilith to view the logs if
> you
> > > > like (see instructions at the bottom of this
> > > > page<
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/DRILL/How+To+Run+Drill+with+Sample+Data#
> > > > >
> > > > ).
> > > >
> > > > Thanks,
> > > > Ben
> > > >
> > > >
> > > > On Mon, Nov 18, 2013 at 10:16 PM, Madhu Borkar <
> backmeupone@gmail.com
> > > > >wrote:
> > > >
> > > > > I have devlopment MapR cluster. I have installed Apache Drill using
> > > > > binaries. I could run queries with sample-data.
> > > > >
> > > > > I decided to load test data using pig. Here is my code
> > > > >
> > > > > ---------------------------------
> > > > >
> > > > > REGISTER file:/home/mapr/lib/parquet-pig-bundle-1.2.5.jar
> > > > >
> > > > > data = LOAD '$DATA'
> > > > >
> > > > > USING PigStorage(',')
> > > > >
> > > > > as (
> > > > >
> > > > > user: chararray,
> > > > >
> > > > > street: chararray,
> > > > >
> > > > > state: chararray,
> > > > >
> > > > > city: chararray,
> > > > >
> > > > > type: chararray
> > > > >
> > > > > );
> > > > >
> > > > >
> > > > >
> > > > > A = LIMIT data 3;
> > > > >
> > > > > dump A;
> > > > >
> > > > > fs -rmr /mapr-hive/parquet/data;
> > > > >
> > > > > STORE data INTO '/mapr-hive/parquet/data' using
> > > > > parquet.pig.ParquetStorer();
> > > > >
> > > > > ----------------------
> > > > >
> > > > > Pig script writes to output. I moved data under apache drill
> > directory.
> > > > >
> > > > >
> > > > >
> > > > > select * from "real-data/users.parquet";
> > > > >
> > > > >
> > > > >
> > > > > but when I try to query this data I get rpc error?
> > > > >
> > > > > Query failed: org.apache.drill.exec.rpc.RpcException: Remote
> failure
> > > > while
> > > > > running query.[error_id: "04ac52e4-b49e-45b8-82c5-cf480154eefd"
> > > > >
> > > > > endpoint {
> > > > >
> > > > > address: "Node.etouch.net"
> > > > >
> > > > > user_port: 31010
> > > > >
> > > > > bit_port: 32011
> > > > >
> > > > > }
> > > > >
> > > > > error_type: 0
> > > > >
> > > > > message: "Failure while running fragment. < NullPointerException"
> > > > >
> > > > > ]
> > > > >
> > > > >
> > > > >
> > > > > Please, provide me help how to load data in Apache Drill.?
> > > > >
> > > > > Is there MapR configuration needed for Drill?
> > > > >
> > > > > Regards
> > > > >
> > > >
> > >
> >
>
Re: How to load data in Drill
Posted by Madhu Borkar <ba...@gmail.com>.
Hi Ben,
I don't have the same sqlline script that you have mentioned. This I found
under bin
bin/sqlline
#!/bin/bash
ARGS=();
while [ $# -gt 0 ]
do
case "$1" in
-q) shift;
QUERY=$1;;
-e) shift;
QUERY=$1;;
-f) shift;
FILE=$1;;
*) ARGS+=($1);;
esac
shift
done
bin=`dirname "${BASH_SOURCE-$0}"`
bin=`cd "$bin">/dev/null; pwd`
. "$bin"/drill-config.sh
if [ -z $JAVA_HOME ]
then
JAVA=`which java`
else
JAVA=`find -L $JAVA_HOME -name java | head -n 1`
fi
if [ -e $JAVA ]; then
echo ""
else
echo "Java not found."
exit 1
fi
$JAVA -version 2>&1 | grep "version" | egrep -e "1.7" > /dev/null
if [ $? -ne 0 ]; then
echo "Java 1.7 is required to run Apache Drill."
exit 1
fi
# get log directory
if [ "$DRILL_LOG_DIR" = "" ]; then
export DRILL_LOG_DIR=/var/log/drill
fi
CP=$DRILL_HOME/jars/*:$CP
CP=$DRILL_HOME/lib/*:$CP
CP=$DRILL_HOME/contrib/*:$CP
CP=$HADOOP_CLASSPATH:$CP
CP=$DRILL_CONF_DIR:$CP
DRILL_SHELL_JAVA_OPTS="$DRILL_SHELL_JAVA_OPTS
-Dlog.path=$DRILL_LOG_DIR/sqlline.log"
if [ -n "$QUERY" ]
then
echo $QUERY | exec $JAVA $DRILL_SHELL_JAVA_OPTS $DRILL_JAVA_OPTS -cp $CP
sqlline.SqlLine "${ARGS[@]}"
elif [ -n "$FILE" ]
then
exec $JAVA $DRILL_SHELL_JAVA_OPTS $DRILL_JAVA_OPTS -cp $CP
sqlline.SqlLine "${ARGS[@]}" --run=$FILE
else
exec $JAVA $DRILL_SHELL_JAVA_OPTS $DRILL_JAVA_OPTS -cp $CP
sqlline.SqlLine "${ARGS[@]}"
fi
-----------------------
Is this because I installed from binary?
Thanks
-Madhu
On Thu, Nov 21, 2013 at 9:50 AM, Ben Becker <bb...@maprtech.com> wrote:
> Hi Madhu,
>
> Could you attach the complete logs during this run? The easiest way is
> probably to edit the sqlline shell script and remove the '
> -Dlogback.configurationFile=...' argument like so:
>
> -exec java -Xmx2G
> -Dlogback.configurationFile=./sqlparser/src/test/resources/logback.xml -cp
> ./:"$(cat .classpath)" sqlline.SqlLine --verbose=true "$@"
> *+exec java -Xmx2G -cp ./:"$(cat .classpath)" sqlline.SqlLine
> --verbose=true "$@"*
>
> This will cause all log output to be directed to the console.
> Alternatively, you may want to log to a file or another facility by
> editing the aforementioned logback.xml file. One last note -- although
> unlikely, parts of the dataset may be included in the logs, so be sure to
> redact any sensitive information from the logs before posting.
>
> Thanks,
> Ben
>
>
>
> On Tue, Nov 19, 2013 at 6:36 PM, Madhu Borkar <ba...@gmail.com>
> wrote:
>
> > Ben,
> > Here is log snippet using lilith..
> > <?xml version='1.0' encoding='UTF-8'?>
> > <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "
> > http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> > <html xmlns='http://www.w3.org/1999/xhtml
> '><head><title>Title</title><link
> > href='detailsView.css' rel='stylesheet'
> > type='text/css'/></head><body><table><tr
> > class='even'><th>Message</th><td><pre>Exception during
> > query</pre></td></tr><tr class='odd'><th>Throwable</th><td
> > class='throwableContainer'><div class='throwable'><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>org.apache.drill.exec.rpc.RpcException</a><pre><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>Remote
> > failure while running query.[error_id:
> > "b9ce54dd-085c-435d-b12f-e9896c60e864"
> > endpoint {
> > address: "Node-0.etouch.net"
> > user_port: 31010
> > bit_port: 32011
> > }
> > error_type: 0
> > message: "Failure while running fragment. < NullPointerException"
> > ]</a></pre><hr/><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>at org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79)'>at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> >
> >
> href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
> >
> >
> href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) ~[netty-codec-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) ~[netty-codec-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173)'>at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173) ~[netty-codec-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)'>at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100)'>at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497)'>at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465)'>at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359)'>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359) ~[netty-transport-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)'>at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101) ~[netty-common-4.0.7.Final.jar:na]</a><a
> >
> >
> href='ste://java.lang.Thread.run(Thread.java:744)'>at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]</a></div></td></tr><tr
> > class='even'><th>Level</th><td>ERROR</td></tr><tr
> >
> >
> class='odd'><th>Logger</th><td>org.apache.drill.sql.client.full.ResultEnumerator</td></tr><tr
> > class='even'><th>Thread</th><td>main (id=1)</td></tr><tr
> > class='odd'><th>Timestamp</th><td>2013-11-19
> 18:26:42.023-0800</td></tr><tr
> > class='even'><th>Call Location</th><td><a
> >
> >
> href='ste://org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)'>at org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)</a></td></tr><tr
> >
> >
> class='odd'><th>Logger Context</th><td><table><tr><th>Name</th><td>default</td></tr><tr><th>Birthtime</th><td>2013-11-19
> >
> >
> 18:26:06.371-0800</td></tr><tr><th>Properties</th><td><table><tr><th>Key</th><th>Value</th></tr><tr><td>HOSTNAME</td><td>
> > Node-0.etouch.net</td></tr></table></td></tr></table></td></tr><tr
> >
> >
> class='even'><th>Source ID</th><td>127.0.0.1-20131119T182606709</td></tr><tr
> > class='odd'><th>Local ID</th><td>71</td></tr></table></body></html>
> >
> >
> > On Tue, Nov 19, 2013 at 9:34 AM, Ben Becker <bb...@maprtech.com>
> wrote:
> >
> > > Hi Madhu,
> > >
> > > Could you report which version you're using (e.g. 1.0.0-m1.rc3), and
> > attach
> > > the logs when this happens? You can use Lilith to view the logs if you
> > > like (see instructions at the bottom of this
> > > page<
> > >
> >
> https://cwiki.apache.org/confluence/display/DRILL/How+To+Run+Drill+with+Sample+Data#
> > > >
> > > ).
> > >
> > > Thanks,
> > > Ben
> > >
> > >
> > > On Mon, Nov 18, 2013 at 10:16 PM, Madhu Borkar <backmeupone@gmail.com
> > > >wrote:
> > >
> > > > I have devlopment MapR cluster. I have installed Apache Drill using
> > > > binaries. I could run queries with sample-data.
> > > >
> > > > I decided to load test data using pig. Here is my code
> > > >
> > > > ---------------------------------
> > > >
> > > > REGISTER file:/home/mapr/lib/parquet-pig-bundle-1.2.5.jar
> > > >
> > > > data = LOAD '$DATA'
> > > >
> > > > USING PigStorage(',')
> > > >
> > > > as (
> > > >
> > > > user: chararray,
> > > >
> > > > street: chararray,
> > > >
> > > > state: chararray,
> > > >
> > > > city: chararray,
> > > >
> > > > type: chararray
> > > >
> > > > );
> > > >
> > > >
> > > >
> > > > A = LIMIT data 3;
> > > >
> > > > dump A;
> > > >
> > > > fs -rmr /mapr-hive/parquet/data;
> > > >
> > > > STORE data INTO '/mapr-hive/parquet/data' using
> > > > parquet.pig.ParquetStorer();
> > > >
> > > > ----------------------
> > > >
> > > > Pig script writes to output. I moved data under apache drill
> directory.
> > > >
> > > >
> > > >
> > > > select * from "real-data/users.parquet";
> > > >
> > > >
> > > >
> > > > but when I try to query this data I get rpc error?
> > > >
> > > > Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure
> > > while
> > > > running query.[error_id: "04ac52e4-b49e-45b8-82c5-cf480154eefd"
> > > >
> > > > endpoint {
> > > >
> > > > address: "Node.etouch.net"
> > > >
> > > > user_port: 31010
> > > >
> > > > bit_port: 32011
> > > >
> > > > }
> > > >
> > > > error_type: 0
> > > >
> > > > message: "Failure while running fragment. < NullPointerException"
> > > >
> > > > ]
> > > >
> > > >
> > > >
> > > > Please, provide me help how to load data in Apache Drill.?
> > > >
> > > > Is there MapR configuration needed for Drill?
> > > >
> > > > Regards
> > > >
> > >
> >
>
Re: How to load data in Drill
Posted by Ben Becker <bb...@maprtech.com>.
Hi Madhu,
Could you attach the complete logs during this run? The easiest way is
probably to edit the sqlline shell script and remove the '
-Dlogback.configurationFile=...' argument like so:
-exec java -Xmx2G
-Dlogback.configurationFile=./sqlparser/src/test/resources/logback.xml -cp
./:"$(cat .classpath)" sqlline.SqlLine --verbose=true "$@"
*+exec java -Xmx2G -cp ./:"$(cat .classpath)" sqlline.SqlLine
--verbose=true "$@"*
This will cause all log output to be directed to the console.
Alternatively, you may want to log to a file or another facility by
editing the aforementioned logback.xml file. One last note -- although
unlikely, parts of the dataset may be included in the logs, so be sure to
redact any sensitive information from the logs before posting.
Thanks,
Ben
On Tue, Nov 19, 2013 at 6:36 PM, Madhu Borkar <ba...@gmail.com> wrote:
> Ben,
> Here is log snippet using lilith..
> <?xml version='1.0' encoding='UTF-8'?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "
> http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> <html xmlns='http://www.w3.org/1999/xhtml'><head><title>Title</title><link
> href='detailsView.css' rel='stylesheet'
> type='text/css'/></head><body><table><tr
> class='even'><th>Message</th><td><pre>Exception during
> query</pre></td></tr><tr class='odd'><th>Throwable</th><td
> class='throwableContainer'><div class='throwable'><a
>
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>org.apache.drill.exec.rpc.RpcException</a><pre><a
>
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>Remote
> failure while running query.[error_id:
> "b9ce54dd-085c-435d-b12f-e9896c60e864"
> endpoint {
> address: "Node-0.etouch.net"
> user_port: 31010
> bit_port: 32011
> }
> error_type: 0
> message: "Failure while running fragment. < NullPointerException"
> ]</a></pre><hr/><a
>
> href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>at org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
>
> href='ste://org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79)'>at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
>
> href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
>
> href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
>
> href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
>
> href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
>
> href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) ~[netty-codec-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) ~[netty-codec-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173)'>at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173) ~[netty-codec-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)'>at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100)'>at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497)'>at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465)'>at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359)'>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359) ~[netty-transport-4.0.7.Final.jar:na]</a><a
>
> href='ste://io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)'>at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101) ~[netty-common-4.0.7.Final.jar:na]</a><a
>
> href='ste://java.lang.Thread.run(Thread.java:744)'>at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]</a></div></td></tr><tr
> class='even'><th>Level</th><td>ERROR</td></tr><tr
>
> class='odd'><th>Logger</th><td>org.apache.drill.sql.client.full.ResultEnumerator</td></tr><tr
> class='even'><th>Thread</th><td>main (id=1)</td></tr><tr
> class='odd'><th>Timestamp</th><td>2013-11-19 18:26:42.023-0800</td></tr><tr
> class='even'><th>Call Location</th><td><a
>
> href='ste://org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)'>at org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)</a></td></tr><tr
>
> class='odd'><th>Logger Context</th><td><table><tr><th>Name</th><td>default</td></tr><tr><th>Birthtime</th><td>2013-11-19
>
> 18:26:06.371-0800</td></tr><tr><th>Properties</th><td><table><tr><th>Key</th><th>Value</th></tr><tr><td>HOSTNAME</td><td>
> Node-0.etouch.net</td></tr></table></td></tr></table></td></tr><tr
>
> class='even'><th>Source ID</th><td>127.0.0.1-20131119T182606709</td></tr><tr
> class='odd'><th>Local ID</th><td>71</td></tr></table></body></html>
>
>
> On Tue, Nov 19, 2013 at 9:34 AM, Ben Becker <bb...@maprtech.com> wrote:
>
> > Hi Madhu,
> >
> > Could you report which version you're using (e.g. 1.0.0-m1.rc3), and
> attach
> > the logs when this happens? You can use Lilith to view the logs if you
> > like (see instructions at the bottom of this
> > page<
> >
> https://cwiki.apache.org/confluence/display/DRILL/How+To+Run+Drill+with+Sample+Data#
> > >
> > ).
> >
> > Thanks,
> > Ben
> >
> >
> > On Mon, Nov 18, 2013 at 10:16 PM, Madhu Borkar <backmeupone@gmail.com
> > >wrote:
> >
> > > I have devlopment MapR cluster. I have installed Apache Drill using
> > > binaries. I could run queries with sample-data.
> > >
> > > I decided to load test data using pig. Here is my code
> > >
> > > ---------------------------------
> > >
> > > REGISTER file:/home/mapr/lib/parquet-pig-bundle-1.2.5.jar
> > >
> > > data = LOAD '$DATA'
> > >
> > > USING PigStorage(',')
> > >
> > > as (
> > >
> > > user: chararray,
> > >
> > > street: chararray,
> > >
> > > state: chararray,
> > >
> > > city: chararray,
> > >
> > > type: chararray
> > >
> > > );
> > >
> > >
> > >
> > > A = LIMIT data 3;
> > >
> > > dump A;
> > >
> > > fs -rmr /mapr-hive/parquet/data;
> > >
> > > STORE data INTO '/mapr-hive/parquet/data' using
> > > parquet.pig.ParquetStorer();
> > >
> > > ----------------------
> > >
> > > Pig script writes to output. I moved data under apache drill directory.
> > >
> > >
> > >
> > > select * from "real-data/users.parquet";
> > >
> > >
> > >
> > > but when I try to query this data I get rpc error?
> > >
> > > Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure
> > while
> > > running query.[error_id: "04ac52e4-b49e-45b8-82c5-cf480154eefd"
> > >
> > > endpoint {
> > >
> > > address: "Node.etouch.net"
> > >
> > > user_port: 31010
> > >
> > > bit_port: 32011
> > >
> > > }
> > >
> > > error_type: 0
> > >
> > > message: "Failure while running fragment. < NullPointerException"
> > >
> > > ]
> > >
> > >
> > >
> > > Please, provide me help how to load data in Apache Drill.?
> > >
> > > Is there MapR configuration needed for Drill?
> > >
> > > Regards
> > >
> >
>
Re: How to load data in Drill
Posted by Madhu Borkar <ba...@gmail.com>.
Ben,
Here is log snippet using lilith..
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "
http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns='http://www.w3.org/1999/xhtml'><head><title>Title</title><link
href='detailsView.css' rel='stylesheet'
type='text/css'/></head><body><table><tr
class='even'><th>Message</th><td><pre>Exception during
query</pre></td></tr><tr class='odd'><th>Throwable</th><td
class='throwableContainer'><div class='throwable'><a
href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>org.apache.drill.exec.rpc.RpcException</a><pre><a
href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>Remote
failure while running query.[error_id:
"b9ce54dd-085c-435d-b12f-e9896c60e864"
endpoint {
address: "Node-0.etouch.net"
user_port: 31010
bit_port: 32011
}
error_type: 0
message: "Failure while running fragment. < NullPointerException"
]</a></pre><hr/><a
href='ste://org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72)'>at org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:72) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
href='ste://org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79)'>at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:79) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:48) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
href='ste://org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33)'>at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:33) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:142) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
href='ste://org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127)'>at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:127) ~[java-exec-1.0.0-m1-rebuffed.jar:1.0.0-m1]</a><a
href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89) ~[netty-codec-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)'>at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) ~[netty-codec-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173)'>at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:173) ~[netty-codec-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334)'>at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:334) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)'>at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)'>at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100)'>at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:100) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497)'>at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:497) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465)'>at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:465) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359)'>at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:359) ~[netty-transport-4.0.7.Final.jar:na]</a><a
href='ste://io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)'>at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101) ~[netty-common-4.0.7.Final.jar:na]</a><a
href='ste://java.lang.Thread.run(Thread.java:744)'>at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]</a></div></td></tr><tr
class='even'><th>Level</th><td>ERROR</td></tr><tr
class='odd'><th>Logger</th><td>org.apache.drill.sql.client.full.ResultEnumerator</td></tr><tr
class='even'><th>Thread</th><td>main (id=1)</td></tr><tr
class='odd'><th>Timestamp</th><td>2013-11-19 18:26:42.023-0800</td></tr><tr
class='even'><th>Call Location</th><td><a
href='ste://org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)'>at org.apache.drill.sql.client.full.ResultEnumerator.moveNext(ResultEnumerator.java:43)</a></td></tr><tr
class='odd'><th>Logger Context</th><td><table><tr><th>Name</th><td>default</td></tr><tr><th>Birthtime</th><td>2013-11-19
18:26:06.371-0800</td></tr><tr><th>Properties</th><td><table><tr><th>Key</th><th>Value</th></tr><tr><td>HOSTNAME</td><td>
Node-0.etouch.net</td></tr></table></td></tr></table></td></tr><tr
class='even'><th>Source ID</th><td>127.0.0.1-20131119T182606709</td></tr><tr
class='odd'><th>Local ID</th><td>71</td></tr></table></body></html>
On Tue, Nov 19, 2013 at 9:34 AM, Ben Becker <bb...@maprtech.com> wrote:
> Hi Madhu,
>
> Could you report which version you're using (e.g. 1.0.0-m1.rc3), and attach
> the logs when this happens? You can use Lilith to view the logs if you
> like (see instructions at the bottom of this
> page<
> https://cwiki.apache.org/confluence/display/DRILL/How+To+Run+Drill+with+Sample+Data#
> >
> ).
>
> Thanks,
> Ben
>
>
> On Mon, Nov 18, 2013 at 10:16 PM, Madhu Borkar <backmeupone@gmail.com
> >wrote:
>
> > I have devlopment MapR cluster. I have installed Apache Drill using
> > binaries. I could run queries with sample-data.
> >
> > I decided to load test data using pig. Here is my code
> >
> > ---------------------------------
> >
> > REGISTER file:/home/mapr/lib/parquet-pig-bundle-1.2.5.jar
> >
> > data = LOAD '$DATA'
> >
> > USING PigStorage(',')
> >
> > as (
> >
> > user: chararray,
> >
> > street: chararray,
> >
> > state: chararray,
> >
> > city: chararray,
> >
> > type: chararray
> >
> > );
> >
> >
> >
> > A = LIMIT data 3;
> >
> > dump A;
> >
> > fs -rmr /mapr-hive/parquet/data;
> >
> > STORE data INTO '/mapr-hive/parquet/data' using
> > parquet.pig.ParquetStorer();
> >
> > ----------------------
> >
> > Pig script writes to output. I moved data under apache drill directory.
> >
> >
> >
> > select * from "real-data/users.parquet";
> >
> >
> >
> > but when I try to query this data I get rpc error?
> >
> > Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure
> while
> > running query.[error_id: "04ac52e4-b49e-45b8-82c5-cf480154eefd"
> >
> > endpoint {
> >
> > address: "Node.etouch.net"
> >
> > user_port: 31010
> >
> > bit_port: 32011
> >
> > }
> >
> > error_type: 0
> >
> > message: "Failure while running fragment. < NullPointerException"
> >
> > ]
> >
> >
> >
> > Please, provide me help how to load data in Apache Drill.?
> >
> > Is there MapR configuration needed for Drill?
> >
> > Regards
> >
>
Re: How to load data in Drill
Posted by Ben Becker <bb...@maprtech.com>.
Hi Madhu,
Could you report which version you're using (e.g. 1.0.0-m1.rc3), and attach
the logs when this happens? You can use Lilith to view the logs if you
like (see instructions at the bottom of this
page<https://cwiki.apache.org/confluence/display/DRILL/How+To+Run+Drill+with+Sample+Data#>
).
Thanks,
Ben
On Mon, Nov 18, 2013 at 10:16 PM, Madhu Borkar <ba...@gmail.com>wrote:
> I have devlopment MapR cluster. I have installed Apache Drill using
> binaries. I could run queries with sample-data.
>
> I decided to load test data using pig. Here is my code
>
> ---------------------------------
>
> REGISTER file:/home/mapr/lib/parquet-pig-bundle-1.2.5.jar
>
> data = LOAD '$DATA'
>
> USING PigStorage(',')
>
> as (
>
> user: chararray,
>
> street: chararray,
>
> state: chararray,
>
> city: chararray,
>
> type: chararray
>
> );
>
>
>
> A = LIMIT data 3;
>
> dump A;
>
> fs -rmr /mapr-hive/parquet/data;
>
> STORE data INTO '/mapr-hive/parquet/data' using
> parquet.pig.ParquetStorer();
>
> ----------------------
>
> Pig script writes to output. I moved data under apache drill directory.
>
>
>
> select * from "real-data/users.parquet";
>
>
>
> but when I try to query this data I get rpc error?
>
> Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
> running query.[error_id: "04ac52e4-b49e-45b8-82c5-cf480154eefd"
>
> endpoint {
>
> address: "Node.etouch.net"
>
> user_port: 31010
>
> bit_port: 32011
>
> }
>
> error_type: 0
>
> message: "Failure while running fragment. < NullPointerException"
>
> ]
>
>
>
> Please, provide me help how to load data in Apache Drill.?
>
> Is there MapR configuration needed for Drill?
>
> Regards
>