You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by lei liu <li...@gmail.com> on 2010/10/25 15:04:26 UTC
I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command,
but ResultSet don't return value
I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command, but
ResultSet don't return value, the code like below:
Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver");
hiveConnection = DriverManager.getConnection("jdbc:hive://
10.232.11.53:10000/default" , "root", "root");
statement = hiveConnection.createStatement();
String sql = "dfs -mkdir
hdfs://hdpnn:9000//group/tbprofile/mianfeng/scratch/test/001/ ";
ResultSet result = statement.executeQuery(sql); //the ResultSet
don't return any value.
while(result.next()) {
System.out.println("day_id =" + result.getString(1) );//the
}
statement.execute("set remove.scratch.dir=ture");
statement.close();
I run the command in CLI, there is output content, the content like below
:
drwxr-x--- - mianfeng cug-tbprofile 0 2010-10-25 20:55
/group/tbprofile/mianfeng/scratch/test/001
Re: I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/"
command, but ResultSet don't return value
Posted by Edward Capriolo <ed...@gmail.com>.
On Mon, Oct 25, 2010 at 1:42 PM, lei liu <li...@gmail.com> wrote:
> Hi Edward Capriolo,
> Thank you for your replay.
>
> I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command , if
> there is the directory in HDFS, the JDBC don't throw Exception, if there is
> not the directory in HDFS, the JDBC will throw Exception. Could you tell I
> can use the way to judge whether the directory is in HDFS?
>
>
> Thanks.
>
> 2010/10/25 Edward Capriolo <ed...@gmail.com>
>>
>> On Mon, Oct 25, 2010 at 9:04 AM, lei liu <li...@gmail.com> wrote:
>> > I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command, but
>> > ResultSet don't return value, the code like below:
>> >
>> > Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver");
>> > hiveConnection =
>> > DriverManager.getConnection("jdbc:hive://10.232.11.53:10000/default" ,
>> > "root", "root");
>> > statement = hiveConnection.createStatement();
>> > String sql = "dfs -mkdir
>> > hdfs://hdpnn:9000//group/tbprofile/mianfeng/scratch/test/001/ ";
>> >
>> > ResultSet result = statement.executeQuery(sql); //the
>> > ResultSet
>> > don't return any value.
>> > while(result.next()) {
>> >
>> > System.out.println("day_id =" + result.getString(1)
>> > );//the
>> > }
>> >
>> > statement.execute("set remove.scratch.dir=ture");
>> > statement.close();
>> >
>> >
>> > I run the command in CLI, there is output content, the content like
>> > below
>> > :
>> > drwxr-x--- - mianfeng cug-tbprofile 0 2010-10-25 20:55
>> > /group/tbprofile/mianfeng/scratch/test/001
>> >
>> >
>> >
>>
>> dfs and ! statements are handled specially in the CLI. They are not
>> run through the Query Processor and do not produce a result set.
>
>
You could suggest possible extensions to hive language to deal with
these type of things.
>From what you are sounds like the existence of a directory causes an
exception to be thrown. You could trap the exeption using try catch.
One other idea comes to mind. Hive has a duel "virtual table" that
allows you do do things like
select 1+2 from duel. You could design a UDF that tests for existence of a file.
select directoryExists("/user/bla") from duel, then use this in your workflow.
Interesting idea. :) If you do not make a directoryExists udf, i might :)
Re: I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/"
command, but ResultSet don't return value
Posted by lei liu <li...@gmail.com>.
Hi Edward Capriolo,
Thank you for your replay.
I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command , if
there is the directory in HDFS, the JDBC don't throw Exception, if there is
not the directory in HDFS, the JDBC will throw Exception. Could you tell I
can use the way to judge whether the directory is in HDFS?
Thanks.
2010/10/25 Edward Capriolo <ed...@gmail.com>
> On Mon, Oct 25, 2010 at 9:04 AM, lei liu <li...@gmail.com> wrote:
> > I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command, but
> > ResultSet don't return value, the code like below:
> >
> > Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver");
> > hiveConnection =
> > DriverManager.getConnection("jdbc:hive://10.232.11.53:10000/default" ,
> > "root", "root");
> > statement = hiveConnection.createStatement();
> > String sql = "dfs -mkdir
> > hdfs://hdpnn:9000//group/tbprofile/mianfeng/scratch/test/001/ ";
> >
> > ResultSet result = statement.executeQuery(sql); //the
> ResultSet
> > don't return any value.
> > while(result.next()) {
> >
> > System.out.println("day_id =" + result.getString(1)
> );//the
> > }
> >
> > statement.execute("set remove.scratch.dir=ture");
> > statement.close();
> >
> >
> > I run the command in CLI, there is output content, the content like
> below
> > :
> > drwxr-x--- - mianfeng cug-tbprofile 0 2010-10-25 20:55
> > /group/tbprofile/mianfeng/scratch/test/001
> >
> >
> >
>
> dfs and ! statements are handled specially in the CLI. They are not
> run through the Query Processor and do not produce a result set.
>
Re: I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/"
command, but ResultSet don't return value
Posted by Edward Capriolo <ed...@gmail.com>.
On Mon, Oct 25, 2010 at 9:04 AM, lei liu <li...@gmail.com> wrote:
> I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command, but
> ResultSet don't return value, the code like below:
>
> Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver");
> hiveConnection =
> DriverManager.getConnection("jdbc:hive://10.232.11.53:10000/default" ,
> "root", "root");
> statement = hiveConnection.createStatement();
> String sql = "dfs -mkdir
> hdfs://hdpnn:9000//group/tbprofile/mianfeng/scratch/test/001/ ";
>
> ResultSet result = statement.executeQuery(sql); //the ResultSet
> don't return any value.
> while(result.next()) {
>
> System.out.println("day_id =" + result.getString(1) );//the
> }
>
> statement.execute("set remove.scratch.dir=ture");
> statement.close();
>
>
> I run the command in CLI, there is output content, the content like below
> :
> drwxr-x--- - mianfeng cug-tbprofile 0 2010-10-25 20:55
> /group/tbprofile/mianfeng/scratch/test/001
>
>
>
dfs and ! statements are handled specially in the CLI. They are not
run through the Query Processor and do not produce a result set.