You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Hans Uhlig <hu...@uhlisys.com> on 2012/02/27 18:17:18 UTC
Adding Hive Partitions Programatically.
How can you add a partition to a table programmatically? I have a mapreduce=
job that needs to perform a "ALTER TABLE x ADD PARTITION IF NOT EXIST (foo=
=3D bar) LOCATION '<Mapreduce output>' after it completes. Is there an exa=
mple of this anwhere?
Re: Adding Hive Partitions Programatically.
Posted by Aniket Mokashi <an...@gmail.com>.
If you have hive-server running somewhere you can do following-
HiveConf hiveConf = new HiveConf(MyClass.class);
hiveConf.set("hive.metastore.local", "false");
hiveConf.set(ConfVars.METASTOREURIS.varname, url);
HiveMetaStoreClient client = new HiveMetaStoreClient(hiveConf);
and then do-
client.add_partition(part);
Thanks,
Aniket
On Mon, Feb 27, 2012 at 9:58 AM, Edward Capriolo <ed...@gmail.com>wrote:
> You can use the metastore API directly if you source all the correct
> hadoop and hive xml files, have all of hives/lib in your classpath and
> network access. It is much more graceful to correct through the
> thrift service.
>
>
> On Mon, Feb 27, 2012 at 12:53 PM, Hans Uhlig <hu...@uhlisys.com> wrote:
> > I am using external tables already. Is the Metastore API not a usable API
> > from inside another Java Application?
> >
> > On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <ed...@gmail.com>
> > wrote:
> >>
> >> The most programmatic way is to use external tables and then use
> >> HiveService (thrift) to add the partition after your MR job has ran
> >> successfully.
> >>
> >>
> >> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <hu...@uhlisys.com>
> wrote:
> >> > How can you add a partition to a table programmatically? I have a
> >> > mapreduce= job that needs to perform a "ALTER TABLE x ADD PARTITION
> IF
> >> > NOT
> >> > EXIST (foo= =3D bar) LOCATION '<Mapreduce output>' after it
> completes.
> >> > Is
> >> > there an exa= mple of this anwhere?
> >
> >
>
--
"...:::Aniket:::... Quetzalco@tl"
Re: Adding Hive Partitions Programatically.
Posted by Roberto Congiu <ro...@openx.com>.
Also consider that if you're using the metastore API, you'll most likely
need to recompile whenever you upgrade hive, while you most likely won't
need to do that if you use thrift.
R.
On Mon, Feb 27, 2012 at 9:58 AM, Edward Capriolo <ed...@gmail.com>wrote:
> You can use the metastore API directly if you source all the correct
> hadoop and hive xml files, have all of hives/lib in your classpath and
> network access. It is much more graceful to correct through the
> thrift service.
>
>
> On Mon, Feb 27, 2012 at 12:53 PM, Hans Uhlig <hu...@uhlisys.com> wrote:
> > I am using external tables already. Is the Metastore API not a usable API
> > from inside another Java Application?
> >
> > On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <ed...@gmail.com>
> > wrote:
> >>
> >> The most programmatic way is to use external tables and then use
> >> HiveService (thrift) to add the partition after your MR job has ran
> >> successfully.
> >>
> >>
> >> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <hu...@uhlisys.com>
> wrote:
> >> > How can you add a partition to a table programmatically? I have a
> >> > mapreduce= job that needs to perform a "ALTER TABLE x ADD PARTITION
> IF
> >> > NOT
> >> > EXIST (foo= =3D bar) LOCATION '<Mapreduce output>' after it
> completes.
> >> > Is
> >> > there an exa= mple of this anwhere?
> >
> >
>
Re: Adding Hive Partitions Programatically.
Posted by Edward Capriolo <ed...@gmail.com>.
You can use the metastore API directly if you source all the correct
hadoop and hive xml files, have all of hives/lib in your classpath and
network access. It is much more graceful to correct through the
thrift service.
On Mon, Feb 27, 2012 at 12:53 PM, Hans Uhlig <hu...@uhlisys.com> wrote:
> I am using external tables already. Is the Metastore API not a usable API
> from inside another Java Application?
>
> On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <ed...@gmail.com>
> wrote:
>>
>> The most programmatic way is to use external tables and then use
>> HiveService (thrift) to add the partition after your MR job has ran
>> successfully.
>>
>>
>> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <hu...@uhlisys.com> wrote:
>> > How can you add a partition to a table programmatically? I have a
>> > mapreduce= job that needs to perform a "ALTER TABLE x ADD PARTITION IF
>> > NOT
>> > EXIST (foo= =3D bar) LOCATION '<Mapreduce output>' after it completes.
>> > Is
>> > there an exa= mple of this anwhere?
>
>
Re: Adding Hive Partitions Programatically.
Posted by Hans Uhlig <hu...@uhlisys.com>.
I am using external tables already. Is the Metastore API not a usable API
from inside another Java Application?
On Mon, Feb 27, 2012 at 9:19 AM, Edward Capriolo <ed...@gmail.com>wrote:
> The most programmatic way is to use external tables and then use
> HiveService (thrift) to add the partition after your MR job has ran
> successfully.
>
>
> On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <hu...@uhlisys.com> wrote:
> > How can you add a partition to a table programmatically? I have a
> > mapreduce= job that needs to perform a "ALTER TABLE x ADD PARTITION IF
> NOT
> > EXIST (foo= =3D bar) LOCATION '<Mapreduce output>' after it completes.
> Is
> > there an exa= mple of this anwhere?
>
Re: Adding Hive Partitions Programatically.
Posted by Edward Capriolo <ed...@gmail.com>.
The most programmatic way is to use external tables and then use
HiveService (thrift) to add the partition after your MR job has ran
successfully.
On Mon, Feb 27, 2012 at 12:17 PM, Hans Uhlig <hu...@uhlisys.com> wrote:
> How can you add a partition to a table programmatically? I have a
> mapreduce= job that needs to perform a "ALTER TABLE x ADD PARTITION IF NOT
> EXIST (foo= =3D bar) LOCATION '<Mapreduce output>' after it completes. Is
> there an exa= mple of this anwhere?