You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Christine Mathiesen <t-...@hotels.com.INVALID> on 2019/11/06 15:08:51 UTC
Iceberg and Hive
Hey again!
A quick question on Iceberg and Hive - is there any work going on around integrating the two together so that you can read and write Iceberg tables from Hive? Or for converting Hive tables to Iceberg?
Thanks!
Christine Mathiesen
Software Development Intern
BDP – Hotels.com
Expedia Group
Re: Iceberg and Hive
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
Hi Christine,
There has been some interest in building Iceberg support in Hive. We did
some preliminary work to look into it and it looks like you'd need to
implement HiveStorageHandler, InputFormat, OutputFormat, and the metastore
interface, RawStore.
Adrien Guillo from AirBnB was looking into this, but I haven't heard how
their effort is going in a while. If you're interested, I'm sure that
contributing here would be appreciated!
On Wed, Nov 6, 2019 at 8:01 AM Xinli shang <sh...@uber.com.invalid> wrote:
> For your second question "converting Hive tables to Iceberg", There is an import
> utility in Iceberg
> <https://github.com/apache/incubator-iceberg/blob/master/spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala#L369-L372>
> in Spark to do that. I wrote several lines of Spark code and tested it and
> it worked.
>
> On Wed, Nov 6, 2019 at 7:09 AM Christine Mathiesen
> <t-...@hotels.com.invalid> wrote:
>
>> Hey again!
>>
>>
>>
>> A quick question on Iceberg and Hive - is there any work going on around
>> integrating the two together so that you can read and write Iceberg tables
>> from Hive? Or for converting Hive tables to Iceberg?
>>
>>
>>
>> Thanks!
>>
>>
>>
>> *Christine Mathiesen *
>>
>> Software Development Intern
>>
>> BDP – Hotels.com
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__Hotels.com&d=DwQGaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=FQ88AmOZ4TMjDdqNBGu-ag&m=cPtdGlNb0486Rw99msUiNbVwMc2h1D5PMcNzhveoSCE&s=0xzg8r79YnXz4yhplD0gB4bphMuR1B0ZsIG_95G6lRg&e=>
>>
>> Expedia Group
>>
>>
>>
>
>
> --
> Xinli Shang
>
--
Ryan Blue
Software Engineer
Netflix
Re: Iceberg and Hive
Posted by Xinli shang <sh...@uber.com.INVALID>.
For your second question "converting Hive tables to Iceberg", There is
an import
utility in Iceberg
<https://github.com/apache/incubator-iceberg/blob/master/spark/src/main/scala/org/apache/iceberg/spark/SparkTableUtil.scala#L369-L372>
in Spark to do that. I wrote several lines of Spark code and tested it and
it worked.
On Wed, Nov 6, 2019 at 7:09 AM Christine Mathiesen
<t-...@hotels.com.invalid> wrote:
> Hey again!
>
>
>
> A quick question on Iceberg and Hive - is there any work going on around
> integrating the two together so that you can read and write Iceberg tables
> from Hive? Or for converting Hive tables to Iceberg?
>
>
>
> Thanks!
>
>
>
> *Christine Mathiesen *
>
> Software Development Intern
>
> BDP – Hotels.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__Hotels.com&d=DwQGaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=FQ88AmOZ4TMjDdqNBGu-ag&m=cPtdGlNb0486Rw99msUiNbVwMc2h1D5PMcNzhveoSCE&s=0xzg8r79YnXz4yhplD0gB4bphMuR1B0ZsIG_95G6lRg&e=>
>
> Expedia Group
>
>
>
--
Xinli Shang