You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by panfei <cn...@gmail.com> on 2015/11/09 11:06:06 UTC
StoreInKiteDataset cannot recognize hive dataset
Hi all:
I use StoreInKiteDataset processor to try to store data in hive by
configuring the target URI to:
*dataset:hive:default/sandwiches*
but the processor reports that* the URI is invalid*. but after replacing
the URI to
*dataset:file:/tmp/sandwiches*
everything works OK.
Is there any way to resolve the Hive dataset issue ? or it is not
supported at all ?
Thank you very much
--
不学习，不知道
Re: StoreInKiteDataset cannot recognize hive dataset
Posted by panfei <cn...@gmail.com>.
Hi Alan:
Thanks for the tips. I found a file named nifi-kite-nar-0.3.0.nar under
the NIFI_HOME/lib directory, there are some other nars in the directory,
nifi-kafka-nar-0.3.0.nar, and PutKafka processor passed my test, so I
think it's not a classpath issue. it just show the following error:
'Target dataset URI' validated against 'dataset:hive:default/sandwiches' is
invalid because Dataset URI is invalid: Unkown dataset URI:
hive:default/sandwiches.
2015-11-10 22:33 GMT+08:00 Alan Jackoway <al...@cloudera.com>:
> Was your error like this:
> Unknown dataset URI: hive:datasetname. Check that JARs for hive datasets
> are on the classpath.; rolling back session:
> org.kitesdk.data.DatasetNotFoundException: Unknown dataset
> URI: hive:datasetname. Check that JARs for hive datasets are on the
> classpath.
>
> I am able to reproduce that issue and talking with some kite people about
> a fix. It seems like we have to include some hive dependencies with the
> kite nar.
>
> On Tue, Nov 10, 2015 at 3:49 AM, panfei <cn...@gmail.com> wrote:
>
>> Hi Bryan:
>>
>> thanks for your reply. I already created the dataset in hive before
>> using StoreInKiteDataset processor.
>>
>> and I will check the code you supply to see if there is any clue to solve
>> the issue.
>>
>> thanks again
>>
>> 2015-11-09 22:19 GMT+08:00 Bryan Bende <bb...@gmail.com>:
>>
>>> Hello,
>>>
>>> I'm not that familiar with Kite, but is it possible that you need to
>>> create the Kite dataset using the Kite CLI before StoreInKiteDataset tries
>>> to write data to it?
>>>
>>> It looks like that is how the test cases for this processor work:
>>>
>>> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/test/java/org/apache/nifi/processors/kite/TestKiteProcessorsCluster.java#L85
>>>
>>> It uses "dataset:hive:ns/test", but calls Datasets.create(...) before
>>> running the processor.
>>>
>>> -Bryan
>>>
>>>
>>> On Mon, Nov 9, 2015 at 5:06 AM, panfei <cn...@gmail.com> wrote:
>>>
>>>> Hi all:
>>>> I use StoreInKiteDataset processor to try to store data in hive by
>>>> configuring the target URI to:
>>>>
>>>> *dataset:hive:default/sandwiches*
>>>>
>>>> but the processor reports that* the URI is invalid*. but after
>>>> replacing the URI to
>>>>
>>>> *dataset:file:/tmp/sandwiches*
>>>>
>>>> everything works OK.
>>>>
>>>>
>>>> Is there any way to resolve the Hive dataset issue ? or it is not
>>>> supported at all ?
>>>>
>>>>
>>>> Thank you very much
>>>> --
>>>> 不学习，不知道
>>>>
>>>
>>>
>>
>>
>> --
>> 不学习，不知道
>>
>
>
--
不学习，不知道
Re: StoreInKiteDataset cannot recognize hive dataset
Posted by Alan Jackoway <al...@cloudera.com>.
Was your error like this:
Unknown dataset URI: hive:datasetname. Check that JARs for hive datasets
are on the classpath.; rolling back session:
org.kitesdk.data.DatasetNotFoundException: Unknown dataset
URI: hive:datasetname. Check that JARs for hive datasets are on the
classpath.
I am able to reproduce that issue and talking with some kite people about a
fix. It seems like we have to include some hive dependencies with the kite
nar.
On Tue, Nov 10, 2015 at 3:49 AM, panfei <cn...@gmail.com> wrote:
> Hi Bryan:
>
> thanks for your reply. I already created the dataset in hive before using StoreInKiteDataset
> processor.
>
> and I will check the code you supply to see if there is any clue to solve
> the issue.
>
> thanks again
>
> 2015-11-09 22:19 GMT+08:00 Bryan Bende <bb...@gmail.com>:
>
>> Hello,
>>
>> I'm not that familiar with Kite, but is it possible that you need to
>> create the Kite dataset using the Kite CLI before StoreInKiteDataset tries
>> to write data to it?
>>
>> It looks like that is how the test cases for this processor work:
>>
>> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/test/java/org/apache/nifi/processors/kite/TestKiteProcessorsCluster.java#L85
>>
>> It uses "dataset:hive:ns/test", but calls Datasets.create(...) before
>> running the processor.
>>
>> -Bryan
>>
>>
>> On Mon, Nov 9, 2015 at 5:06 AM, panfei <cn...@gmail.com> wrote:
>>
>>> Hi all:
>>> I use StoreInKiteDataset processor to try to store data in hive by
>>> configuring the target URI to:
>>>
>>> *dataset:hive:default/sandwiches*
>>>
>>> but the processor reports that* the URI is invalid*. but after
>>> replacing the URI to
>>>
>>> *dataset:file:/tmp/sandwiches*
>>>
>>> everything works OK.
>>>
>>>
>>> Is there any way to resolve the Hive dataset issue ? or it is not
>>> supported at all ?
>>>
>>>
>>> Thank you very much
>>> --
>>> 不学习，不知道
>>>
>>
>>
>
>
> --
> 不学习，不知道
>
Re: StoreInKiteDataset cannot recognize hive dataset
Posted by panfei <cn...@gmail.com>.
Hi Bryan:
thanks for your reply. I already created the dataset in hive before
using StoreInKiteDataset
processor.
and I will check the code you supply to see if there is any clue to solve
the issue.
thanks again
2015-11-09 22:19 GMT+08:00 Bryan Bende <bb...@gmail.com>:
> Hello,
>
> I'm not that familiar with Kite, but is it possible that you need to
> create the Kite dataset using the Kite CLI before StoreInKiteDataset tries
> to write data to it?
>
> It looks like that is how the test cases for this processor work:
>
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/test/java/org/apache/nifi/processors/kite/TestKiteProcessorsCluster.java#L85
>
> It uses "dataset:hive:ns/test", but calls Datasets.create(...) before
> running the processor.
>
> -Bryan
>
>
> On Mon, Nov 9, 2015 at 5:06 AM, panfei <cn...@gmail.com> wrote:
>
>> Hi all:
>> I use StoreInKiteDataset processor to try to store data in hive by
>> configuring the target URI to:
>>
>> *dataset:hive:default/sandwiches*
>>
>> but the processor reports that* the URI is invalid*. but after replacing
>> the URI to
>>
>> *dataset:file:/tmp/sandwiches*
>>
>> everything works OK.
>>
>>
>> Is there any way to resolve the Hive dataset issue ? or it is not
>> supported at all ?
>>
>>
>> Thank you very much
>> --
>> 不学习，不知道
>>
>
>
--
不学习，不知道
Re: StoreInKiteDataset cannot recognize hive dataset
Posted by Bryan Bende <bb...@gmail.com>.
Hello,
I'm not that familiar with Kite, but is it possible that you need to create
the Kite dataset using the Kite CLI before StoreInKiteDataset tries to
write data to it?
It looks like that is how the test cases for this processor work:
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/test/java/org/apache/nifi/processors/kite/TestKiteProcessorsCluster.java#L85
It uses "dataset:hive:ns/test", but calls Datasets.create(...) before
running the processor.
-Bryan
On Mon, Nov 9, 2015 at 5:06 AM, panfei <cn...@gmail.com> wrote:
> Hi all:
> I use StoreInKiteDataset processor to try to store data in hive by
> configuring the target URI to:
>
> *dataset:hive:default/sandwiches*
>
> but the processor reports that* the URI is invalid*. but after replacing
> the URI to
>
> *dataset:file:/tmp/sandwiches*
>
> everything works OK.
>
>
> Is there any way to resolve the Hive dataset issue ? or it is not
> supported at all ?
>
>
> Thank you very much
> --
> 不学习，不知道
>