You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@impala.apache.org by Dinushka <di...@yahoo.com> on 2019/09/05 04:31:26 UTC

Impala and kudu without HDFS

Hi..I'm trying to only using Impala and Kudu without HDFS. But i get an error saying "Currently configured default filesystem: ProxyLocalFileSystem. fs.defaultFS (file:///) is not supported" only goes away when i install and start HDFS. can Impala and Kudu work without HDFS?

Re: Impala and kudu without HDFS

Posted by Tim Armstrong <ta...@cloudera.com>.
If you set -abort_on_config_error=false on the impala daemons that might
allow you to run with file:// as the default filesystem. We had tested it
in the mode before.

On Tue, Sep 10, 2019 at 5:35 AM Cliff Resnick <cr...@gmail.com> wrote:

> We use a large Impala/Kudu cluster for our analytic reporting. HDFS is not
> on the critical path for this, and after experimenting with a cluster
> configuration without it, we simply added a co-located HDFS cluster. It
> turns out we use HDFS for our dimension staging, and I'm guessing Impala
> uses it for things like exporting to parquet. In all, the HDFS component is
> stable, a seldom used but still essential part of the cluster. My advice?
> Don't fight it, and you will appreciate it later.
>
> Cliff
>
> On Tue, Sep 10, 2019, 12:42 AM Dinushka <di...@yahoo.com> wrote:
>
>> Hi,
>> I'm using in a clustered environment.
>> In my testing, I found that Impala only need HDFS on startup.
>> what kinds of dependency does Impala have on HDFS?
>> Thanks
>>
>> On Mon, Sep 9, 2019 at 9:20 PM, Jeszy
>> <je...@gmail.com> wrote:
>> Hey,
>>
>> Impala right now needs HDFS out of the box. It would probably be a lot of
>> work to remove that dependency.
>> Kudu doesn't have an HDFS dependency, you could use Spark standalone or
>> Kudu's API to query it.
>>
>> HTH
>>
>> On 2019. Sep 5., Thu at 6:31, Dinushka <di...@yahoo.com> wrote:
>>
>> Hi..
>> I'm trying to only using Impala and Kudu without HDFS. But i get an error
>> saying "Currently configured default filesystem: ProxyLocalFileSystem.
>> fs.defaultFS (file:///) is not supported" only goes away when i install and
>> start HDFS. can Impala and Kudu work without HDFS?
>>
>>

Re: Impala and kudu without HDFS

Posted by Cliff Resnick <cr...@gmail.com>.
We use a large Impala/Kudu cluster for our analytic reporting. HDFS is not
on the critical path for this, and after experimenting with a cluster
configuration without it, we simply added a co-located HDFS cluster. It
turns out we use HDFS for our dimension staging, and I'm guessing Impala
uses it for things like exporting to parquet. In all, the HDFS component is
stable, a seldom used but still essential part of the cluster. My advice?
Don't fight it, and you will appreciate it later.

Cliff

On Tue, Sep 10, 2019, 12:42 AM Dinushka <di...@yahoo.com> wrote:

> Hi,
> I'm using in a clustered environment.
> In my testing, I found that Impala only need HDFS on startup.
> what kinds of dependency does Impala have on HDFS?
> Thanks
>
> On Mon, Sep 9, 2019 at 9:20 PM, Jeszy
> <je...@gmail.com> wrote:
> Hey,
>
> Impala right now needs HDFS out of the box. It would probably be a lot of
> work to remove that dependency.
> Kudu doesn't have an HDFS dependency, you could use Spark standalone or
> Kudu's API to query it.
>
> HTH
>
> On 2019. Sep 5., Thu at 6:31, Dinushka <di...@yahoo.com> wrote:
>
> Hi..
> I'm trying to only using Impala and Kudu without HDFS. But i get an error
> saying "Currently configured default filesystem: ProxyLocalFileSystem.
> fs.defaultFS (file:///) is not supported" only goes away when i install and
> start HDFS. can Impala and Kudu work without HDFS?
>
>

Re: Impala and kudu without HDFS

Posted by Dinushka <di...@yahoo.com>.
Hi, I'm using in a clustered environment.
In my testing, I found that Impala only need HDFS on startup.what kinds of dependency does Impala have on HDFS?Thanks 
 
  On Mon, Sep 9, 2019 at 9:20 PM, Jeszy<je...@gmail.com> wrote:   Hey,
Impala right now needs HDFS out of the box. It would probably be a lot of work to remove that dependency.Kudu doesn't have an HDFS dependency, you could use Spark standalone or Kudu's API to query it.
HTH
On 2019. Sep 5., Thu at 6:31, Dinushka <di...@yahoo.com> wrote:

Hi..I'm trying to only using Impala and Kudu without HDFS. But i get an error saying "Currently configured default filesystem: ProxyLocalFileSystem. fs.defaultFS (file:///) is not supported" only goes away when i install and start HDFS. can Impala and Kudu work without HDFS?
  

Re: Impala and kudu without HDFS

Posted by Jeszy <je...@gmail.com>.
Hey,

Impala right now needs HDFS out of the box. It would probably be a lot of
work to remove that dependency.
Kudu doesn't have an HDFS dependency, you could use Spark standalone or
Kudu's API to query it.

HTH

On 2019. Sep 5., Thu at 6:31, Dinushka <di...@yahoo.com> wrote:

> Hi..
> I'm trying to only using Impala and Kudu without HDFS. But i get an error
> saying "Currently configured default filesystem: ProxyLocalFileSystem.
> fs.defaultFS (file:///) is not supported" only goes away when i install and
> start HDFS. can Impala and Kudu work without HDFS?