You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Masayuki Takahashi <ma...@gmail.com> on 2017/03/25 05:23:24 UTC
Integrate IGFS and Impala
Hi,
I want to use Impala with IGFS. But it failed.
The reason why is that Impala depends on
org.apache.hadoop.hdfs.DistributedFileSystem.
https://github.com/cloudera/Impala/blob/2717f7378c406195eab241b34b184eee5f574f91/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L719
IgniteHadoopFileSystem does not extend it.
https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v1/IgniteHadoopFileSystem.html
https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v2/IgniteHadoopFileSystem.html
Have anyone tried it?
thanks.
--
Masayuki Takahashi
Re: Integrate IGFS and Impala
Posted by Vladimir Ozerov <vo...@gridgain.com>.
It looks like Impala expect some specific file system types. We can do
nothing with it. IGFS will never extend "org.apache.hadoop.hdfs.
DistributedFileSystem" because it is completely different implementation.
Question should be addressed to Impala developers.
On Sun, Mar 26, 2017 at 9:19 PM, Denis Magda <dm...@apache.org> wrote:
> Cross-posting to the dev list.
>
> IGFS gurus, are you aware of any existing and workable plug-n-play
> integration of Impala and IGFS? Looks this is unattainable unless Impala is
> supported directly by our in-memory file system.
>
> —
> Denis
>
> > On Mar 24, 2017, at 10:23 PM, Masayuki Takahashi <ma...@gmail.com>
> wrote:
> >
> > Hi,
> >
> > I want to use Impala with IGFS. But it failed.
> > The reason why is that Impala depends on
> > org.apache.hadoop.hdfs.DistributedFileSystem.
> >
> > https://github.com/cloudera/Impala/blob/2717f7378c406195eab241b34b184e
> ee5f574f91/fe/src/main/java/org/apache/impala/service/
> JniFrontend.java#L719
> >
> > IgniteHadoopFileSystem does not extend it.
> >
> > https://ignite.apache.org/releases/mobile/org/apache/
> ignite/hadoop/fs/v1/IgniteHadoopFileSystem.html
> > https://ignite.apache.org/releases/mobile/org/apache/
> ignite/hadoop/fs/v2/IgniteHadoopFileSystem.html
> >
> > Have anyone tried it?
> >
> > thanks.
> >
> > --
> > Masayuki Takahashi
>
>
Re: Integrate IGFS and Impala
Posted by Vladimir Ozerov <vo...@gridgain.com>.
It looks like Impala expect some specific file system types. We can do
nothing with it. IGFS will never extend "org.apache.hadoop.hdfs.
DistributedFileSystem" because it is completely different implementation.
Question should be addressed to Impala developers.
On Sun, Mar 26, 2017 at 9:19 PM, Denis Magda <dm...@apache.org> wrote:
> Cross-posting to the dev list.
>
> IGFS gurus, are you aware of any existing and workable plug-n-play
> integration of Impala and IGFS? Looks this is unattainable unless Impala is
> supported directly by our in-memory file system.
>
> —
> Denis
>
> > On Mar 24, 2017, at 10:23 PM, Masayuki Takahashi <ma...@gmail.com>
> wrote:
> >
> > Hi,
> >
> > I want to use Impala with IGFS. But it failed.
> > The reason why is that Impala depends on
> > org.apache.hadoop.hdfs.DistributedFileSystem.
> >
> > https://github.com/cloudera/Impala/blob/2717f7378c406195eab241b34b184e
> ee5f574f91/fe/src/main/java/org/apache/impala/service/
> JniFrontend.java#L719
> >
> > IgniteHadoopFileSystem does not extend it.
> >
> > https://ignite.apache.org/releases/mobile/org/apache/
> ignite/hadoop/fs/v1/IgniteHadoopFileSystem.html
> > https://ignite.apache.org/releases/mobile/org/apache/
> ignite/hadoop/fs/v2/IgniteHadoopFileSystem.html
> >
> > Have anyone tried it?
> >
> > thanks.
> >
> > --
> > Masayuki Takahashi
>
>
Re: Integrate IGFS and Impala
Posted by Denis Magda <dm...@apache.org>.
Cross-posting to the dev list.
IGFS gurus, are you aware of any existing and workable plug-n-play integration of Impala and IGFS? Looks this is unattainable unless Impala is supported directly by our in-memory file system.
—
Denis
> On Mar 24, 2017, at 10:23 PM, Masayuki Takahashi <ma...@gmail.com> wrote:
>
> Hi,
>
> I want to use Impala with IGFS. But it failed.
> The reason why is that Impala depends on
> org.apache.hadoop.hdfs.DistributedFileSystem.
>
> https://github.com/cloudera/Impala/blob/2717f7378c406195eab241b34b184eee5f574f91/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L719
>
> IgniteHadoopFileSystem does not extend it.
>
> https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v1/IgniteHadoopFileSystem.html
> https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v2/IgniteHadoopFileSystem.html
>
> Have anyone tried it?
>
> thanks.
>
> --
> Masayuki Takahashi
Re: Integrate IGFS and Impala
Posted by Denis Magda <dm...@apache.org>.
Cross-posting to the dev list.
IGFS gurus, are you aware of any existing and workable plug-n-play integration of Impala and IGFS? Looks this is unattainable unless Impala is supported directly by our in-memory file system.
—
Denis
> On Mar 24, 2017, at 10:23 PM, Masayuki Takahashi <ma...@gmail.com> wrote:
>
> Hi,
>
> I want to use Impala with IGFS. But it failed.
> The reason why is that Impala depends on
> org.apache.hadoop.hdfs.DistributedFileSystem.
>
> https://github.com/cloudera/Impala/blob/2717f7378c406195eab241b34b184eee5f574f91/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L719
>
> IgniteHadoopFileSystem does not extend it.
>
> https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v1/IgniteHadoopFileSystem.html
> https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v2/IgniteHadoopFileSystem.html
>
> Have anyone tried it?
>
> thanks.
>
> --
> Masayuki Takahashi