Enhancing Ambari to support other file systems in addition to HDFS

[Jeff Vance <jv...@redhat.com>]

A team at Red Hat is looking to expand the filesystem choices supported in Ambari to include other Hadoop Compatible File Systems (HCFS), including Red Hat's filesystem based on GlusterFS.

We'd like to start a discussion on how to architect this capability in a way that fits in with existing Ambari service configurations, is expandable, supports differences among the various Hadoop compatible file systems, and is compatible with existing Ambari deployments.

There could be different levels of Hadoop Compatible File System "support". For example, a bare-bones level could be that Ambari does not require HDFS to be running and does not interfere with the HCFS service. The next level could be that Ambari provides some monitoring and statistics for the HCFS. A higher level could be that Ambari can deploy the HCFS, etc.

We have opened JIRA AMBARI-1817 as an umbrella JIRA for HCFS support in Ambari. We welcome community input.

Thanks,
Red Hat Big Data Team

Re: Enhancing Ambari to support other file systems in addition to HDFS

[Mahadev Konar <ma...@hortonworks.com>]

Jeff,
 Great to see this effort going. As you stated, the first part would
be to start with bare bones level (no HDFS). Waiting to see more on
this.

thanks
mahadev

On Tue, Apr 9, 2013 at 4:23 PM, Jeff Vance <jv...@redhat.com> wrote:
> A team at Red Hat is looking to expand the filesystem choices supported in Ambari to include other Hadoop Compatible File Systems (HCFS), including Red Hat's filesystem based on GlusterFS.
>
> We'd like to start a discussion on how to architect this capability in a way that fits in with existing Ambari service configurations, is expandable, supports differences among the various Hadoop compatible file systems, and is compatible with existing Ambari deployments.
>
> There could be different levels of Hadoop Compatible File System "support". For example, a bare-bones level could be that Ambari does not require HDFS to be running and does not interfere with the HCFS service. The next level could be that Ambari provides some monitoring and statistics for the HCFS. A higher level could be that Ambari can deploy the HCFS, etc.
>
> We have opened JIRA AMBARI-1817 as an umbrella JIRA for HCFS support in Ambari. We welcome community input.
>
> Thanks,
> Red Hat Big Data Team