You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by "Choudhury, Iftekhar [USA]" <Ch...@bah.com> on 2016/01/26 16:02:50 UTC
Question about differences between Atlas and Falcon
Hi,
I am a Hadoop Architect for Booz Allen Hamilton and my company is looking at the data governance tools that Hortonworks provides. I have done some research on Falcon and Atlas and I have a question about these two tools. If Falcon can perform the data governance pretty well with all its policies, monitoring tools, and lineage, what was the need to introduce Atlas? What are the core differences between Atlas and Falcon? What is there in Atlas that Falcon couldn't provide?
Thank you,
Iftekhar Choudhury
[Booz Allen Hamilton]
Iftekhar Choudhury
Staff Technologist
Office 571-346-4775
www.boozallen.com<http://www.boozallen.com/>
Re: Question about differences between Atlas and Falcon
Posted by Ajay Yadav <aj...@gmail.com>.
I am not an expert on Atlas but if I were to answer it in one line then
Falcon is more focused on data, while atlas is more focus on metadata.
For example, Falcon will import a data from database, then replicate it,
process it and do retention on it. However if you would like to view that
your processed data was originated from this column from this table in the
database, then Falcon doesn't provide an easy way to do it. Atlas has other
features built on metadata, like, security and policy based on metadata
(tags etc.). Atlas may leverage falcon for some of these tasks like
enforcing policies, capturing metadata etc.
Hope it helps.
On Tue, Jan 26, 2016 at 8:32 PM, Choudhury, Iftekhar [USA] <
Choudhury_Iftekhar@bah.com> wrote:
> Hi,
>
>
>
> I am a Hadoop Architect for Booz Allen Hamilton and my company is looking
> at the data governance tools that Hortonworks provides. I have done some
> research on Falcon and Atlas and I have a question about these two tools.
> If Falcon can perform the data governance pretty well with all its
> policies, monitoring tools, and lineage, what was the need to introduce
> Atlas? What are the core differences between Atlas and Falcon? What is
> there in Atlas that Falcon couldn’t provide?
>
>
>
> Thank you,
>
> Iftekhar Choudhury
>
>
>
> [image: Booz Allen Hamilton]
>
>
>
> Iftekhar Choudhury
>
> Staff Technologist
>
> Office 571-346-4775
>
> www.boozallen.com
>
>
>
Re: Question about differences between Atlas and Falcon
Posted by Shwetha Shivalingamurthy <ss...@hortonworks.com>.
Hi,
Falcon is a data management system which makes acquisition, retention, replication and processing of recurring data easier. In doing so, for the data it manages, it also provides lineage, and monitoring.
Atlas is a pure metadata management system, and doesn't manage data. It captures metadata across all the components in the hadoop stack, not just falcon. Currently, it has integration with hive, falcon, sqoop, storm and kafka. It also has the ability to model business taxonomy, capture lineage across components, tagging, tag based policies(using ranger).
Makes sense?
Regards,
Shwetha
From: "Choudhury, Iftekhar [USA]" <Ch...@bah.com>>
Reply-To: "dev@falcon.apache.org<ma...@falcon.apache.org>" <de...@falcon.apache.org>>
Date: Tuesday, 26 January 2016 8:32 pm
To: "venkatesh@apache.org<ma...@apache.org>" <ve...@apache.org>>, "sriksun@apache.org<ma...@apache.org>" <sr...@apache.org>>, Hortonworks Communications <co...@hortonworks.com>>, "dev@falcon.incubator.apache.org<ma...@falcon.incubator.apache.org>" <de...@falcon.incubator.apache.org>>
Subject: Question about differences between Atlas and Falcon
Hi,
I am a Hadoop Architect for Booz Allen Hamilton and my company is looking at the data governance tools that Hortonworks provides. I have done some research on Falcon and Atlas and I have a question about these two tools. If Falcon can perform the data governance pretty well with all its policies, monitoring tools, and lineage, what was the need to introduce Atlas? What are the core differences between Atlas and Falcon? What is there in Atlas that Falcon couldn't provide?
Thank you,
Iftekhar Choudhury
[Booz Allen Hamilton]
Iftekhar Choudhury
Staff Technologist
Office 571-346-4775
www.boozallen.com<http://www.boozallen.com/>