You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Jim Bates <jb...@maprtech.com> on 2015/04/15 16:16:44 UTC
Accessing multiple hadoop clusters
I have sued drill to access local files and DFS files in the same query.
Based on that I think the answer is yes but....
Can I use drill to join data across multiple Hadoop clusters? If so, I'm
assuming I would have to have all the drill bits register to the same
zookeeper group so I can find them all.
Anyone done anything like this?
Re: Accessing multiple hadoop clusters
Posted by Jim Bates <jb...@maprtech.com>.
LOL... Thats my best skill... Typos that end up being other words.
On Wed, Apr 15, 2015 at 10:48 AM, mufy <mu...@gmail.com> wrote:
> Aw c'mon Jim! You shouldn't have sued Drill for that :-).
>
> *I have sued drill to access...*
>
> More worthy answers will follow, I'm sure ;-).
>
>
> ---
> Mufeed Usman
> My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My
> Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal
> <http://mufeed.livejournal.com>
>
>
>
>
> On Wed, Apr 15, 2015 at 7:46 PM, Jim Bates <jb...@maprtech.com> wrote:
>
> > I have sued drill to access local files and DFS files in the same query.
> > Based on that I think the answer is yes but....
> >
> > Can I use drill to join data across multiple Hadoop clusters? If so, I'm
> > assuming I would have to have all the drill bits register to the same
> > zookeeper group so I can find them all.
> >
> > Anyone done anything like this?
> >
>
Re: Accessing multiple hadoop clusters
Posted by mufy <mu...@gmail.com>.
Aw c'mon Jim! You shouldn't have sued Drill for that :-).
*I have sued drill to access...*
More worthy answers will follow, I'm sure ;-).
---
Mufeed Usman
My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My
Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal
<http://mufeed.livejournal.com>
On Wed, Apr 15, 2015 at 7:46 PM, Jim Bates <jb...@maprtech.com> wrote:
> I have sued drill to access local files and DFS files in the same query.
> Based on that I think the answer is yes but....
>
> Can I use drill to join data across multiple Hadoop clusters? If so, I'm
> assuming I would have to have all the drill bits register to the same
> zookeeper group so I can find them all.
>
> Anyone done anything like this?
>
Re: Accessing multiple hadoop clusters
Posted by Ted Dunning <te...@gmail.com>.
Jacques,
It sounds like you are implying that the Drill cluster could span both MapR
clusters. I think that is true.
But I also think that most practical situations, as well as what Jim was
asking about, is a case which will have all the drillbits in question next
to one cluster and accessing a remote cluster.
That should also work.
On Wed, Apr 15, 2015 at 6:01 PM, Jacques Nadeau <ja...@apache.org> wrote:
> Should work fine as long as you sign a EULA saying you won't sue.
>
> Note you may want to bump up the `planner.affinity_factor` to something
> like 5000.0 to make sure that you get maximum locality for reads for each
> cluster.
>
> On Wed, Apr 15, 2015 at 7:16 AM, Jim Bates <jb...@maprtech.com> wrote:
>
> > I have sued drill to access local files and DFS files in the same query.
> > Based on that I think the answer is yes but....
> >
> > Can I use drill to join data across multiple Hadoop clusters? If so, I'm
> > assuming I would have to have all the drill bits register to the same
> > zookeeper group so I can find them all.
> >
> > Anyone done anything like this?
> >
>
Re: Accessing multiple hadoop clusters
Posted by Jacques Nadeau <ja...@apache.org>.
Should work fine as long as you sign a EULA saying you won't sue.
Note you may want to bump up the `planner.affinity_factor` to something
like 5000.0 to make sure that you get maximum locality for reads for each
cluster.
On Wed, Apr 15, 2015 at 7:16 AM, Jim Bates <jb...@maprtech.com> wrote:
> I have sued drill to access local files and DFS files in the same query.
> Based on that I think the answer is yes but....
>
> Can I use drill to join data across multiple Hadoop clusters? If so, I'm
> assuming I would have to have all the drill bits register to the same
> zookeeper group so I can find them all.
>
> Anyone done anything like this?
>