You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Jason Altekruse <al...@gmail.com> on 2013/11/06 20:08:32 UTC
facebook presto SQL query engine
Facebook has been working internally on a Java based distributed SQL engine
for the past year, today they open sourced it. Here is the official
announcement:
https://www.facebook.com/notes/facebook-engineering/presto-interacting-with-petabytes-of-data-at-facebook/10151786197628920
-Jason
Re: facebook presto SQL query engine
Posted by Dhruv <yo...@gmail.com>.
Can anybody with Mac export
https://github.com/facebook/presto/blob/master/presto-docs/src/main/resources/design.graffle
to SVG/PNG ?
I tried https://code.google.com/p/graffle2svg/ but got poor output :-(
nothing readable.
-Dhruv
On 11/07/2013 02:43 AM, Timothy Chen wrote:
> Yes as for internals they don't really say much, but they say a lot more in
> their blog post.
>
> Looks like they have their own ByteCode generation package, I don't believe
> they have a columnar format or vectorization.
>
> Tim
>
>
> On Wed, Nov 6, 2013 at 12:52 PM, Jason Altekruse
> <al...@gmail.com>wrote:
>
>> I thought the docs were very sparse, especially from a development
>> perspective. I was wondering if they were using vectorization or doing
>> record materialization at read time like Impala. Can't imagine they would
>> get any performance out of it if they didn't do vectorization, as they
>> already have to live within the constraints of java. Would also be
>> interesting to see if they used code generation or the netty/unsafe APIs at
>> all. Will be taking a closer look in the next few weeks to see if there are
>> some good ideas we can incorporate into Drill.
>>
>> -Jason
>>
>>
>> On Wed, Nov 6, 2013 at 1:24 PM, Timothy Chen <tn...@gmail.com> wrote:
>>
>>> I've been waiting for this since their data conference earlier this year
>> :)
>>> The concepts and architecture diagram also resembles like Drill, and also
>>> is developed in Java.
>>>
>>> I do like their docs that lists the functions available and high level
>>> diagrams:
>>>
>>> http://prestodb.io/docs/current/
>>>
>>> Tim
>>>
>>>
>>>
>>>
>>> On Wed, Nov 6, 2013 at 11:08 AM, Jason Altekruse
>>> <al...@gmail.com>wrote:
>>>
>>>> Facebook has been working internally on a Java based distributed SQL
>>> engine
>>>> for the past year, today they open sourced it. Here is the official
>>>> announcement:
>>>>
>>>>
>>>>
>> https://www.facebook.com/notes/facebook-engineering/presto-interacting-with-petabytes-of-data-at-facebook/10151786197628920
>>>> -Jason
>>>>
Re: facebook presto SQL query engine
Posted by Jacques Nadeau <ja...@apache.org>.
I hope to look at it soon. If I'm able to, I'll post back anything I learn.
Jacques
On Wed, Nov 6, 2013 at 1:13 PM, Timothy Chen <tn...@gmail.com> wrote:
> Yes as for internals they don't really say much, but they say a lot more in
> their blog post.
>
> Looks like they have their own ByteCode generation package, I don't believe
> they have a columnar format or vectorization.
>
> Tim
>
>
> On Wed, Nov 6, 2013 at 12:52 PM, Jason Altekruse
> <al...@gmail.com>wrote:
>
> > I thought the docs were very sparse, especially from a development
> > perspective. I was wondering if they were using vectorization or doing
> > record materialization at read time like Impala. Can't imagine they would
> > get any performance out of it if they didn't do vectorization, as they
> > already have to live within the constraints of java. Would also be
> > interesting to see if they used code generation or the netty/unsafe APIs
> at
> > all. Will be taking a closer look in the next few weeks to see if there
> are
> > some good ideas we can incorporate into Drill.
> >
> > -Jason
> >
> >
> > On Wed, Nov 6, 2013 at 1:24 PM, Timothy Chen <tn...@gmail.com> wrote:
> >
> > > I've been waiting for this since their data conference earlier this
> year
> > :)
> > >
> > > The concepts and architecture diagram also resembles like Drill, and
> also
> > > is developed in Java.
> > >
> > > I do like their docs that lists the functions available and high level
> > > diagrams:
> > >
> > > http://prestodb.io/docs/current/
> > >
> > > Tim
> > >
> > >
> > >
> > >
> > > On Wed, Nov 6, 2013 at 11:08 AM, Jason Altekruse
> > > <al...@gmail.com>wrote:
> > >
> > > > Facebook has been working internally on a Java based distributed SQL
> > > engine
> > > > for the past year, today they open sourced it. Here is the official
> > > > announcement:
> > > >
> > > >
> > > >
> > >
> >
> https://www.facebook.com/notes/facebook-engineering/presto-interacting-with-petabytes-of-data-at-facebook/10151786197628920
> > > >
> > > > -Jason
> > > >
> > >
> >
>
Re: facebook presto SQL query engine
Posted by Timothy Chen <tn...@gmail.com>.
Yes as for internals they don't really say much, but they say a lot more in
their blog post.
Looks like they have their own ByteCode generation package, I don't believe
they have a columnar format or vectorization.
Tim
On Wed, Nov 6, 2013 at 12:52 PM, Jason Altekruse
<al...@gmail.com>wrote:
> I thought the docs were very sparse, especially from a development
> perspective. I was wondering if they were using vectorization or doing
> record materialization at read time like Impala. Can't imagine they would
> get any performance out of it if they didn't do vectorization, as they
> already have to live within the constraints of java. Would also be
> interesting to see if they used code generation or the netty/unsafe APIs at
> all. Will be taking a closer look in the next few weeks to see if there are
> some good ideas we can incorporate into Drill.
>
> -Jason
>
>
> On Wed, Nov 6, 2013 at 1:24 PM, Timothy Chen <tn...@gmail.com> wrote:
>
> > I've been waiting for this since their data conference earlier this year
> :)
> >
> > The concepts and architecture diagram also resembles like Drill, and also
> > is developed in Java.
> >
> > I do like their docs that lists the functions available and high level
> > diagrams:
> >
> > http://prestodb.io/docs/current/
> >
> > Tim
> >
> >
> >
> >
> > On Wed, Nov 6, 2013 at 11:08 AM, Jason Altekruse
> > <al...@gmail.com>wrote:
> >
> > > Facebook has been working internally on a Java based distributed SQL
> > engine
> > > for the past year, today they open sourced it. Here is the official
> > > announcement:
> > >
> > >
> > >
> >
> https://www.facebook.com/notes/facebook-engineering/presto-interacting-with-petabytes-of-data-at-facebook/10151786197628920
> > >
> > > -Jason
> > >
> >
>
Re: facebook presto SQL query engine
Posted by Jason Altekruse <al...@gmail.com>.
I thought the docs were very sparse, especially from a development
perspective. I was wondering if they were using vectorization or doing
record materialization at read time like Impala. Can't imagine they would
get any performance out of it if they didn't do vectorization, as they
already have to live within the constraints of java. Would also be
interesting to see if they used code generation or the netty/unsafe APIs at
all. Will be taking a closer look in the next few weeks to see if there are
some good ideas we can incorporate into Drill.
-Jason
On Wed, Nov 6, 2013 at 1:24 PM, Timothy Chen <tn...@gmail.com> wrote:
> I've been waiting for this since their data conference earlier this year :)
>
> The concepts and architecture diagram also resembles like Drill, and also
> is developed in Java.
>
> I do like their docs that lists the functions available and high level
> diagrams:
>
> http://prestodb.io/docs/current/
>
> Tim
>
>
>
>
> On Wed, Nov 6, 2013 at 11:08 AM, Jason Altekruse
> <al...@gmail.com>wrote:
>
> > Facebook has been working internally on a Java based distributed SQL
> engine
> > for the past year, today they open sourced it. Here is the official
> > announcement:
> >
> >
> >
> https://www.facebook.com/notes/facebook-engineering/presto-interacting-with-petabytes-of-data-at-facebook/10151786197628920
> >
> > -Jason
> >
>
Re: facebook presto SQL query engine
Posted by Timothy Chen <tn...@gmail.com>.
I've been waiting for this since their data conference earlier this year :)
The concepts and architecture diagram also resembles like Drill, and also
is developed in Java.
I do like their docs that lists the functions available and high level
diagrams:
http://prestodb.io/docs/current/
Tim
On Wed, Nov 6, 2013 at 11:08 AM, Jason Altekruse
<al...@gmail.com>wrote:
> Facebook has been working internally on a Java based distributed SQL engine
> for the past year, today they open sourced it. Here is the official
> announcement:
>
>
> https://www.facebook.com/notes/facebook-engineering/presto-interacting-with-petabytes-of-data-at-facebook/10151786197628920
>
> -Jason
>