You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Andrew Lamb <al...@influxdata.com> on 2022/10/02 11:00:03 UTC
[WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
We are working on a series of blog posts[1][2][3] that we plan to publish
to the arrow blog about how structured data is represented using the two
formats.
Among other things, I hope this may help other Arrow / Parquet implementers
as well as draw attention to the fact that parquet and arrow both support
nested structures, albeit differently.
We would welcome any feedback from the community,
Andrew
[1] https://github.com/apache/arrow-site/pull/245
[2] https://github.com/apache/arrow-site/pull/246
[3] https://github.com/apache/arrow-site/pull/247
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Andrew Lamb <al...@influxdata.com>.
I have received the invite -- thank you
On Thu, Oct 20, 2022 at 4:41 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> Hi Andrew,
>
> I've sent you ( https://twitter.com/andrewlamb1111 ) an
> invitation to join the @ApacheArrow 's team members. Could
> you confirm the invitation? You can tweet as @ApacheArrow
> from https://tweetdeck.twitter.com/ after you accept the
> invitation.
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: [WEBSITE] Blog posts on representing Structured Data with Parquet
> and Arrow" on Thu, 20 Oct 2022 07:17:11 -0400,
> Andrew Lamb <al...@influxdata.com> wrote:
>
> > I think it is a great idea to tweet it out -- can someone help me do so?
> >
> > On Wed, Oct 19, 2022 at 8:26 PM David Li <li...@apache.org> wrote:
> >
> >> Andrew, what do you think about tweeting this series out through
> >> @ApacheArrow?
> >>
> >> -David
> >>
> >> On Mon, Oct 17, 2022, at 16:50, Andrew Lamb wrote:
> >> > And the final installment:
> >> >
> https://arrow.apache.org/blog/2022/10/17/arrow-parquet-encoding-part-3/
> >> >
> >> > On Sat, Oct 8, 2022 at 9:47 AM Andrew Lamb <al...@influxdata.com>
> wrote:
> >> >
> >> >> Second post is now live (this time with non localhost url
> :thumbsup:):
> >> >>
> >> >>
> https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/
> >> >>
> >> >> Andrew
> >> >>
> >> >> On Wed, Oct 5, 2022 at 4:57 PM Andrew Lamb <al...@influxdata.com>
> >> wrote:
> >> >>
> >> >>> Let's just say the posting process has been a comedy of errors by me
> >> >>> 🤦♂️. I apologize for the noise
> >> >>>
> >> >>> On Wed, Oct 5, 2022 at 4:17 PM Will Jones <wi...@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>>>
> >> https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
> >> >>>>
> >> >>>> On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <
> >> >>>> krassovskysasha@gmail.com>
> >> >>>> wrote:
> >> >>>>
> >> >>>> > Hi, we aren’t able to connect to your localhost 😀
> >> >>>> >
> >> >>>> > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <alamb@influxdata.com
> >
> >> >>>> wrote:
> >> >>>> > >
> >> >>>> > > We have published the first post:
> >> >>>> > >
> >> http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
> >> >>>> > >
> >> >>>> > > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <
> alamb@influxdata.com>
> >> >>>> wrote:
> >> >>>> > >
> >> >>>> > >> We are working on a series of blog posts[1][2][3] that we
> plan to
> >> >>>> > publish
> >> >>>> > >> to the arrow blog about how structured data is represented
> using
> >> >>>> the two
> >> >>>> > >> formats.
> >> >>>> > >>
> >> >>>> > >> Among other things, I hope this may help other Arrow / Parquet
> >> >>>> > >> implementers as well as draw attention to the fact that
> parquet
> >> and
> >> >>>> > arrow
> >> >>>> > >> both support nested structures, albeit differently.
> >> >>>> > >>
> >> >>>> > >> We would welcome any feedback from the community,
> >> >>>> > >> Andrew
> >> >>>> > >>
> >> >>>> > >> [1] https://github.com/apache/arrow-site/pull/245
> >> >>>> > >> [2] https://github.com/apache/arrow-site/pull/246
> >> >>>> > >> [3] https://github.com/apache/arrow-site/pull/247
> >> >>>> > >>
> >> >>>> >
> >> >>>> >
> >> >>>>
> >> >>>
> >>
>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi Andrew,
I've sent you ( https://twitter.com/andrewlamb1111 ) an
invitation to join the @ApacheArrow 's team members. Could
you confirm the invitation? You can tweet as @ApacheArrow
from https://tweetdeck.twitter.com/ after you accept the
invitation.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow" on Thu, 20 Oct 2022 07:17:11 -0400,
Andrew Lamb <al...@influxdata.com> wrote:
> I think it is a great idea to tweet it out -- can someone help me do so?
>
> On Wed, Oct 19, 2022 at 8:26 PM David Li <li...@apache.org> wrote:
>
>> Andrew, what do you think about tweeting this series out through
>> @ApacheArrow?
>>
>> -David
>>
>> On Mon, Oct 17, 2022, at 16:50, Andrew Lamb wrote:
>> > And the final installment:
>> > https://arrow.apache.org/blog/2022/10/17/arrow-parquet-encoding-part-3/
>> >
>> > On Sat, Oct 8, 2022 at 9:47 AM Andrew Lamb <al...@influxdata.com> wrote:
>> >
>> >> Second post is now live (this time with non localhost url :thumbsup:):
>> >>
>> >> https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/
>> >>
>> >> Andrew
>> >>
>> >> On Wed, Oct 5, 2022 at 4:57 PM Andrew Lamb <al...@influxdata.com>
>> wrote:
>> >>
>> >>> Let's just say the posting process has been a comedy of errors by me
>> >>> 🤦♂️. I apologize for the noise
>> >>>
>> >>> On Wed, Oct 5, 2022 at 4:17 PM Will Jones <wi...@gmail.com>
>> >>> wrote:
>> >>>
>> >>>>
>> https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
>> >>>>
>> >>>> On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <
>> >>>> krassovskysasha@gmail.com>
>> >>>> wrote:
>> >>>>
>> >>>> > Hi, we aren’t able to connect to your localhost 😀
>> >>>> >
>> >>>> > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com>
>> >>>> wrote:
>> >>>> > >
>> >>>> > > We have published the first post:
>> >>>> > >
>> http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
>> >>>> > >
>> >>>> > > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com>
>> >>>> wrote:
>> >>>> > >
>> >>>> > >> We are working on a series of blog posts[1][2][3] that we plan to
>> >>>> > publish
>> >>>> > >> to the arrow blog about how structured data is represented using
>> >>>> the two
>> >>>> > >> formats.
>> >>>> > >>
>> >>>> > >> Among other things, I hope this may help other Arrow / Parquet
>> >>>> > >> implementers as well as draw attention to the fact that parquet
>> and
>> >>>> > arrow
>> >>>> > >> both support nested structures, albeit differently.
>> >>>> > >>
>> >>>> > >> We would welcome any feedback from the community,
>> >>>> > >> Andrew
>> >>>> > >>
>> >>>> > >> [1] https://github.com/apache/arrow-site/pull/245
>> >>>> > >> [2] https://github.com/apache/arrow-site/pull/246
>> >>>> > >> [3] https://github.com/apache/arrow-site/pull/247
>> >>>> > >>
>> >>>> >
>> >>>> >
>> >>>>
>> >>>
>>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Andrew Lamb <al...@influxdata.com>.
I think it is a great idea to tweet it out -- can someone help me do so?
On Wed, Oct 19, 2022 at 8:26 PM David Li <li...@apache.org> wrote:
> Andrew, what do you think about tweeting this series out through
> @ApacheArrow?
>
> -David
>
> On Mon, Oct 17, 2022, at 16:50, Andrew Lamb wrote:
> > And the final installment:
> > https://arrow.apache.org/blog/2022/10/17/arrow-parquet-encoding-part-3/
> >
> > On Sat, Oct 8, 2022 at 9:47 AM Andrew Lamb <al...@influxdata.com> wrote:
> >
> >> Second post is now live (this time with non localhost url :thumbsup:):
> >>
> >> https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/
> >>
> >> Andrew
> >>
> >> On Wed, Oct 5, 2022 at 4:57 PM Andrew Lamb <al...@influxdata.com>
> wrote:
> >>
> >>> Let's just say the posting process has been a comedy of errors by me
> >>> 🤦♂️. I apologize for the noise
> >>>
> >>> On Wed, Oct 5, 2022 at 4:17 PM Will Jones <wi...@gmail.com>
> >>> wrote:
> >>>
> >>>>
> https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
> >>>>
> >>>> On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <
> >>>> krassovskysasha@gmail.com>
> >>>> wrote:
> >>>>
> >>>> > Hi, we aren’t able to connect to your localhost 😀
> >>>> >
> >>>> > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com>
> >>>> wrote:
> >>>> > >
> >>>> > > We have published the first post:
> >>>> > >
> http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
> >>>> > >
> >>>> > > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com>
> >>>> wrote:
> >>>> > >
> >>>> > >> We are working on a series of blog posts[1][2][3] that we plan to
> >>>> > publish
> >>>> > >> to the arrow blog about how structured data is represented using
> >>>> the two
> >>>> > >> formats.
> >>>> > >>
> >>>> > >> Among other things, I hope this may help other Arrow / Parquet
> >>>> > >> implementers as well as draw attention to the fact that parquet
> and
> >>>> > arrow
> >>>> > >> both support nested structures, albeit differently.
> >>>> > >>
> >>>> > >> We would welcome any feedback from the community,
> >>>> > >> Andrew
> >>>> > >>
> >>>> > >> [1] https://github.com/apache/arrow-site/pull/245
> >>>> > >> [2] https://github.com/apache/arrow-site/pull/246
> >>>> > >> [3] https://github.com/apache/arrow-site/pull/247
> >>>> > >>
> >>>> >
> >>>> >
> >>>>
> >>>
>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by David Li <li...@apache.org>.
Andrew, what do you think about tweeting this series out through @ApacheArrow?
-David
On Mon, Oct 17, 2022, at 16:50, Andrew Lamb wrote:
> And the final installment:
> https://arrow.apache.org/blog/2022/10/17/arrow-parquet-encoding-part-3/
>
> On Sat, Oct 8, 2022 at 9:47 AM Andrew Lamb <al...@influxdata.com> wrote:
>
>> Second post is now live (this time with non localhost url :thumbsup:):
>>
>> https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/
>>
>> Andrew
>>
>> On Wed, Oct 5, 2022 at 4:57 PM Andrew Lamb <al...@influxdata.com> wrote:
>>
>>> Let's just say the posting process has been a comedy of errors by me
>>> 🤦♂️. I apologize for the noise
>>>
>>> On Wed, Oct 5, 2022 at 4:17 PM Will Jones <wi...@gmail.com>
>>> wrote:
>>>
>>>> https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
>>>>
>>>> On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <
>>>> krassovskysasha@gmail.com>
>>>> wrote:
>>>>
>>>> > Hi, we aren’t able to connect to your localhost 😀
>>>> >
>>>> > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com>
>>>> wrote:
>>>> > >
>>>> > > We have published the first post:
>>>> > > http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
>>>> > >
>>>> > > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com>
>>>> wrote:
>>>> > >
>>>> > >> We are working on a series of blog posts[1][2][3] that we plan to
>>>> > publish
>>>> > >> to the arrow blog about how structured data is represented using
>>>> the two
>>>> > >> formats.
>>>> > >>
>>>> > >> Among other things, I hope this may help other Arrow / Parquet
>>>> > >> implementers as well as draw attention to the fact that parquet and
>>>> > arrow
>>>> > >> both support nested structures, albeit differently.
>>>> > >>
>>>> > >> We would welcome any feedback from the community,
>>>> > >> Andrew
>>>> > >>
>>>> > >> [1] https://github.com/apache/arrow-site/pull/245
>>>> > >> [2] https://github.com/apache/arrow-site/pull/246
>>>> > >> [3] https://github.com/apache/arrow-site/pull/247
>>>> > >>
>>>> >
>>>> >
>>>>
>>>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Andrew Lamb <al...@influxdata.com>.
And the final installment:
https://arrow.apache.org/blog/2022/10/17/arrow-parquet-encoding-part-3/
On Sat, Oct 8, 2022 at 9:47 AM Andrew Lamb <al...@influxdata.com> wrote:
> Second post is now live (this time with non localhost url :thumbsup:):
>
> https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/
>
> Andrew
>
> On Wed, Oct 5, 2022 at 4:57 PM Andrew Lamb <al...@influxdata.com> wrote:
>
>> Let's just say the posting process has been a comedy of errors by me
>> 🤦♂️. I apologize for the noise
>>
>> On Wed, Oct 5, 2022 at 4:17 PM Will Jones <wi...@gmail.com>
>> wrote:
>>
>>> https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
>>>
>>> On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <
>>> krassovskysasha@gmail.com>
>>> wrote:
>>>
>>> > Hi, we aren’t able to connect to your localhost 😀
>>> >
>>> > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com>
>>> wrote:
>>> > >
>>> > > We have published the first post:
>>> > > http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
>>> > >
>>> > > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com>
>>> wrote:
>>> > >
>>> > >> We are working on a series of blog posts[1][2][3] that we plan to
>>> > publish
>>> > >> to the arrow blog about how structured data is represented using
>>> the two
>>> > >> formats.
>>> > >>
>>> > >> Among other things, I hope this may help other Arrow / Parquet
>>> > >> implementers as well as draw attention to the fact that parquet and
>>> > arrow
>>> > >> both support nested structures, albeit differently.
>>> > >>
>>> > >> We would welcome any feedback from the community,
>>> > >> Andrew
>>> > >>
>>> > >> [1] https://github.com/apache/arrow-site/pull/245
>>> > >> [2] https://github.com/apache/arrow-site/pull/246
>>> > >> [3] https://github.com/apache/arrow-site/pull/247
>>> > >>
>>> >
>>> >
>>>
>>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Andrew Lamb <al...@influxdata.com>.
Second post is now live (this time with non localhost url :thumbsup:):
https://arrow.apache.org/blog/2022/10/08/arrow-parquet-encoding-part-2/
Andrew
On Wed, Oct 5, 2022 at 4:57 PM Andrew Lamb <al...@influxdata.com> wrote:
> Let's just say the posting process has been a comedy of errors by me
> 🤦♂️. I apologize for the noise
>
> On Wed, Oct 5, 2022 at 4:17 PM Will Jones <wi...@gmail.com> wrote:
>
>> https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
>>
>> On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <
>> krassovskysasha@gmail.com>
>> wrote:
>>
>> > Hi, we aren’t able to connect to your localhost 😀
>> >
>> > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com>
>> wrote:
>> > >
>> > > We have published the first post:
>> > > http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
>> > >
>> > > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com>
>> wrote:
>> > >
>> > >> We are working on a series of blog posts[1][2][3] that we plan to
>> > publish
>> > >> to the arrow blog about how structured data is represented using the
>> two
>> > >> formats.
>> > >>
>> > >> Among other things, I hope this may help other Arrow / Parquet
>> > >> implementers as well as draw attention to the fact that parquet and
>> > arrow
>> > >> both support nested structures, albeit differently.
>> > >>
>> > >> We would welcome any feedback from the community,
>> > >> Andrew
>> > >>
>> > >> [1] https://github.com/apache/arrow-site/pull/245
>> > >> [2] https://github.com/apache/arrow-site/pull/246
>> > >> [3] https://github.com/apache/arrow-site/pull/247
>> > >>
>> >
>> >
>>
>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Andrew Lamb <al...@influxdata.com>.
Let's just say the posting process has been a comedy of errors by me 🤦♂️.
I apologize for the noise
On Wed, Oct 5, 2022 at 4:17 PM Will Jones <wi...@gmail.com> wrote:
> https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
>
> On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <
> krassovskysasha@gmail.com>
> wrote:
>
> > Hi, we aren’t able to connect to your localhost 😀
> >
> > > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com> wrote:
> > >
> > > We have published the first post:
> > > http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
> > >
> > > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com>
> wrote:
> > >
> > >> We are working on a series of blog posts[1][2][3] that we plan to
> > publish
> > >> to the arrow blog about how structured data is represented using the
> two
> > >> formats.
> > >>
> > >> Among other things, I hope this may help other Arrow / Parquet
> > >> implementers as well as draw attention to the fact that parquet and
> > arrow
> > >> both support nested structures, albeit differently.
> > >>
> > >> We would welcome any feedback from the community,
> > >> Andrew
> > >>
> > >> [1] https://github.com/apache/arrow-site/pull/245
> > >> [2] https://github.com/apache/arrow-site/pull/246
> > >> [3] https://github.com/apache/arrow-site/pull/247
> > >>
> >
> >
>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Will Jones <wi...@gmail.com>.
https://arrow.apache.org/blog/2022/10/05/arrow-parquet-encoding-part-1/
On Wed, Oct 5, 2022 at 12:48 PM Sasha Krassovsky <kr...@gmail.com>
wrote:
> Hi, we aren’t able to connect to your localhost 😀
>
> > On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com> wrote:
> >
> > We have published the first post:
> > http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
> >
> > On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com> wrote:
> >
> >> We are working on a series of blog posts[1][2][3] that we plan to
> publish
> >> to the arrow blog about how structured data is represented using the two
> >> formats.
> >>
> >> Among other things, I hope this may help other Arrow / Parquet
> >> implementers as well as draw attention to the fact that parquet and
> arrow
> >> both support nested structures, albeit differently.
> >>
> >> We would welcome any feedback from the community,
> >> Andrew
> >>
> >> [1] https://github.com/apache/arrow-site/pull/245
> >> [2] https://github.com/apache/arrow-site/pull/246
> >> [3] https://github.com/apache/arrow-site/pull/247
> >>
>
>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Sasha Krassovsky <kr...@gmail.com>.
Hi, we aren’t able to connect to your localhost 😀
> On Oct 5, 2022, at 12:44 PM, Andrew Lamb <al...@influxdata.com> wrote:
>
> We have published the first post:
> http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
>
> On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com> wrote:
>
>> We are working on a series of blog posts[1][2][3] that we plan to publish
>> to the arrow blog about how structured data is represented using the two
>> formats.
>>
>> Among other things, I hope this may help other Arrow / Parquet
>> implementers as well as draw attention to the fact that parquet and arrow
>> both support nested structures, albeit differently.
>>
>> We would welcome any feedback from the community,
>> Andrew
>>
>> [1] https://github.com/apache/arrow-site/pull/245
>> [2] https://github.com/apache/arrow-site/pull/246
>> [3] https://github.com/apache/arrow-site/pull/247
>>
Re: [WEBSITE] Blog posts on representing Structured Data with Parquet and Arrow
Posted by Andrew Lamb <al...@influxdata.com>.
We have published the first post:
http://localhost:4000/blog/2022/10/05/arrow-parquet-encoding-part-1/
On Sun, Oct 2, 2022 at 7:00 AM Andrew Lamb <al...@influxdata.com> wrote:
> We are working on a series of blog posts[1][2][3] that we plan to publish
> to the arrow blog about how structured data is represented using the two
> formats.
>
> Among other things, I hope this may help other Arrow / Parquet
> implementers as well as draw attention to the fact that parquet and arrow
> both support nested structures, albeit differently.
>
> We would welcome any feedback from the community,
> Andrew
>
> [1] https://github.com/apache/arrow-site/pull/245
> [2] https://github.com/apache/arrow-site/pull/246
> [3] https://github.com/apache/arrow-site/pull/247
>