Move Dremel paper to parquet-format

[Nandor Kollar <nk...@cloudera.com>]

Hi All,

I'm wondering if we can move the Dremel paper to parquet-format wiki. Right
now, every reference to this paper in Github (both parquet-mr and
parquet-format readme) and the website refers to Julien's Github
<https://github.com/julienledem/redelm/wiki/The-striping-and-assembly-algorithms-from-the-Dremel-paper>,
it would be nice if we can make this consistent, and move the mentioned
page to a Github Wiki page inside apache/parquet-format.

Regards,
Nandor

Re: Move Dremel paper to parquet-format

["Uwe L. Korn" <uw...@xhochy.com>]

Hello,

in Arrow we have the website in the main repository [1] as Markdown files. Review is done with patches like any other project. The deployment to the actual Apache servers is done using a separate (magic) git-repo. This is working really nice for us (and also some other Apache projects are also happy with this approach). I would also recommend to use this for Parquet. The only negative point I see is that the deployment is still done manually: a committer needs to run a script locally that builds the site and push the rendered version back to the magic git repo so that site is updated.

Uwe

[1]

On Tue, May 29, 2018, at 1:42 PM, Zoltan Ivanfi wrote:
> Hi,
> 
> Taking a step back, are we satisfied with the current web page mechanism? I
> find its dependence on subversion a real pain (checking it out, making
> patches for review, and the reviews themselves are a lot more complicated
> than with github). I think that's one of the main reasons it's so neglected
> (it describes Parquet as of 2003). Can't we use a Wiki for the same
> purpose? Or .md files in the github repo? Or can we migrate the web page to
> its own github repo?
> 
> Best,
> 
> Zoltan
> 
> On Tue, May 29, 2018 at 1:21 PM Uwe L. Korn <uw...@xhochy.com> wrote:
> 
> > Hello Nandor,
> >
> > as it seems that wiki contents were written by Julian and as they are on
> > github wiki, they are markdown in the backend.
> >
> > The easiest thing from an IP side would be if Julien could contribute as
> > plain markdown files to the parquet-format repo. I don't think we want/can
> > to enable the wiki for the parquet-format repo.
> >
> > Uwe
> >
> > On Tue, May 29, 2018, at 12:36 PM, Nandor Kollar wrote:
> > > Hi All,
> > >
> > > I'm wondering if we can move the Dremel paper to parquet-format wiki.
> > > Right
> > > now, every reference to this paper in Github (both parquet-mr and
> > > parquet-format readme) and the website refers to Julien's Github
> > > <
> > https://github.com/julienledem/redelm/wiki/The-striping-and-assembly-algorithms-from-the-Dremel-paper
> > >,
> > > it would be nice if we can make this consistent, and move the mentioned
> > > page to a Github Wiki page inside apache/parquet-format.
> > >
> > > Regards,
> > > Nandor
> >

Re: Move Dremel paper to parquet-format

[Nandor Kollar <nk...@cloudera.com>]

Moving the details about details, implementation, types etc. to Github is a
good idea. In my opinion the website on parquet.apache.org should provide
just a very high level overview of Parquet with links to Github pages and
contact information.

Nandor

On Tue, May 29, 2018 at 1:42 PM, Zoltan Ivanfi <zi...@cloudera.com> wrote:

> Hi,
>
> Taking a step back, are we satisfied with the current web page mechanism? I
> find its dependence on subversion a real pain (checking it out, making
> patches for review, and the reviews themselves are a lot more complicated
> than with github). I think that's one of the main reasons it's so neglected
> (it describes Parquet as of 2003). Can't we use a Wiki for the same
> purpose? Or .md files in the github repo? Or can we migrate the web page to
> its own github repo?
>
> Best,
>
> Zoltan
>
> On Tue, May 29, 2018 at 1:21 PM Uwe L. Korn <uw...@xhochy.com> wrote:
>
> > Hello Nandor,
> >
> > as it seems that wiki contents were written by Julian and as they are on
> > github wiki, they are markdown in the backend.
> >
> > The easiest thing from an IP side would be if Julien could contribute as
> > plain markdown files to the parquet-format repo. I don't think we
> want/can
> > to enable the wiki for the parquet-format repo.
> >
> > Uwe
> >
> > On Tue, May 29, 2018, at 12:36 PM, Nandor Kollar wrote:
> > > Hi All,
> > >
> > > I'm wondering if we can move the Dremel paper to parquet-format wiki.
> > > Right
> > > now, every reference to this paper in Github (both parquet-mr and
> > > parquet-format readme) and the website refers to Julien's Github
> > > <
> > https://github.com/julienledem/redelm/wiki/The-striping-and-assembly-
> algorithms-from-the-Dremel-paper
> > >,
> > > it would be nice if we can make this consistent, and move the mentioned
> > > page to a Github Wiki page inside apache/parquet-format.
> > >
> > > Regards,
> > > Nandor
> >
>

Re: Move Dremel paper to parquet-format

[Zoltan Ivanfi <zi...@cloudera.com>]

Hi,

Taking a step back, are we satisfied with the current web page mechanism? I
find its dependence on subversion a real pain (checking it out, making
patches for review, and the reviews themselves are a lot more complicated
than with github). I think that's one of the main reasons it's so neglected
(it describes Parquet as of 2003). Can't we use a Wiki for the same
purpose? Or .md files in the github repo? Or can we migrate the web page to
its own github repo?

Best,

Zoltan

On Tue, May 29, 2018 at 1:21 PM Uwe L. Korn <uw...@xhochy.com> wrote:

> Hello Nandor,
>
> as it seems that wiki contents were written by Julian and as they are on
> github wiki, they are markdown in the backend.
>
> The easiest thing from an IP side would be if Julien could contribute as
> plain markdown files to the parquet-format repo. I don't think we want/can
> to enable the wiki for the parquet-format repo.
>
> Uwe
>
> On Tue, May 29, 2018, at 12:36 PM, Nandor Kollar wrote:
> > Hi All,
> >
> > I'm wondering if we can move the Dremel paper to parquet-format wiki.
> > Right
> > now, every reference to this paper in Github (both parquet-mr and
> > parquet-format readme) and the website refers to Julien's Github
> > <
> https://github.com/julienledem/redelm/wiki/The-striping-and-assembly-algorithms-from-the-Dremel-paper
> >,
> > it would be nice if we can make this consistent, and move the mentioned
> > page to a Github Wiki page inside apache/parquet-format.
> >
> > Regards,
> > Nandor
>

Re: Move Dremel paper to parquet-format

["Uwe L. Korn" <uw...@xhochy.com>]

Hello Nandor,

as it seems that wiki contents were written by Julian and as they are on github wiki, they are markdown in the backend.

The easiest thing from an IP side would be if Julien could contribute as plain markdown files to the parquet-format repo. I don't think we want/can to enable the wiki for the parquet-format repo.

Uwe

On Tue, May 29, 2018, at 12:36 PM, Nandor Kollar wrote:
> Hi All,
> 
> I'm wondering if we can move the Dremel paper to parquet-format wiki. 
> Right
> now, every reference to this paper in Github (both parquet-mr and
> parquet-format readme) and the website refers to Julien's Github
> <https://github.com/julienledem/redelm/wiki/The-striping-and-assembly-algorithms-from-the-Dremel-paper>,
> it would be nice if we can make this consistent, and move the mentioned
> page to a Github Wiki page inside apache/parquet-format.
> 
> Regards,
> Nandor