You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Julian Hyde <jh...@apache.org> on 2020/07/16 00:02:26 UTC
[DISCUSS] Binary files for testing InnoDB adapter
TL;DR: PMC members, would you vote for a release 1.24 if it includes
binary files necessary for testing?
I would like to include the InnoDB adapter [1] in release 1.24. It is
well written, well documented, and it is ready.
There is one problem: there are some binary files (in InnoDB format)
[2] that are included for testing. As a general rule, Apache does not
release binary files as part of the source release because they are
difficult to audit for provenance.
I think we should make an exception, for just release 1.24, because
the files are small (just EMP and DEPT tables) and generated by hand.
I have asked the author to do a follow-up task to remove the files
before next release.
PMC members, please reply to this email and indicate whether this
would cause you to vote -1 on the upcoming release.
Julian
[1] https://issues.apache.org/jira/browse/CALCITE-4034
[2] https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Enrico Olivelli <eo...@gmail.com>.
IMHO If you need those files for tests and you don't have a way to generate
them it is allowed to keep them.
You can add some readme file that explain their nature. You can also add a
check sum file.
Just my 2 cents
Enrico
Il Gio 16 Lug 2020, 05:40 Francis Chuang <fr...@apache.org> ha
scritto:
> I am +1 for including the files in this release as long as they are
> removed in the next release.
>
> Francis
>
> On 16/07/2020 1:08 pm, Haisheng Yuan wrote:
> > +1
> >
> > I am fine to make an exception for this release.
> > Let's see what's author's plan to remove the binary files in next
> release.
> >
> > On 2020/07/16 00:02:26, Julian Hyde <jh...@apache.org> wrote:
> >> TL;DR: PMC members, would you vote for a release 1.24 if it includes
> >> binary files necessary for testing?
> >>
> >> I would like to include the InnoDB adapter [1] in release 1.24. It is
> >> well written, well documented, and it is ready.
> >>
> >> There is one problem: there are some binary files (in InnoDB format)
> >> [2] that are included for testing. As a general rule, Apache does not
> >> release binary files as part of the source release because they are
> >> difficult to audit for provenance.
> >>
> >> I think we should make an exception, for just release 1.24, because
> >> the files are small (just EMP and DEPT tables) and generated by hand.
> >>
> >> I have asked the author to do a follow-up task to remove the files
> >> before next release.
> >>
> >> PMC members, please reply to this email and indicate whether this
> >> would cause you to vote -1 on the upcoming release.
> >>
> >> Julian
> >>
> >> [1] https://issues.apache.org/jira/browse/CALCITE-4034
> >>
> >> [2]
> https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
> >>
>
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Francis Chuang <fr...@apache.org>.
I am +1 for including the files in this release as long as they are
removed in the next release.
Francis
On 16/07/2020 1:08 pm, Haisheng Yuan wrote:
> +1
>
> I am fine to make an exception for this release.
> Let's see what's author's plan to remove the binary files in next release.
>
> On 2020/07/16 00:02:26, Julian Hyde <jh...@apache.org> wrote:
>> TL;DR: PMC members, would you vote for a release 1.24 if it includes
>> binary files necessary for testing?
>>
>> I would like to include the InnoDB adapter [1] in release 1.24. It is
>> well written, well documented, and it is ready.
>>
>> There is one problem: there are some binary files (in InnoDB format)
>> [2] that are included for testing. As a general rule, Apache does not
>> release binary files as part of the source release because they are
>> difficult to audit for provenance.
>>
>> I think we should make an exception, for just release 1.24, because
>> the files are small (just EMP and DEPT tables) and generated by hand.
>>
>> I have asked the author to do a follow-up task to remove the files
>> before next release.
>>
>> PMC members, please reply to this email and indicate whether this
>> would cause you to vote -1 on the upcoming release.
>>
>> Julian
>>
>> [1] https://issues.apache.org/jira/browse/CALCITE-4034
>>
>> [2] https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
>>
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Haisheng Yuan <hy...@apache.org>.
+1
I am fine to make an exception for this release.
Let's see what's author's plan to remove the binary files in next release.
On 2020/07/16 00:02:26, Julian Hyde <jh...@apache.org> wrote:
> TL;DR: PMC members, would you vote for a release 1.24 if it includes
> binary files necessary for testing?
>
> I would like to include the InnoDB adapter [1] in release 1.24. It is
> well written, well documented, and it is ready.
>
> There is one problem: there are some binary files (in InnoDB format)
> [2] that are included for testing. As a general rule, Apache does not
> release binary files as part of the source release because they are
> difficult to audit for provenance.
>
> I think we should make an exception, for just release 1.24, because
> the files are small (just EMP and DEPT tables) and generated by hand.
>
> I have asked the author to do a follow-up task to remove the files
> before next release.
>
> PMC members, please reply to this email and indicate whether this
> would cause you to vote -1 on the upcoming release.
>
> Julian
>
> [1] https://issues.apache.org/jira/browse/CALCITE-4034
>
> [2] https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
>
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Forward Xu <fo...@gmail.com>.
+1
If we can add the function(Example: innodb-java-writer) of generating ibd
files in the next version, I think it will be better. In this way, InnoDB
adapter initially has the capability of a database, and can further improve
the current unit test.
best
Forward
Stamatis Zampetakis <za...@gmail.com> 于2020年7月17日周五 上午6:25写道：
> Hello,
>
> It's good to avoid non-readable files when possible but I don't think we
> have to worry too much about it.
>
> The ASF foundation does not require all files in a release to be
> protected by copyright law and lists a few exceptions [1].
>
> The file is a database dump so I don't see much creativity involved so it
> could be classified under the following case.
>
> "A file without any degree of creativity in either its literal elements or
> its structure is not protected by copyright law; therefore, such a file
> does not require a license header."
>
> Based on my understanding of the policy, I don't see a problem leaving this
> file as is even after 1.24 but if there are doubts I can ping legal@.
>
> Best,
> Stamatis
>
> [1] https://www.apache.org/legal/src-headers.html#faq-exceptions
>
>
> On Thu, Jul 16, 2020 at 10:54 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Il Gio 16 Lug 2020, 21:28 Julian Hyde <jh...@apache.org> ha scritto:
> >
> > > Michael,
> > >
> > > I think that sentence covers the situation where, say, we ship a
> > > .class file without shipping the .java it was generated from.
> > >
> > > In our case, we are shipping the equivalent of the .class file and the
> > > .java file (namely EMP.ibd and the scott.sql that MySQL generated it
> > > from). We have to ship the .class file equivalent (.ibd) because we
> > > don't require that people running the tests have the javac equivalent
> > > (MySQL).
> > >
> > > The concern about auditability remains.
> > >
> >
> > You can wrap the files in a text based format with a readable license
> > header and the test case can unpack the files and use them. It is tricky
> > but it can work
> >
> >
> > Enrico
> >
> >
> > > Julian
> > >
> > > On Thu, Jul 16, 2020 at 8:34 AM Michael Mior <mm...@apache.org> wrote:
> > > >
> > > > I don't personally have a problem with this, but it seems as though
> it
> > > > might violate the release policy. Specifically the statement "It is
> > > > also necessary for the PMC to ensure that the source package is
> > > > sufficient to build any binary artifacts associated with the
> release."
> > > >
> > > > https://www.apache.org/legal/release-policy.html#what
> > > >
> > > > --
> > > > Michael Mior
> > > > mmior@apache.org
> > > >
> > > > Le mer. 15 juil. 2020 à 20:02, Julian Hyde <jh...@apache.org> a
> écrit
> > :
> > > > >
> > > > > TL;DR: PMC members, would you vote for a release 1.24 if it
> includes
> > > > > binary files necessary for testing?
> > > > >
> > > > > I would like to include the InnoDB adapter [1] in release 1.24. It
> is
> > > > > well written, well documented, and it is ready.
> > > > >
> > > > > There is one problem: there are some binary files (in InnoDB
> format)
> > > > > [2] that are included for testing. As a general rule, Apache does
> not
> > > > > release binary files as part of the source release because they are
> > > > > difficult to audit for provenance.
> > > > >
> > > > > I think we should make an exception, for just release 1.24, because
> > > > > the files are small (just EMP and DEPT tables) and generated by
> hand.
> > > > >
> > > > > I have asked the author to do a follow-up task to remove the files
> > > > > before next release.
> > > > >
> > > > > PMC members, please reply to this email and indicate whether this
> > > > > would cause you to vote -1 on the upcoming release.
> > > > >
> > > > > Julian
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/CALCITE-4034
> > > > >
> > > > > [2]
> > >
> >
> https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
> > >
> >
>
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Stamatis Zampetakis <za...@gmail.com>.
Hello,
It's good to avoid non-readable files when possible but I don't think we
have to worry too much about it.
The ASF foundation does not require all files in a release to be
protected by copyright law and lists a few exceptions [1].
The file is a database dump so I don't see much creativity involved so it
could be classified under the following case.
"A file without any degree of creativity in either its literal elements or
its structure is not protected by copyright law; therefore, such a file
does not require a license header."
Based on my understanding of the policy, I don't see a problem leaving this
file as is even after 1.24 but if there are doubts I can ping legal@.
Best,
Stamatis
[1] https://www.apache.org/legal/src-headers.html#faq-exceptions
On Thu, Jul 16, 2020 at 10:54 PM Enrico Olivelli <eo...@gmail.com>
wrote:
> Il Gio 16 Lug 2020, 21:28 Julian Hyde <jh...@apache.org> ha scritto:
>
> > Michael,
> >
> > I think that sentence covers the situation where, say, we ship a
> > .class file without shipping the .java it was generated from.
> >
> > In our case, we are shipping the equivalent of the .class file and the
> > .java file (namely EMP.ibd and the scott.sql that MySQL generated it
> > from). We have to ship the .class file equivalent (.ibd) because we
> > don't require that people running the tests have the javac equivalent
> > (MySQL).
> >
> > The concern about auditability remains.
> >
>
> You can wrap the files in a text based format with a readable license
> header and the test case can unpack the files and use them. It is tricky
> but it can work
>
>
> Enrico
>
>
> > Julian
> >
> > On Thu, Jul 16, 2020 at 8:34 AM Michael Mior <mm...@apache.org> wrote:
> > >
> > > I don't personally have a problem with this, but it seems as though it
> > > might violate the release policy. Specifically the statement "It is
> > > also necessary for the PMC to ensure that the source package is
> > > sufficient to build any binary artifacts associated with the release."
> > >
> > > https://www.apache.org/legal/release-policy.html#what
> > >
> > > --
> > > Michael Mior
> > > mmior@apache.org
> > >
> > > Le mer. 15 juil. 2020 à 20:02, Julian Hyde <jh...@apache.org> a écrit
> :
> > > >
> > > > TL;DR: PMC members, would you vote for a release 1.24 if it includes
> > > > binary files necessary for testing?
> > > >
> > > > I would like to include the InnoDB adapter [1] in release 1.24. It is
> > > > well written, well documented, and it is ready.
> > > >
> > > > There is one problem: there are some binary files (in InnoDB format)
> > > > [2] that are included for testing. As a general rule, Apache does not
> > > > release binary files as part of the source release because they are
> > > > difficult to audit for provenance.
> > > >
> > > > I think we should make an exception, for just release 1.24, because
> > > > the files are small (just EMP and DEPT tables) and generated by hand.
> > > >
> > > > I have asked the author to do a follow-up task to remove the files
> > > > before next release.
> > > >
> > > > PMC members, please reply to this email and indicate whether this
> > > > would cause you to vote -1 on the upcoming release.
> > > >
> > > > Julian
> > > >
> > > > [1] https://issues.apache.org/jira/browse/CALCITE-4034
> > > >
> > > > [2]
> >
> https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
> >
>
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Enrico Olivelli <eo...@gmail.com>.
Il Gio 16 Lug 2020, 21:28 Julian Hyde <jh...@apache.org> ha scritto:
> Michael,
>
> I think that sentence covers the situation where, say, we ship a
> .class file without shipping the .java it was generated from.
>
> In our case, we are shipping the equivalent of the .class file and the
> .java file (namely EMP.ibd and the scott.sql that MySQL generated it
> from). We have to ship the .class file equivalent (.ibd) because we
> don't require that people running the tests have the javac equivalent
> (MySQL).
>
> The concern about auditability remains.
>
You can wrap the files in a text based format with a readable license
header and the test case can unpack the files and use them. It is tricky
but it can work
Enrico
> Julian
>
> On Thu, Jul 16, 2020 at 8:34 AM Michael Mior <mm...@apache.org> wrote:
> >
> > I don't personally have a problem with this, but it seems as though it
> > might violate the release policy. Specifically the statement "It is
> > also necessary for the PMC to ensure that the source package is
> > sufficient to build any binary artifacts associated with the release."
> >
> > https://www.apache.org/legal/release-policy.html#what
> >
> > --
> > Michael Mior
> > mmior@apache.org
> >
> > Le mer. 15 juil. 2020 à 20:02, Julian Hyde <jh...@apache.org> a écrit :
> > >
> > > TL;DR: PMC members, would you vote for a release 1.24 if it includes
> > > binary files necessary for testing?
> > >
> > > I would like to include the InnoDB adapter [1] in release 1.24. It is
> > > well written, well documented, and it is ready.
> > >
> > > There is one problem: there are some binary files (in InnoDB format)
> > > [2] that are included for testing. As a general rule, Apache does not
> > > release binary files as part of the source release because they are
> > > difficult to audit for provenance.
> > >
> > > I think we should make an exception, for just release 1.24, because
> > > the files are small (just EMP and DEPT tables) and generated by hand.
> > >
> > > I have asked the author to do a follow-up task to remove the files
> > > before next release.
> > >
> > > PMC members, please reply to this email and indicate whether this
> > > would cause you to vote -1 on the upcoming release.
> > >
> > > Julian
> > >
> > > [1] https://issues.apache.org/jira/browse/CALCITE-4034
> > >
> > > [2]
> https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
>
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Michael Mior <mm...@apache.org>.
That explanation makes sense to me. +1 Since it can't be easily
automated, I'd suggest we include a very brief description of how to
recreate the file.
--
Michael Mior
mmior@apache.org
Le jeu. 16 juil. 2020 à 15:28, Julian Hyde <jh...@apache.org> a écrit :
>
> Michael,
>
> I think that sentence covers the situation where, say, we ship a
> .class file without shipping the .java it was generated from.
>
> In our case, we are shipping the equivalent of the .class file and the
> .java file (namely EMP.ibd and the scott.sql that MySQL generated it
> from). We have to ship the .class file equivalent (.ibd) because we
> don't require that people running the tests have the javac equivalent
> (MySQL).
>
> The concern about auditability remains.
>
> Julian
>
> On Thu, Jul 16, 2020 at 8:34 AM Michael Mior <mm...@apache.org> wrote:
> >
> > I don't personally have a problem with this, but it seems as though it
> > might violate the release policy. Specifically the statement "It is
> > also necessary for the PMC to ensure that the source package is
> > sufficient to build any binary artifacts associated with the release."
> >
> > https://www.apache.org/legal/release-policy.html#what
> >
> > --
> > Michael Mior
> > mmior@apache.org
> >
> > Le mer. 15 juil. 2020 à 20:02, Julian Hyde <jh...@apache.org> a écrit :
> > >
> > > TL;DR: PMC members, would you vote for a release 1.24 if it includes
> > > binary files necessary for testing?
> > >
> > > I would like to include the InnoDB adapter [1] in release 1.24. It is
> > > well written, well documented, and it is ready.
> > >
> > > There is one problem: there are some binary files (in InnoDB format)
> > > [2] that are included for testing. As a general rule, Apache does not
> > > release binary files as part of the source release because they are
> > > difficult to audit for provenance.
> > >
> > > I think we should make an exception, for just release 1.24, because
> > > the files are small (just EMP and DEPT tables) and generated by hand.
> > >
> > > I have asked the author to do a follow-up task to remove the files
> > > before next release.
> > >
> > > PMC members, please reply to this email and indicate whether this
> > > would cause you to vote -1 on the upcoming release.
> > >
> > > Julian
> > >
> > > [1] https://issues.apache.org/jira/browse/CALCITE-4034
> > >
> > > [2] https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Julian Hyde <jh...@apache.org>.
Michael,
I think that sentence covers the situation where, say, we ship a
.class file without shipping the .java it was generated from.
In our case, we are shipping the equivalent of the .class file and the
.java file (namely EMP.ibd and the scott.sql that MySQL generated it
from). We have to ship the .class file equivalent (.ibd) because we
don't require that people running the tests have the javac equivalent
(MySQL).
The concern about auditability remains.
Julian
On Thu, Jul 16, 2020 at 8:34 AM Michael Mior <mm...@apache.org> wrote:
>
> I don't personally have a problem with this, but it seems as though it
> might violate the release policy. Specifically the statement "It is
> also necessary for the PMC to ensure that the source package is
> sufficient to build any binary artifacts associated with the release."
>
> https://www.apache.org/legal/release-policy.html#what
>
> --
> Michael Mior
> mmior@apache.org
>
> Le mer. 15 juil. 2020 à 20:02, Julian Hyde <jh...@apache.org> a écrit :
> >
> > TL;DR: PMC members, would you vote for a release 1.24 if it includes
> > binary files necessary for testing?
> >
> > I would like to include the InnoDB adapter [1] in release 1.24. It is
> > well written, well documented, and it is ready.
> >
> > There is one problem: there are some binary files (in InnoDB format)
> > [2] that are included for testing. As a general rule, Apache does not
> > release binary files as part of the source release because they are
> > difficult to audit for provenance.
> >
> > I think we should make an exception, for just release 1.24, because
> > the files are small (just EMP and DEPT tables) and generated by hand.
> >
> > I have asked the author to do a follow-up task to remove the files
> > before next release.
> >
> > PMC members, please reply to this email and indicate whether this
> > would cause you to vote -1 on the upcoming release.
> >
> > Julian
> >
> > [1] https://issues.apache.org/jira/browse/CALCITE-4034
> >
> > [2] https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data
Re: [DISCUSS] Binary files for testing InnoDB adapter
Posted by Michael Mior <mm...@apache.org>.
I don't personally have a problem with this, but it seems as though it
might violate the release policy. Specifically the statement "It is
also necessary for the PMC to ensure that the source package is
sufficient to build any binary artifacts associated with the release."
https://www.apache.org/legal/release-policy.html#what
--
Michael Mior
mmior@apache.org
Le mer. 15 juil. 2020 à 20:02, Julian Hyde <jh...@apache.org> a écrit :
>
> TL;DR: PMC members, would you vote for a release 1.24 if it includes
> binary files necessary for testing?
>
> I would like to include the InnoDB adapter [1] in release 1.24. It is
> well written, well documented, and it is ready.
>
> There is one problem: there are some binary files (in InnoDB format)
> [2] that are included for testing. As a general rule, Apache does not
> release binary files as part of the source release because they are
> difficult to audit for provenance.
>
> I think we should make an exception, for just release 1.24, because
> the files are small (just EMP and DEPT tables) and generated by hand.
>
> I have asked the author to do a follow-up task to remove the files
> before next release.
>
> PMC members, please reply to this email and indicate whether this
> would cause you to vote -1 on the upcoming release.
>
> Julian
>
> [1] https://issues.apache.org/jira/browse/CALCITE-4034
>
> [2] https://github.com/apache/calcite/tree/b5e1622e7a43a3468a880c374f9161eee3ffa1ea/innodb/src/test/resources/data