You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Joe Skora <js...@apache.org> on 2017/07/19 17:58:25 UTC
Binaries in embedded redis jar files
I noticed a new jar dependency for Redis includes binaries.
$ jar -xf
> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/embedded-redis-0.6.jar
> $ ls -l
> total 6292
> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
> $ file redis-server*
> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
> redis-server-2.8.19.app: Mach-O 64-bit executable
> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS Windows
>
So, installing NiFi will include externally built binaries, not just
scripts but full ELF, MacOS, and Windows PE binaries.
Is this a problem for Apache? Does this cause concern for anyone else?
Regards,
Joe
Re: Binaries in embedded redis jar files
Posted by Joe Skora <js...@gmail.com>.
My concern is that an unknown, unsigned binary could be downloaded and
executed with each build regardless of the operating system.
On Wed, Jul 19, 2017 at 2:19 PM, Michael Moser <mo...@gmail.com> wrote:
> Whoa, this causes serious concerns for me. While I can understand
> that the source code is the official ASF artifact, the nifi.apache.org
> site provides convenience binaries which I'm sure many people use.
> When NiFi 1.4.0 releases, we should write very conspicuous warnings on
> the Downloads page to advise users about this.
>
> -- Mike
>
>
> On Wed, Jul 19, 2017 at 2:04 PM, Joe Witt <jo...@gmail.com> wrote:
> > It is not a problem. It would be an issue if those were part of our
> > source release which is the official apache release artifact.
> >
> >
> >
> > On Wed, Jul 19, 2017 at 1:58 PM, Joe Skora <js...@apache.org> wrote:
> >> I noticed a new jar dependency for Redis includes binaries.
> >>
> >> $ jar -xf
> >>> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/
> embedded-redis-0.6.jar
> >>> $ ls -l
> >>> total 6292
> >>> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
> >>> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
> >>> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
> >>> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
> >>> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
> >>> $ file redis-server*
> >>> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
> >>> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
> >>> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
> >>> redis-server-2.8.19.app: Mach-O 64-bit executable
> >>> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS
> Windows
> >>>
> >>
> >> So, installing NiFi will include externally built binaries, not just
> >> scripts but full ELF, MacOS, and Windows PE binaries.
> >>
> >> Is this a problem for Apache? Does this cause concern for anyone else?
> >>
> >> Regards,
> >> Joe
>
Re: Binaries in embedded redis jar files
Posted by Pierre Villard <pi...@gmail.com>.
Mike, I think this is just something that will be downloaded in your Maven
repository in case you're building NiFi. But I don't think this will be
included in the NiFi binary itself. At least, that's what I understand.
Pierre
2017-07-19 20:19 GMT+02:00 Michael Moser <mo...@gmail.com>:
> Whoa, this causes serious concerns for me. While I can understand
> that the source code is the official ASF artifact, the nifi.apache.org
> site provides convenience binaries which I'm sure many people use.
> When NiFi 1.4.0 releases, we should write very conspicuous warnings on
> the Downloads page to advise users about this.
>
> -- Mike
>
>
> On Wed, Jul 19, 2017 at 2:04 PM, Joe Witt <jo...@gmail.com> wrote:
> > It is not a problem. It would be an issue if those were part of our
> > source release which is the official apache release artifact.
> >
> >
> >
> > On Wed, Jul 19, 2017 at 1:58 PM, Joe Skora <js...@apache.org> wrote:
> >> I noticed a new jar dependency for Redis includes binaries.
> >>
> >> $ jar -xf
> >>> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/
> embedded-redis-0.6.jar
> >>> $ ls -l
> >>> total 6292
> >>> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
> >>> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
> >>> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
> >>> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
> >>> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
> >>> $ file redis-server*
> >>> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
> >>> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
> >>> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
> >>> redis-server-2.8.19.app: Mach-O 64-bit executable
> >>> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS
> Windows
> >>>
> >>
> >> So, installing NiFi will include externally built binaries, not just
> >> scripts but full ELF, MacOS, and Windows PE binaries.
> >>
> >> Is this a problem for Apache? Does this cause concern for anyone else?
> >>
> >> Regards,
> >> Joe
>
Re: Binaries in embedded redis jar files
Posted by Michael Moser <mo...@gmail.com>.
OK, I see that embedded-redis.jar is only used in the test scope, so
this isn't really relevant to the convenience binaries. But I cannot
in good conscience allow my NiFi users to execute these unit tests if
they want to build on my corporate network.
-- Mike
On Wed, Jul 19, 2017 at 2:19 PM, Michael Moser <mo...@gmail.com> wrote:
> Whoa, this causes serious concerns for me. While I can understand
> that the source code is the official ASF artifact, the nifi.apache.org
> site provides convenience binaries which I'm sure many people use.
> When NiFi 1.4.0 releases, we should write very conspicuous warnings on
> the Downloads page to advise users about this.
>
> -- Mike
>
>
> On Wed, Jul 19, 2017 at 2:04 PM, Joe Witt <jo...@gmail.com> wrote:
>> It is not a problem. It would be an issue if those were part of our
>> source release which is the official apache release artifact.
>>
>>
>>
>> On Wed, Jul 19, 2017 at 1:58 PM, Joe Skora <js...@apache.org> wrote:
>>> I noticed a new jar dependency for Redis includes binaries.
>>>
>>> $ jar -xf
>>>> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/embedded-redis-0.6.jar
>>>> $ ls -l
>>>> total 6292
>>>> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
>>>> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
>>>> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
>>>> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
>>>> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
>>>> $ file redis-server*
>>>> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
>>>> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
>>>> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
>>>> redis-server-2.8.19.app: Mach-O 64-bit executable
>>>> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS Windows
>>>>
>>>
>>> So, installing NiFi will include externally built binaries, not just
>>> scripts but full ELF, MacOS, and Windows PE binaries.
>>>
>>> Is this a problem for Apache? Does this cause concern for anyone else?
>>>
>>> Regards,
>>> Joe
Re: Binaries in embedded redis jar files
Posted by Joey Frazee <jo...@icloud.com>.
In addition to only being for testing, these binaries are only used for an integration test, so we don't even need to worry about whether they're going to work on everyone's OS/cause test failures if the binaries don't work for you.
Now it is packaging up the entire Redis server binary, though, which is a little different than a class file or dll, so I think a different question is whether we want ITs that bring their own server in in this way?
A mock server might be sufficient or I've seen projects using docker-maven to provide all the server resources needed for ITs. This could be helpful for us for the Elastic, C*, etc. ITs too if we did it.
So binaries = ok and no warnings needed, but it might still be a valid question about whether we want to use this redis-embedded lib; but it doesn't strike me as a very urgent thing.
> On Jul 19, 2017, at 1:54 PM, Tony Kurc <tr...@gmail.com> wrote:
>
> Mike -
> Is there something about these binaries versus all of the other myriad
> binaries (jars, .class files, dlls and .so files) included in the
> convenience jar? I wouldn't think so, these are just more binary artifacts.
> I honestly can't think of another apache project that provides similar
> convenience binaries that have warnings such as those you mentioned.
>
> Tony
>
>> On Wed, Jul 19, 2017 at 2:19 PM, Michael Moser <mo...@gmail.com> wrote:
>>
>> Whoa, this causes serious concerns for me. While I can understand
>> that the source code is the official ASF artifact, the nifi.apache.org
>> site provides convenience binaries which I'm sure many people use.
>> When NiFi 1.4.0 releases, we should write very conspicuous warnings on
>> the Downloads page to advise users about this.
>>
>> -- Mike
>>
>>
>>> On Wed, Jul 19, 2017 at 2:04 PM, Joe Witt <jo...@gmail.com> wrote:
>>> It is not a problem. It would be an issue if those were part of our
>>> source release which is the official apache release artifact.
>>>
>>>
>>>
>>>> On Wed, Jul 19, 2017 at 1:58 PM, Joe Skora <js...@apache.org> wrote:
>>>> I noticed a new jar dependency for Redis includes binaries.
>>>>
>>>> $ jar -xf
>>>>> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/
>> embedded-redis-0.6.jar
>>>>> $ ls -l
>>>>> total 6292
>>>>> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
>>>>> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
>>>>> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
>>>>> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
>>>>> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
>>>>> $ file redis-server*
>>>>> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
>>>>> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
>>>>> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
>>>>> redis-server-2.8.19.app: Mach-O 64-bit executable
>>>>> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS
>> Windows
>>>>>
>>>>
>>>> So, installing NiFi will include externally built binaries, not just
>>>> scripts but full ELF, MacOS, and Windows PE binaries.
>>>>
>>>> Is this a problem for Apache? Does this cause concern for anyone else?
>>>>
>>>> Regards,
>>>> Joe
>>
Re: Binaries in embedded redis jar files
Posted by Tony Kurc <tr...@gmail.com>.
Mike -
Is there something about these binaries versus all of the other myriad
binaries (jars, .class files, dlls and .so files) included in the
convenience jar? I wouldn't think so, these are just more binary artifacts.
I honestly can't think of another apache project that provides similar
convenience binaries that have warnings such as those you mentioned.
Tony
On Wed, Jul 19, 2017 at 2:19 PM, Michael Moser <mo...@gmail.com> wrote:
> Whoa, this causes serious concerns for me. While I can understand
> that the source code is the official ASF artifact, the nifi.apache.org
> site provides convenience binaries which I'm sure many people use.
> When NiFi 1.4.0 releases, we should write very conspicuous warnings on
> the Downloads page to advise users about this.
>
> -- Mike
>
>
> On Wed, Jul 19, 2017 at 2:04 PM, Joe Witt <jo...@gmail.com> wrote:
> > It is not a problem. It would be an issue if those were part of our
> > source release which is the official apache release artifact.
> >
> >
> >
> > On Wed, Jul 19, 2017 at 1:58 PM, Joe Skora <js...@apache.org> wrote:
> >> I noticed a new jar dependency for Redis includes binaries.
> >>
> >> $ jar -xf
> >>> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/
> embedded-redis-0.6.jar
> >>> $ ls -l
> >>> total 6292
> >>> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
> >>> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
> >>> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
> >>> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
> >>> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
> >>> $ file redis-server*
> >>> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
> >>> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
> >>> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
> >>> redis-server-2.8.19.app: Mach-O 64-bit executable
> >>> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS
> Windows
> >>>
> >>
> >> So, installing NiFi will include externally built binaries, not just
> >> scripts but full ELF, MacOS, and Windows PE binaries.
> >>
> >> Is this a problem for Apache? Does this cause concern for anyone else?
> >>
> >> Regards,
> >> Joe
>
Re: Binaries in embedded redis jar files
Posted by Michael Moser <mo...@gmail.com>.
Whoa, this causes serious concerns for me. While I can understand
that the source code is the official ASF artifact, the nifi.apache.org
site provides convenience binaries which I'm sure many people use.
When NiFi 1.4.0 releases, we should write very conspicuous warnings on
the Downloads page to advise users about this.
-- Mike
On Wed, Jul 19, 2017 at 2:04 PM, Joe Witt <jo...@gmail.com> wrote:
> It is not a problem. It would be an issue if those were part of our
> source release which is the official apache release artifact.
>
>
>
> On Wed, Jul 19, 2017 at 1:58 PM, Joe Skora <js...@apache.org> wrote:
>> I noticed a new jar dependency for Redis includes binaries.
>>
>> $ jar -xf
>>> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/embedded-redis-0.6.jar
>>> $ ls -l
>>> total 6292
>>> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
>>> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
>>> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
>>> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
>>> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
>>> $ file redis-server*
>>> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
>>> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
>>> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
>>> redis-server-2.8.19.app: Mach-O 64-bit executable
>>> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS Windows
>>>
>>
>> So, installing NiFi will include externally built binaries, not just
>> scripts but full ELF, MacOS, and Windows PE binaries.
>>
>> Is this a problem for Apache? Does this cause concern for anyone else?
>>
>> Regards,
>> Joe
Re: Binaries in embedded redis jar files
Posted by Joe Witt <jo...@gmail.com>.
It is not a problem. It would be an issue if those were part of our
source release which is the official apache release artifact.
On Wed, Jul 19, 2017 at 1:58 PM, Joe Skora <js...@apache.org> wrote:
> I noticed a new jar dependency for Redis includes binaries.
>
> $ jar -xf
>> ~/.m2/repository/com/github/kstyrc/embedded-redis/0.6/embedded-redis-0.6.jar
>> $ ls -l
>> total 6292
>> drwxr-xr-x. 3 user1 users 36 Apr 12 2015 META-INF/
>> drwxr-xr-x. 3 user1 users 21 Apr 12 2015 redis/
>> -rw-r--r--. 1 user1 users 4236300 Apr 12 2015 redis-server-2.8.19
>> -rw-r--r--. 1 user1 users 806944 Apr 12 2015 redis-server-2.8.19.app
>> -rw-r--r--. 1 user1 users 1392640 Apr 12 2015 redis-server-2.8.19.exe
>> $ file redis-server*
>> redis-server-2.8.19: ELF 64-bit LSB executable, x86-64, version 1
>> (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
>> BuildID[sha1]=c59ab61e1d5b94e26d079b04e9e593b6be3f0230, not stripped
>> redis-server-2.8.19.app: Mach-O 64-bit executable
>> redis-server-2.8.19.exe: PE32+ executable (console) x86-64, for MS Windows
>>
>
> So, installing NiFi will include externally built binaries, not just
> scripts but full ELF, MacOS, and Windows PE binaries.
>
> Is this a problem for Apache? Does this cause concern for anyone else?
>
> Regards,
> Joe