You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mark Claassen <mc...@ocie.net> on 2006/05/23 16:46:25 UTC
Binary build procedures
My boss has implemented some new procedures with regard to open source
projects. He believes the source distributions are trustworthy, but he is
not sure if he trusts the binary distributions. I think the reasoning is
that he is uncertain if the binary distributions are controlled as well as
the source ones are. And if they are not, someone could inject some
malicious code to expose customer data or something.
Can someone give me a brief explanation on how the binary distributions are
created for 5.5? Are the binary distributions created automatically from
the repository, leaving no chance for nefarious tampering?
Thanks,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Binary build procedures
Posted by Mark Claassen <mc...@ocie.net>.
Thanks for all the information. To paraphrase what you are saying, the
sources and binary distros are tightly controlled. The binary builds (for
the whole Apache Foundation) are created and maintained with security in
mind by people who know what they are doing.
-----Original Message-----
From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf Of Yoav
Shapira
Sent: Tuesday, May 23, 2006 10:53 AM
To: Tomcat Developers List
Subject: Re: Binary build procedures
Mark,
The binary distributions are handled with the same security precautions as
the source ones. Each distribution file is accompanied by its MD5 checksum
and is PGP-signed by the release manager.
The MD5 checksums, PGP signatures, and KEYS files (available with the distro
as well as on the main download pages) are all unmirrored, residing only on
the original apache.org servers. So in addition to the security granted by
MD5 and PGP, someone would have to hack apache.org and modify those very
files in order to get you to trust the release. I'm not aware of that ever
happening in the past.
Besides noting that the security for source distros (which you already
trust) are the same as binary distros, I'd further note that these
procedures are standard across the Foundation (i.e. Tomcat doesn't do
anything special here), and as such have been devised, verified, and are
monitored by a number of folks who know a whole lot more than I do about
distro integrity.
Finally, if you still don't trust binaries but do trust sources, you always
have the option of grabbing the latter distro and building the binary
yourself ;)
Yoav
On 5/23/06, Mark Claassen <mc...@ocie.net> wrote:
>
> My boss has implemented some new procedures with regard to open source
> projects. He believes the source distributions are trustworthy, but
> he is not sure if he trusts the binary distributions. I think the
> reasoning is that he is uncertain if the binary distributions are
> controlled as well as the source ones are. And if they are not,
> someone could inject some malicious code to expose customer data or
something.
>
> Can someone give me a brief explanation on how the binary
> distributions are created for 5.5? Are the binary distributions
> created automatically from the repository, leaving no chance for nefarious
tampering?
>
> Thanks,
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For
> additional commands, e-mail: dev-help@tomcat.apache.org
>
>
--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
yoavs@computer.org / www.yoavshapira.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Binary build procedures
Posted by Yoav Shapira <yo...@apache.org>.
Mark,
The binary distributions are handled with the same security
precautions as the source ones. Each distribution file is accompanied
by its MD5 checksum and is PGP-signed by the release manager.
The MD5 checksums, PGP signatures, and KEYS files (available with the
distro as well as on the main download pages) are all unmirrored,
residing only on the original apache.org servers. So in addition to
the security granted by MD5 and PGP, someone would have to hack
apache.org and modify those very files in order to get you to trust
the release. I'm not aware of that ever happening in the past.
Besides noting that the security for source distros (which you already
trust) are the same as binary distros, I'd further note that these
procedures are standard across the Foundation (i.e. Tomcat doesn't do
anything special here), and as such have been devised, verified, and
are monitored by a number of folks who know a whole lot more than I do
about distro integrity.
Finally, if you still don't trust binaries but do trust sources, you
always have the option of grabbing the latter distro and building the
binary yourself ;)
Yoav
On 5/23/06, Mark Claassen <mc...@ocie.net> wrote:
>
> My boss has implemented some new procedures with regard to open source
> projects. He believes the source distributions are trustworthy, but he is
> not sure if he trusts the binary distributions. I think the reasoning is
> that he is uncertain if the binary distributions are controlled as well as
> the source ones are. And if they are not, someone could inject some
> malicious code to expose customer data or something.
>
> Can someone give me a brief explanation on how the binary distributions are
> created for 5.5? Are the binary distributions created automatically from
> the repository, leaving no chance for nefarious tampering?
>
> Thanks,
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
--
Yoav Shapira
Nimalex LLC
1 Mifflin Place, Suite 310
Cambridge, MA, USA
yoavs@computer.org / www.yoavshapira.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Binary build procedures
Posted by Henri Yandell <fl...@gmail.com>.
On 5/25/06, Mark Claassen <mc...@ocie.net> wrote:
> I have a question about two optional packages: mx4j and junit.
>
> Are they really optional? I realize that some build processes may use junit
> to test the build, but does it need to in the distribution?
Looking at the latest binary, junit doesn't get shipped.
> Also, what is mx4j used for. If I don't it, can I not use JMX?
connectors/jk/java/org/apache/jk/common/JkMX.java is the only class
using mx4j. Check its javadoc out to decide if you want to be
distributing mx4j in your Tomcat build.
Hen
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Binary build procedures
Posted by Mark Claassen <mc...@ocie.net>.
I have a question about two optional packages: mx4j and junit.
Are they really optional? I realize that some build processes may use junit
to test the build, but does it need to in the distribution?
Also, what is mx4j used for. If I don't it, can I not use JMX?
Thanks,
Mark
-----Original Message-----
From: Mark Claassen [mailto:mclaassen@ocie.net]
Sent: Thursday, May 25, 2006 10:04 AM
To: 'Tomcat Developers List'
Subject: RE: Binary build procedures
> Just out of curiosity, when you say above "if something goes wrong" do
> you
mean with your stuff or with the Tomcat build?
Here is an anecdote to describe what I mean.
While using 5.0, we had problems with the CGIServlet. It would not handle
uploads well at all. The problem was that the read was implemented with a
single 'read(buffer)' where buffer was the size of the content.
This works with small content lengths, but with larger ones it has problems.
The correct method is, of course, something like:
while (soFar < total)
read(buffer,soFar,total-soFar)
Anyway, I located the problem and fixed it. (I also submitted a bug report
and it was immediately added into the 5.5 builds.)
So, I believe the theory is that if we can build it from source, then we
know that we have what we need to attempt to track down something that comes
up.
I, personally, am a bit fearful of the whole build process. The binary
build of tomcat has been thoroughly tested, and I am going to try to
shoe-string one together as best I can. In the anecdote described above, I
actually started by downloading the source from the archives when we had the
problem (at that point I was allowed to just used the binary builds). I
then, instead of building a whole tomcat installation, made my own
CGIServlet based on the Tomcat CGIServlet code. But, it was just a lucky
break that the CGIServlet was just one file. (Heavier used pieces of Tomcat
are undoubtedly more thoroughly tested and probably much harder for someone
like me to patch.)
Mark
-----Original Message-----
From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf Of Yoav
Shapira
Sent: Wednesday, May 24, 2006 4:36 PM
To: Tomcat Developers List
Subject: Re: Binary build procedures
Hi,
> As it turns out there is a second motivation for building everything
> from source, and that is so if something goes wrong, we can find the
> problem and fix it. If we go through all the hoops now, then we won't
> have this additional problem when we are hurrying to fix and issue.
Just out of curiosity, when you say above "if something goes wrong" do you
mean with your stuff or with the Tomcat build?
> In light of this, I was wondering if there was a way to not include
> the eclipse stuff. Is there an option to use just the Sun JDK
> compiler? I doesn't look like that is an option in the build.xml
> file. However, even if it has to be included to make the build happy,
> is there a runtime configuration for this?
To be clear: when building Tomcat distributions, the Sun JDK compiler is
used. The Eclipse compiler is used only to compile JSPs at runtime on a
running Tomcat server. And yes, you can choose to use the Sun JDK compiler
at runtime instead of the Eclipse one: RTFM on the "compiler" option at
http://tomcat.apache.org/tomcat-5.5-doc/jasper-howto.html.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Binary build procedures
Posted by Mark Claassen <mc...@ocie.net>.
> Just out of curiosity, when you say above "if something goes wrong" do you
mean with your stuff or with the Tomcat build?
Here is an anecdote to describe what I mean.
While using 5.0, we had problems with the CGIServlet. It would not handle
uploads well at all. The problem was that the read was implemented with a
single 'read(buffer)' where buffer was the size of the content.
This works with small content lengths, but with larger ones it has problems.
The correct method is, of course, something like:
while (soFar < total)
read(buffer,soFar,total-soFar)
Anyway, I located the problem and fixed it. (I also submitted a bug report
and it was immediately added into the 5.5 builds.)
So, I believe the theory is that if we can build it from source, then we
know that we have what we need to attempt to track down something that comes
up.
I, personally, am a bit fearful of the whole build process. The binary
build of tomcat has been thoroughly tested, and I am going to try to
shoe-string one together as best I can. In the anecdote described above, I
actually started by downloading the source from the archives when we had the
problem (at that point I was allowed to just used the binary builds). I
then, instead of building a whole tomcat installation, made my own
CGIServlet based on the Tomcat CGIServlet code. But, it was just a lucky
break that the CGIServlet was just one file. (Heavier used pieces of Tomcat
are undoubtedly more thoroughly tested and probably much harder for someone
like me to patch.)
Mark
-----Original Message-----
From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf Of Yoav
Shapira
Sent: Wednesday, May 24, 2006 4:36 PM
To: Tomcat Developers List
Subject: Re: Binary build procedures
Hi,
> As it turns out there is a second motivation for building everything
> from source, and that is so if something goes wrong, we can find the
> problem and fix it. If we go through all the hoops now, then we won't
> have this additional problem when we are hurrying to fix and issue.
Just out of curiosity, when you say above "if something goes wrong" do you
mean with your stuff or with the Tomcat build?
> In light of this, I was wondering if there was a way to not include
> the eclipse stuff. Is there an option to use just the Sun JDK
> compiler? I doesn't look like that is an option in the build.xml
> file. However, even if it has to be included to make the build happy,
> is there a runtime configuration for this?
To be clear: when building Tomcat distributions, the Sun JDK compiler is
used. The Eclipse compiler is used only to compile JSPs at runtime on a
running Tomcat server. And yes, you can choose to use the Sun JDK compiler
at runtime instead of the Eclipse one: RTFM on the "compiler" option at
http://tomcat.apache.org/tomcat-5.5-doc/jasper-howto.html.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Binary build procedures
Posted by Yoav Shapira <yo...@apache.org>.
Hi,
> As it turns out there is a second motivation for building everything from
> source, and that is so if something goes wrong, we can find the problem and
> fix it. If we go through all the hoops now, then we won't have this
> additional problem when we are hurrying to fix and issue.
Just out of curiosity, when you say above "if something goes wrong" do
you mean with your stuff or with the Tomcat build?
> In light of this, I was wondering if there was a way to not include the
> eclipse stuff. Is there an option to use just the Sun JDK compiler? I
> doesn't look like that is an option in the build.xml file. However, even if
> it has to be included to make the build happy, is there a runtime
> configuration for this?
To be clear: when building Tomcat distributions, the Sun JDK compiler
is used. The Eclipse compiler is used only to compile JSPs at runtime
on a running Tomcat server. And yes, you can choose to use the Sun
JDK compiler at runtime instead of the Eclipse one: RTFM on the
"compiler" option at
http://tomcat.apache.org/tomcat-5.5-doc/jasper-howto.html.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Binary build procedures
Posted by Mark Claassen <mc...@ocie.net>.
Thanks again for the advice.
As it turns out there is a second motivation for building everything from
source, and that is so if something goes wrong, we can find the problem and
fix it. If we go through all the hoops now, then we won't have this
additional problem when we are hurrying to fix and issue.
In light of this, I was wondering if there was a way to not include the
eclipse stuff. Is there an option to use just the Sun JDK compiler? I
doesn't look like that is an option in the build.xml file. However, even if
it has to be included to make the build happy, is there a runtime
configuration for this?
Mark
-----Original Message-----
From: yoavshapira@gmail.com [mailto:yoavshapira@gmail.com] On Behalf Of Yoav
Shapira
Sent: Tuesday, May 23, 2006 5:14 PM
To: Tomcat Developers List
Subject: Re: Binary build procedures
Hola,
On 5/23/06, Mark Claassen <mc...@ocie.net> wrote:
> Thanks for your help. I am convinced, however I am not the one that
> needs convincing. Do you have any idea if other companies trust the
> binary builds? (Is this 'paranoia' unique to my company?)
It's not unique to your company, but in the minority AFAIK: in my
experience, almost every company I know that uses Tomcat uses the binary
builds. People tend to use the source builds only for customizations,
redistributions, etc.
> So I have all my ducks in a row, I have been looking into the src and
> binary distros a bit further, I noticed the tomcat-native library is
> included as source in the binary distro as a tar.gz file. Is this a
> necessary component of Tomcat? What is it for? The only platform
> specific files I found in the binary distro were the scripts and the
Windows startup executable.
The native connector is not a necessary component, it's optional.
RTFM it at http://tomcat.apache.org/tomcat-5.5-doc/apr.html.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Binary build procedures
Posted by Yoav Shapira <yo...@apache.org>.
Hola,
On 5/23/06, Mark Claassen <mc...@ocie.net> wrote:
> Thanks for your help. I am convinced, however I am not the one that needs
> convincing. Do you have any idea if other companies trust the binary
> builds? (Is this 'paranoia' unique to my company?)
It's not unique to your company, but in the minority AFAIK: in my
experience, almost every company I know that uses Tomcat uses the
binary builds. People tend to use the source builds only for
customizations, redistributions, etc.
> So I have all my ducks in a row, I have been looking into the src and binary
> distros a bit further, I noticed the tomcat-native library is included as
> source in the binary distro as a tar.gz file. Is this a necessary component
> of Tomcat? What is it for? The only platform specific files I found in the
> binary distro were the scripts and the Windows startup executable.
The native connector is not a necessary component, it's optional.
RTFM it at http://tomcat.apache.org/tomcat-5.5-doc/apr.html.
Yoav
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
RE: Binary build procedures
Posted by Mark Claassen <mc...@ocie.net>.
Thanks for your help. I am convinced, however I am not the one that needs
convincing. Do you have any idea if other companies trust the binary
builds? (Is this 'paranoia' unique to my company?)
So I have all my ducks in a row, I have been looking into the src and binary
distros a bit further, I noticed the tomcat-native library is included as
source in the binary distro as a tar.gz file. Is this a necessary component
of Tomcat? What is it for? The only platform specific files I found in the
binary distro were the scripts and the Windows startup executable.
Mark
-----Original Message-----
From: Tim Funk [mailto:funkman@joedog.org]
Sent: Tuesday, May 23, 2006 11:09 AM
To: Tomcat Developers List
Subject: Re: Binary build procedures
The release manager (RM)
- creates a binary from his copy of source.
- Generates a checksum key to allow validation of no tampering of the RM's
build.
The RM could insert malicious code into the build. If that were to happen -
the RM would probably be kicked out of the project in a hurry.
Its not valid to trust a source release download either. Its easy to tamper
with the source just as it is the binary. But having the source at this
point does allow for easy audits.
If you are really paranoid - build your binary from the appropriate TAG
would be safest since you are getting the original source - not repackaged
versions.
-Tim
Mark Claassen wrote:
> My boss has implemented some new procedures with regard to open source
> projects. He believes the source distributions are trustworthy, but
> he is not sure if he trusts the binary distributions. I think the
> reasoning is that he is uncertain if the binary distributions are
> controlled as well as the source ones are. And if they are not,
> someone could inject some malicious code to expose customer data or
something.
>
> Can someone give me a brief explanation on how the binary
> distributions are created for 5.5? Are the binary distributions
> created automatically from the repository, leaving no chance for nefarious
tampering?
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org For additional
commands, e-mail: dev-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Binary build procedures
Posted by Tim Funk <fu...@joedog.org>.
The release manager (RM)
- creates a binary from his copy of source.
- Generates a checksum key to allow validation of no tampering of the RM's build.
The RM could insert malicious code into the build. If that were to happen -
the RM would probably be kicked out of the project in a hurry.
Its not valid to trust a source release download either. Its easy to tamper
with the source just as it is the binary. But having the source at this point
does allow for easy audits.
If you are really paranoid - build your binary from the appropriate TAG would
be safest since you are getting the original source - not repackaged versions.
-Tim
Mark Claassen wrote:
> My boss has implemented some new procedures with regard to open source
> projects. He believes the source distributions are trustworthy, but he is
> not sure if he trusts the binary distributions. I think the reasoning is
> that he is uncertain if the binary distributions are controlled as well as
> the source ones are. And if they are not, someone could inject some
> malicious code to expose customer data or something.
>
> Can someone give me a brief explanation on how the binary distributions are
> created for 5.5? Are the binary distributions created automatically from
> the repository, leaving no chance for nefarious tampering?
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org