You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Steve Amerige <St...@sas.com> on 2015/07/28 14:26:00 UTC
Remove Version from File Names in Distributions; Add Manifest
Hi all,
Every time we take a download of the latest Groovy software, we have to
do the same task: remove version numbers from file names. As of the
2.4.4 release, there are 39 files, shown below, that have the version
number as part of the distribution. So, why is this a problem?
* IDEs cannot silently be updated to use a mandated Groovy version.
With 2.4.4 dealing with a zero-day vulnerability issue, we want to
push this out. However, the version numbers in files mean that
users must participate in the updating. This is not desirable.
* Links that users might have at the OS level are broken with each new
release.
* Version numbers in files makes it more difficult to diff between
releases.
* Version numbers as a part of a filename is a specific case of
metadata as part of file names and, as such, we consider it to be a
bad practice. This information is better kept in a file, preferably
machine readable in a format such as JSON or XML, or in manifest
files that can be consumed by software that might do version number
validation as part of security efforts in maintaining code.
It is reasonable that the root directory include a version number. But,
under that directory, we'd expect that the contents are version-less. A
good example of a version-less Apache project is the HTTP Server
<http://httpd.apache.org/download.cgi>. The download is presently a file
named *httpd-2.4.16.tar.gz*, and when extracted produces a top-level
directory named *httpd-2.4.16*. No file name contains the version number
string. The two files *CHANGES *and *httpd.spec *contain the version
number string. I believe that Groovy should follow this example, and
possibly go one step better by having an explicit manifest file with all
pertinent metadata for the Groovy release that includes metadata such as
the version number, license name, checksums of files (for security
checking), etc.
If you agree, how can we start the process of making this change?
Thanks,
Steve Amerige
Principal Software Developer, Fraud and Compliance Solutions Development
SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
./lib/groovy-sql-2.4.4.jar
./lib/groovy-testng-2.4.4.jar
./lib/groovy-jsr223-2.4.4.jar
./lib/groovy-servlet-2.4.4.jar
./lib/groovy-json-2.4.4.jar
./lib/groovy-jmx-2.4.4.jar
./lib/groovy-test-2.4.4.jar
./lib/groovy-bsf-2.4.4.jar
./lib/groovy-groovydoc-2.4.4.jar
./lib/groovy-nio-2.4.4.jar
./lib/groovy-console-2.4.4.jar
./lib/groovy-xml-2.4.4.jar
./lib/groovy-ant-2.4.4.jar
./lib/groovy-docgenerator-2.4.4.jar
./lib/groovy-groovysh-2.4.4.jar
./lib/groovy-templates-2.4.4.jar
./lib/groovy-swing-2.4.4.jar
./lib/groovy-2.4.4.jar
./apache-groovy-src-2.4.4-incubating.zip
./embeddable/groovy-all-2.4.4-indy.jar
./embeddable/groovy-all-2.4.4.jar
./indy/groovy-json-2.4.4-indy.jar
./indy/groovy-console-2.4.4-indy.jar
./indy/groovy-2.4.4-indy.jar
./indy/groovy-sql-2.4.4-indy.jar
./indy/groovy-jmx-2.4.4-indy.jar
./indy/groovy-servlet-2.4.4-indy.jar
./indy/groovy-xml-2.4.4-indy.jar
./indy/groovy-swing-2.4.4-indy.jar
./indy/groovy-templates-2.4.4-indy.jar
./indy/groovy-ant-2.4.4-indy.jar
./indy/groovy-groovydoc-2.4.4-indy.jar
./indy/groovy-nio-2.4.4-indy.jar
./indy/groovy-test-2.4.4-indy.jar
./indy/groovy-testng-2.4.4-indy.jar
./indy/groovy-groovysh-2.4.4-indy.jar
./indy/groovy-docgenerator-2.4.4-indy.jar
./indy/groovy-bsf-2.4.4-indy.jar
./indy/groovy-jsr223-2.4.4-indy.jar
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Mauro Molinari <ma...@tiscali.it>.
Il 29/07/2015 13:52, Steve Amerige ha scritto:
> Some issues that large enterprises typically face include that dynamic
> tools that resolve dependencies are not permitted. There are many
> reasons for this: everything from legal (third-party jars, including
> version changes, must be approved), to having centralized control of
> which jars engineers have access to for their particular product and
> for their particular intended release event. And, developers tend to
> use either Eclipse or IntelliJ IDEs, while some few do everything
> command line with environment variables or scripts helping compilation
> to find jars. Developer environments need to include the specific
> network-accessible jars they need. These jars are organized into
> groups that are approved and are consistent for a particular release
> event.
I don't think any of these reasons prevents the use of a build tool. For
instance, having an internal repository server like Nexus or Artifactory
would allow to control exactly which dependencies are made available to
developers.
> Fortunately, we have excellent build and release engineers who take
> third-party jars and fit them into our system so that all concerns are
> addressed. Since it doesn't seem that our needs are shared by others,
> it's just fine to leave it at that.
What it seems like to me is that these build and release engineers are
just reinventing the wheel or probably that the environment they work in
predates the modern build tools like Gradle or Maven.
In my experience, I saw "big" enterprises manage things in insane ways
(like environments where deployments consist of the unpacking a ZIP file
- yes, NO way to delete obsolete files unless you contact a support team
that MANUALLY performs these kind of operations!!! Another way to say
that you can't perform refactoring on classes!!) in the name of
"security" or "control". Just because people do crazy things it doesn't
mean that they should be supported :-)
We may discuss about the fact that version number of file names might be
redundant (once we have manifests), but it's also true that most (if not
all) package management system I know of seem to go in this direction.
Apart from Maven repositories, OSGi does the same, as well as Linux
packages (deb and rpm files have versions in their names).
Just yet another 2 cents by one that doesn't miss the "JAR hell" of some
years ago.
Mauro
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Steve Amerige <St...@sas.com>.
On 7/29/2015 1:41 AM, Jochen Theodorou wrote:
> But Steve, I do have a question...
> Normally the things you describe are part of the installation process
> as well as making a package for the system installation. That means
> normally you want a system package. Installing software packages
> manually in a system with a packaging manager is usually a bad idea.
>
> GVM was mentioned, but GVM is not for a system-wide installation like
> Steve wants it. And it is (currently) not a way to provide libraries
> for usage by applications outside of a GVM package.
Some issues that large enterprises typically face include that dynamic
tools that resolve dependencies are not permitted. There are many
reasons for this: everything from legal (third-party jars, including
version changes, must be approved), to having centralized control of
which jars engineers have access to for their particular product and for
their particular intended release event. And, developers tend to use
either Eclipse or IntelliJ IDEs, while some few do everything command
line with environment variables or scripts helping compilation to find
jars. Developer environments need to include the specific
network-accessible jars they need. These jars are organized into groups
that are approved and are consistent for a particular release event.
Having release numbers on the jars themselves would cause problems in
which developer environments would need to be reconfigured when jar
updates are mandated. What seems reasonable for small to medium
projects sometimes isn't appropriate for very large scale projects
involving thousands of developers.
Fortunately, we have excellent build and release engineers who take
third-party jars and fit them into our system so that all concerns are
addressed. Since it doesn't seem that our needs are shared by others,
it's just fine to leave it at that.
Enjoy,
Steve Amerige
Principal Software Developer, Fraud and Compliance Solutions Development
SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Guillaume Laforge <gl...@gmail.com>.
This sounds like an anti-pattern to me to remove version names of the JARs.
In a distant past when we were storing version-less JAR files in CVS, it
was a real hell :-)
That's build tools responsibilities to bring in the right artifacts in the
build deliverables.
On Wed, Jul 29, 2015 at 7:41 AM, Jochen Theodorou <bl...@gmx.org> wrote:
> Am 28.07.2015 14:26, schrieb Steve Amerige:
>
>> Hi all,
>>
>> Every time we take a download of the latest Groovy software, we have to
>> do the same task: remove version numbers from file names. As of the
>> 2.4.4 release, there are 39 files, shown below, that have the version
>> number as part of the distribution.
>>
>
> first of all, I got the impression that the discussion so far was a tiny
> bit misguided. We have the zip distribution and the jars that go to maven.
> The maven artifacts of course would stay as they are, this is only about
> the zip distribution.
>
> [...]
>
>> It is reasonable that the root directory include a version number. But,
>> under that directory, we'd expect that the contents are version-less.
>>
>
> The reason that we have the version numbers there basically is, that the
> jars in the zip are the same jars as for maven, that's why they have
> version numbers. It was not all that of a big deal before we started having
> 17+ modules.
>
> [...]
>
>> I believe that Groovy should follow this example, and
>> possibly go one step better by having an explicit manifest file with all
>> pertinent metadata for the Groovy release that includes metadata such as
>> the version number, license name, checksums of files (for security
>> checking), etc.
>>
>
> license information must go into the LICENSE and NOTICE (and DISCLAIMER)
> files as per ASF requirements. Checksums are imho not making sense as part
> of the zip, because if the zip is manipulated, then it is no use to use the
> zip contents to verify the zip contents. As for the version number... I
> think a file VERSION is often used for that.
>
> But Steve, I do have a question...
> Normally the things you describe are part of the installation process as
> well as making a package for the system installation. That means normally
> you want a system package. Installing software packages manually in a
> system with a packaging manager is usually a bad idea.
>
> GVM was mentioned, but GVM is not for a system-wide installation like
> Steve wants it. And it is (currently) not a way to provide libraries for
> usage by applications outside of a GVM package.
>
> As for how.... changing distSpec in assemble.groovy to include some
> renames may do it. Anyone interested in doing a pull request? Of course I
> would also like to hear Cedric as our current build master about this.
>
> bye blackdrag
>
> --
> Jochen "blackdrag" Theodorou
> blog: http://blackdragsview.blogspot.com/
>
>
--
Guillaume Laforge
Apache Groovy committer & PMC member
Product Ninja & Advocate at Restlet <http://restlet.com>
Blog: http://glaforge.appspot.com/
Social: @glaforge <http://twitter.com/glaforge> / Google+
<https://plus.google.com/u/0/114130972232398734985/posts>
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Jochen Theodorou <bl...@gmx.org>.
Am 28.07.2015 14:26, schrieb Steve Amerige:
> Hi all,
>
> Every time we take a download of the latest Groovy software, we have to
> do the same task: remove version numbers from file names. As of the
> 2.4.4 release, there are 39 files, shown below, that have the version
> number as part of the distribution.
first of all, I got the impression that the discussion so far was a tiny
bit misguided. We have the zip distribution and the jars that go to
maven. The maven artifacts of course would stay as they are, this is
only about the zip distribution.
[...]
> It is reasonable that the root directory include a version number. But,
> under that directory, we'd expect that the contents are version-less.
The reason that we have the version numbers there basically is, that the
jars in the zip are the same jars as for maven, that's why they have
version numbers. It was not all that of a big deal before we started
having 17+ modules.
[...]
> I believe that Groovy should follow this example, and
> possibly go one step better by having an explicit manifest file with all
> pertinent metadata for the Groovy release that includes metadata such as
> the version number, license name, checksums of files (for security
> checking), etc.
license information must go into the LICENSE and NOTICE (and DISCLAIMER)
files as per ASF requirements. Checksums are imho not making sense as
part of the zip, because if the zip is manipulated, then it is no use to
use the zip contents to verify the zip contents. As for the version
number... I think a file VERSION is often used for that.
But Steve, I do have a question...
Normally the things you describe are part of the installation process as
well as making a package for the system installation. That means
normally you want a system package. Installing software packages
manually in a system with a packaging manager is usually a bad idea.
GVM was mentioned, but GVM is not for a system-wide installation like
Steve wants it. And it is (currently) not a way to provide libraries for
usage by applications outside of a GVM package.
As for how.... changing distSpec in assemble.groovy to include some
renames may do it. Anyone interested in doing a pull request? Of course
I would also like to hear Cedric as our current build master about this.
bye blackdrag
--
Jochen "blackdrag" Theodorou
blog: http://blackdragsview.blogspot.com/
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Keegan Witt <ke...@gmail.com>.
The Tomcat example was the first thing that came to my mind. What is your
use case that makes this problematic? If symlinks are the main issue, why
not use your distribution's package manager instead?
-Keegan
On Tue, Jul 28, 2015 at 10:38 AM, Henrik Martin <he...@netgate.net> wrote:
> I'm not part of the contributor team, so I can't speak for the Groovy
> team, but I would strongly disagree with you. If you use Maven or Gradle,
> it's easy to maintain dependencies on particular versions of jar files, and
> have your IDE immediately pick up the new version. In fact, the default
> behavior for both Maven and Gradle is to explicitly maintain version
> numbers in artifacts. Removing this would be a step back to the 90s.
> Sometimes jar files have to copied into other directories outside of their
> normal home. An example is when deploying Tomcat. Stuff like jdbc drivers
> etc typically get copied into $CATALINA_BASE/lib. It's worth gold to
> immediately be able to tell which particular version of those jar files are
> in there, vs just seeing "foobar.jar".
>
> I would argue that you should probably change the practice of creating
> symlinks to explicitly versioned jar files as this is obviously a pain when
> new versions are introduced.
>
> Just my $0.02.
>
> -H
>
>
> On 7/28/15 5:26 AM, Steve Amerige wrote:
>
> Hi all,
>
> Every time we take a download of the latest Groovy software, we have to do
> the same task: remove version numbers from file names. As of the 2.4.4
> release, there are 39 files, shown below, that have the version number as
> part of the distribution. So, why is this a problem?
>
> - IDEs cannot silently be updated to use a mandated Groovy version.
> With 2.4.4 dealing with a zero-day vulnerability issue, we want to push
> this out. However, the version numbers in files mean that users must
> participate in the updating. This is not desirable.
> - Links that users might have at the OS level are broken with each new
> release.
> - Version numbers in files makes it more difficult to diff between
> releases.
> - Version numbers as a part of a filename is a specific case of
> metadata as part of file names and, as such, we consider it to be a bad
> practice. This information is better kept in a file, preferably machine
> readable in a format such as JSON or XML, or in manifest files that can be
> consumed by software that might do version number validation as part of
> security efforts in maintaining code.
>
> It is reasonable that the root directory include a version number. But,
> under that directory, we'd expect that the contents are version-less. A
> good example of a version-less Apache project is the HTTP Server
> <http://httpd.apache.org/download.cgi>. The download is presently a file
> named *httpd-2.4.16.tar.gz*, and when extracted produces a top-level
> directory named *httpd-2.4.16*. No file name contains the version number
> string. The two files *CHANGES *and *httpd.spec *contain the version
> number string. I believe that Groovy should follow this example, and
> possibly go one step better by having an explicit manifest file with all
> pertinent metadata for the Groovy release that includes metadata such as
> the version number, license name, checksums of files (for security
> checking), etc.
>
> If you agree, how can we start the process of making this change?
>
> Thanks,
> Steve Amerige
> Principal Software Developer, Fraud and Compliance Solutions Development
> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>
>
>
> ./lib/groovy-sql-2.4.4.jar
> ./lib/groovy-testng-2.4.4.jar
> ./lib/groovy-jsr223-2.4.4.jar
> ./lib/groovy-servlet-2.4.4.jar
> ./lib/groovy-json-2.4.4.jar
> ./lib/groovy-jmx-2.4.4.jar
> ./lib/groovy-test-2.4.4.jar
> ./lib/groovy-bsf-2.4.4.jar
> ./lib/groovy-groovydoc-2.4.4.jar
> ./lib/groovy-nio-2.4.4.jar
> ./lib/groovy-console-2.4.4.jar
> ./lib/groovy-xml-2.4.4.jar
> ./lib/groovy-ant-2.4.4.jar
> ./lib/groovy-docgenerator-2.4.4.jar
> ./lib/groovy-groovysh-2.4.4.jar
> ./lib/groovy-templates-2.4.4.jar
> ./lib/groovy-swing-2.4.4.jar
> ./lib/groovy-2.4.4.jar
> ./apache-groovy-src-2.4.4-incubating.zip
> ./embeddable/groovy-all-2.4.4-indy.jar
> ./embeddable/groovy-all-2.4.4.jar
> ./indy/groovy-json-2.4.4-indy.jar
> ./indy/groovy-console-2.4.4-indy.jar
> ./indy/groovy-2.4.4-indy.jar
> ./indy/groovy-sql-2.4.4-indy.jar
> ./indy/groovy-jmx-2.4.4-indy.jar
> ./indy/groovy-servlet-2.4.4-indy.jar
> ./indy/groovy-xml-2.4.4-indy.jar
> ./indy/groovy-swing-2.4.4-indy.jar
> ./indy/groovy-templates-2.4.4-indy.jar
> ./indy/groovy-ant-2.4.4-indy.jar
> ./indy/groovy-groovydoc-2.4.4-indy.jar
> ./indy/groovy-nio-2.4.4-indy.jar
> ./indy/groovy-test-2.4.4-indy.jar
> ./indy/groovy-testng-2.4.4-indy.jar
> ./indy/groovy-groovysh-2.4.4-indy.jar
> ./indy/groovy-docgenerator-2.4.4-indy.jar
> ./indy/groovy-bsf-2.4.4-indy.jar
> ./indy/groovy-jsr223-2.4.4-indy.jar
>
>
>
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Jeff Adamson <jw...@gmail.com>.
The embeddable jars are covertly named ;-)
./embeddable/groovy-all-2.4.4-indy.jar
./embeddable/groovy-all-2.4.4.jar
They aren't used (as far as I know) by groovy_home and the intention of
those is to rip them out of the install and copy them around. That sets a
different bar for me for expected usage patterns. The groovy_home/lib
directory should not have its jars pillaged like that.
I don't buy your second paragraph because it basically comes down to: if
you use a maven dependency system then using maven dependencies is easy.
That feels like the next best thing to a tooling-tautology. If the names
hadn't been munged with versions, then Steve's example wouldn't be needing
to strip them out or symlink or any of those hoops. It would be just as
simple and robust as the idealized result that you need to use maven to
achieve.
If the only argument against referencing the manifest is the tooling, it
seems more that it is so easy that elaborate tools haven't caught on.
Unzipping is about the crudest way you could accomplish that, but also
incredibly obvious. More refined exist very easily via google/stack
overflow/etc: JDGUI, Jar Inspector, Jarzilla, and even if you had to do it
from scratch one could just wrap `unzip -q -c "$1" META-INF/MANIFEST.MF`.
But that is getting a bit off topic.
In Steve's case of httpd, the archive had the version number, not every
file within the archive. Appealing to generalities like "Most archives" is
an argument from popularity and additionally it is obvious we just travel
in different circles. I do have exposure to more than just java+maven
platforms.
Perhaps, my real take question/point is that if the groovy jars in a
groovy-2.4.4/lib directory are not all groovy-2.4.4, there is something
extremely bad already happening and abandon yea all hope. Having the
"feature" of multiple jars of same name with different versions yields
undefined behavior and and would rather just have one jar to verify than
things working inconsistently.
The version number for those files is either redundant or should be.
--Jeff
On Tue, Jul 28, 2015 at 5:04 PM, Henrik Martin <he...@netgate.net> wrote:
>
> On 7/28/15 8:55 AM, Jeff Adamson wrote:
>
> My non-contributor $0.02 would be exactly the opposite Henrik, for very
> similar reasons.
>
> I would agree that the embeddable jar should include version number by
> default because that is supposed to be dropped around, but the rest of the
> lib is already version locked to a groovy version. If someone is mucking
> around with the directory contents of the groovy lib, they already have
> made a bad decision causing drift from a traceable setup.
>
> Not sure what you mean by the "embeddable jar", but it sounds to me like
> you're arguing for having two different conventions then. Some jar files
> with version numbers, and some without. I'd rather have consistency.
> Virtually every jar file in existence these days is built by either Maven,
> Gradle, or some similar build/dependency management tools. Since the
> default behavior of these tools is to version the archives, it seems
> counter productive to move away from that. Like it or not, but it has
> become the industry standard.
>
> The very argument of the maven-gradle shows how having version numbers
> adds a burden to other processes. Now the build-system and non-maven
> consumers of the jars need specific specialized (but common) logic to do
> something that has negligible benefit. Do you trust your dependency system
> to have pulled the correct versions or not? If you don't, then you should
> look into other dependency management tools.
>
> I would argue that in Steve's use case, they've added the burden by not
> utilizing the build system to do the dependency management. It seems a lot
> easier and a lot less error prone, to simply have your build system
> download the correct Groovy version and ensure that the build depends on
> that version only, rather than pushing out zip archives, unzipping them,
> and then having scripts deal with stripping out the version numbers from
> the jars. That seems like a much more brittle and error prone system than
> simply using something like Gradle. In the most recent environments I've
> worked, we've used Gradle to ensure that all developers depend on the same
> versions of all build files, whether or not we're building from the command
> line or from within the IDE. No need for symlinks (which aren't that
> portable anyway) or renaming of anything. Our developers use various
> versions of Windows, MacOS, and Linux. It would be a much harder task to
> follow Steve's practice of managing symlinks across several different types
> of OS'es and numerous flavors of those.
>
> The authoritative place for version meta-data is in the jar file, not
> it's name; there even is a standard place for this MANIFEST.MF.
>
> The problem with the manifest is that in order to read it, you have to
> unzip the archive. How many tools out there actually utilize that feature
> in reality? A jar file is an archive, not much different from any other
> archive type like tar, etc. Most archives contain the version information
> in the file name.
>
> I remember the 90's being the era when the fashion was to jam every bit
> of meta-data was jammed in file names (see cvs or tarball versioning), not
> the other way around. Maven is from 2001, therefore it seems more correct
> to attribute version numbering file names to be a hold-over from the 90s.
> --Jeff
>
> I don't remember seeing much of jar file naming with version information
> from the 90s, but hey, it's been a while. Maybe my memory is failing me :-)
>
> -H
>
>
> On Tue, Jul 28, 2015 at 10:38 AM, Henrik Martin < <he...@netgate.net>
> henrik@netgate.net> wrote:
>
>> I'm not part of the contributor team, so I can't speak for the Groovy
>> team, but I would strongly disagree with you. If you use Maven or Gradle,
>> it's easy to maintain dependencies on particular versions of jar files, and
>> have your IDE immediately pick up the new version. In fact, the default
>> behavior for both Maven and Gradle is to explicitly maintain version
>> numbers in artifacts. Removing this would be a step back to the 90s.
>> Sometimes jar files have to copied into other directories outside of their
>> normal home. An example is when deploying Tomcat. Stuff like jdbc drivers
>> etc typically get copied into $CATALINA_BASE/lib. It's worth gold to
>> immediately be able to tell which particular version of those jar files are
>> in there, vs just seeing "foobar.jar".
>>
>> I would argue that you should probably change the practice of creating
>> symlinks to explicitly versioned jar files as this is obviously a pain when
>> new versions are introduced.
>>
>> Just my $0.02.
>>
>> -H
>>
>>
>> On 7/28/15 5:26 AM, Steve Amerige wrote:
>>
>> Hi all,
>>
>> Every time we take a download of the latest Groovy software, we have to
>> do the same task: remove version numbers from file names. As of the 2.4.4
>> release, there are 39 files, shown below, that have the version number as
>> part of the distribution. So, why is this a problem?
>>
>> - IDEs cannot silently be updated to use a mandated Groovy version.
>> With 2.4.4 dealing with a zero-day vulnerability issue, we want to push
>> this out. However, the version numbers in files mean that users must
>> participate in the updating. This is not desirable.
>> - Links that users might have at the OS level are broken with each
>> new release.
>> - Version numbers in files makes it more difficult to diff between
>> releases.
>> - Version numbers as a part of a filename is a specific case of
>> metadata as part of file names and, as such, we consider it to be a bad
>> practice. This information is better kept in a file, preferably machine
>> readable in a format such as JSON or XML, or in manifest files that can be
>> consumed by software that might do version number validation as part of
>> security efforts in maintaining code.
>>
>> It is reasonable that the root directory include a version number.
>> But, under that directory, we'd expect that the contents are version-less.
>> A good example of a version-less Apache project is the HTTP Server
>> <http://httpd.apache.org/download.cgi>. The download is presently a file
>> named *httpd-2.4.16.tar.gz*, and when extracted produces a top-level
>> directory named *httpd-2.4.16*. No file name contains the version
>> number string. The two files *CHANGES *and *httpd.spec *contain the
>> version number string. I believe that Groovy should follow this example,
>> and possibly go one step better by having an explicit manifest file with
>> all pertinent metadata for the Groovy release that includes metadata such
>> as the version number, license name, checksums of files (for security
>> checking), etc.
>>
>> If you agree, how can we start the process of making this change?
>>
>> Thanks,
>> Steve Amerige
>> Principal Software Developer, Fraud and Compliance Solutions Development
>> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>>
>>
>>
>> ./lib/groovy-sql-2.4.4.jar
>> ./lib/groovy-testng-2.4.4.jar
>> ./lib/groovy-jsr223-2.4.4.jar
>> ./lib/groovy-servlet-2.4.4.jar
>> ./lib/groovy-json-2.4.4.jar
>> ./lib/groovy-jmx-2.4.4.jar
>> ./lib/groovy-test-2.4.4.jar
>> ./lib/groovy-bsf-2.4.4.jar
>> ./lib/groovy-groovydoc-2.4.4.jar
>> ./lib/groovy-nio-2.4.4.jar
>> ./lib/groovy-console-2.4.4.jar
>> ./lib/groovy-xml-2.4.4.jar
>> ./lib/groovy-ant-2.4.4.jar
>> ./lib/groovy-docgenerator-2.4.4.jar
>> ./lib/groovy-groovysh-2.4.4.jar
>> ./lib/groovy-templates-2.4.4.jar
>> ./lib/groovy-swing-2.4.4.jar
>> ./lib/groovy-2.4.4.jar
>> ./apache-groovy-src-2.4.4-incubating.zip
>> ./embeddable/groovy-all-2.4.4-indy.jar
>> ./embeddable/groovy-all-2.4.4.jar
>> ./indy/groovy-json-2.4.4-indy.jar
>> ./indy/groovy-console-2.4.4-indy.jar
>> ./indy/groovy-2.4.4-indy.jar
>> ./indy/groovy-sql-2.4.4-indy.jar
>> ./indy/groovy-jmx-2.4.4-indy.jar
>> ./indy/groovy-servlet-2.4.4-indy.jar
>> ./indy/groovy-xml-2.4.4-indy.jar
>> ./indy/groovy-swing-2.4.4-indy.jar
>> ./indy/groovy-templates-2.4.4-indy.jar
>> ./indy/groovy-ant-2.4.4-indy.jar
>> ./indy/groovy-groovydoc-2.4.4-indy.jar
>> ./indy/groovy-nio-2.4.4-indy.jar
>> ./indy/groovy-test-2.4.4-indy.jar
>> ./indy/groovy-testng-2.4.4-indy.jar
>> ./indy/groovy-groovysh-2.4.4-indy.jar
>> ./indy/groovy-docgenerator-2.4.4-indy.jar
>> ./indy/groovy-bsf-2.4.4-indy.jar
>> ./indy/groovy-jsr223-2.4.4-indy.jar
>>
>>
>>
>
>
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Henrik Martin <he...@netgate.net>.
On 7/28/15 8:55 AM, Jeff Adamson wrote:
> My non-contributor $0.02 would be exactly the opposite Henrik, for
> very similar reasons.
>
> I would agree that the embeddable jar should include version number by
> default because that is supposed to be dropped around, but the rest of
> the lib is already version locked to a groovy version. If someone is
> mucking around with the directory contents of the groovy lib, they
> already have made a bad decision causing drift from a traceable setup.
Not sure what you mean by the "embeddable jar", but it sounds to me like
you're arguing for having two different conventions then. Some jar files
with version numbers, and some without. I'd rather have consistency.
Virtually every jar file in existence these days is built by either
Maven, Gradle, or some similar build/dependency management tools. Since
the default behavior of these tools is to version the archives, it seems
counter productive to move away from that. Like it or not, but it has
become the industry standard.
> The very argument of the maven-gradle shows how having version numbers
> adds a burden to other processes. Now the build-system and non-maven
> consumers of the jars need specific specialized (but common) logic to
> do something that has negligible benefit. Do you trust your dependency
> system to have pulled the correct versions or not? If you don't, then
> you should look into other dependency management tools.
I would argue that in Steve's use case, they've added the burden by not
utilizing the build system to do the dependency management. It seems a
lot easier and a lot less error prone, to simply have your build system
download the correct Groovy version and ensure that the build depends on
that version only, rather than pushing out zip archives, unzipping them,
and then having scripts deal with stripping out the version numbers from
the jars. That seems like a much more brittle and error prone system
than simply using something like Gradle. In the most recent environments
I've worked, we've used Gradle to ensure that all developers depend on
the same versions of all build files, whether or not we're building from
the command line or from within the IDE. No need for symlinks (which
aren't that portable anyway) or renaming of anything. Our developers use
various versions of Windows, MacOS, and Linux. It would be a much harder
task to follow Steve's practice of managing symlinks across several
different types of OS'es and numerous flavors of those.
> The authoritative place for version meta-data is in the jar file, not
> it's name; there even is a standard place for this MANIFEST.MF.
The problem with the manifest is that in order to read it, you have to
unzip the archive. How many tools out there actually utilize that
feature in reality? A jar file is an archive, not much different from
any other archive type like tar, etc. Most archives contain the version
information in the file name.
> I remember the 90's being the era when the fashion was to jam every
> bit of meta-data was jammed in file names (see cvs or tarball
> versioning), not the other way around. Maven is from 2001, therefore
> it seems more correct to attribute version numbering file names to be
> a hold-over from the 90s.
> --Jeff
I don't remember seeing much of jar file naming with version information
from the 90s, but hey, it's been a while. Maybe my memory is failing me :-)
-H
>
> On Tue, Jul 28, 2015 at 10:38 AM, Henrik Martin <henrik@netgate.net
> <ma...@netgate.net>> wrote:
>
> I'm not part of the contributor team, so I can't speak for the
> Groovy team, but I would strongly disagree with you. If you use
> Maven or Gradle, it's easy to maintain dependencies on particular
> versions of jar files, and have your IDE immediately pick up the
> new version. In fact, the default behavior for both Maven and
> Gradle is to explicitly maintain version numbers in artifacts.
> Removing this would be a step back to the 90s. Sometimes jar files
> have to copied into other directories outside of their normal
> home. An example is when deploying Tomcat. Stuff like jdbc drivers
> etc typically get copied into $CATALINA_BASE/lib. It's worth gold
> to immediately be able to tell which particular version of those
> jar files are in there, vs just seeing "foobar.jar".
>
> I would argue that you should probably change the practice of
> creating symlinks to explicitly versioned jar files as this is
> obviously a pain when new versions are introduced.
>
> Just my $0.02.
>
> -H
>
>
> On 7/28/15 5:26 AM, Steve Amerige wrote:
>> Hi all,
>>
>> Every time we take a download of the latest Groovy software, we
>> have to do the same task: remove version numbers from file
>> names. As of the 2.4.4 release, there are 39 files, shown below,
>> that have the version number as part of the distribution. So,
>> why is this a problem?
>>
>> * IDEs cannot silently be updated to use a mandated Groovy
>> version. With 2.4.4 dealing with a zero-day vulnerability
>> issue, we want to push this out. However, the version
>> numbers in files mean that users must participate in the
>> updating. This is not desirable.
>> * Links that users might have at the OS level are broken with
>> each new release.
>> * Version numbers in files makes it more difficult to diff
>> between releases.
>> * Version numbers as a part of a filename is a specific case of
>> metadata as part of file names and, as such, we consider it
>> to be a bad practice. This information is better kept in a
>> file, preferably machine readable in a format such as JSON or
>> XML, or in manifest files that can be consumed by software
>> that might do version number validation as part of security
>> efforts in maintaining code.
>>
>> It is reasonable that the root directory include a version
>> number. But, under that directory, we'd expect that the contents
>> are version-less. A good example of a version-less Apache project
>> is the HTTP Server <http://httpd.apache.org/download.cgi>. The
>> download is presently a file named *httpd-2.4.16.tar.gz*, and
>> when extracted produces a top-level directory named
>> *httpd-2.4.16*. No file name contains the version number
>> string. The two files *CHANGES *and *httpd.spec *contain the
>> version number string. I believe that Groovy should follow this
>> example, and possibly go one step better by having an explicit
>> manifest file with all pertinent metadata for the Groovy release
>> that includes metadata such as the version number, license name,
>> checksums of files (for security checking), etc.
>>
>> If you agree, how can we start the process of making this change?
>>
>> Thanks,
>> Steve Amerige
>> Principal Software Developer, Fraud and Compliance Solutions
>> Development
>> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>>
>>
>>
>> ./lib/groovy-sql-2.4.4.jar
>> ./lib/groovy-testng-2.4.4.jar
>> ./lib/groovy-jsr223-2.4.4.jar
>> ./lib/groovy-servlet-2.4.4.jar
>> ./lib/groovy-json-2.4.4.jar
>> ./lib/groovy-jmx-2.4.4.jar
>> ./lib/groovy-test-2.4.4.jar
>> ./lib/groovy-bsf-2.4.4.jar
>> ./lib/groovy-groovydoc-2.4.4.jar
>> ./lib/groovy-nio-2.4.4.jar
>> ./lib/groovy-console-2.4.4.jar
>> ./lib/groovy-xml-2.4.4.jar
>> ./lib/groovy-ant-2.4.4.jar
>> ./lib/groovy-docgenerator-2.4.4.jar
>> ./lib/groovy-groovysh-2.4.4.jar
>> ./lib/groovy-templates-2.4.4.jar
>> ./lib/groovy-swing-2.4.4.jar
>> ./lib/groovy-2.4.4.jar
>> ./apache-groovy-src-2.4.4-incubating.zip
>> ./embeddable/groovy-all-2.4.4-indy.jar
>> ./embeddable/groovy-all-2.4.4.jar
>> ./indy/groovy-json-2.4.4-indy.jar
>> ./indy/groovy-console-2.4.4-indy.jar
>> ./indy/groovy-2.4.4-indy.jar
>> ./indy/groovy-sql-2.4.4-indy.jar
>> ./indy/groovy-jmx-2.4.4-indy.jar
>> ./indy/groovy-servlet-2.4.4-indy.jar
>> ./indy/groovy-xml-2.4.4-indy.jar
>> ./indy/groovy-swing-2.4.4-indy.jar
>> ./indy/groovy-templates-2.4.4-indy.jar
>> ./indy/groovy-ant-2.4.4-indy.jar
>> ./indy/groovy-groovydoc-2.4.4-indy.jar
>> ./indy/groovy-nio-2.4.4-indy.jar
>> ./indy/groovy-test-2.4.4-indy.jar
>> ./indy/groovy-testng-2.4.4-indy.jar
>> ./indy/groovy-groovysh-2.4.4-indy.jar
>> ./indy/groovy-docgenerator-2.4.4-indy.jar
>> ./indy/groovy-bsf-2.4.4-indy.jar
>> ./indy/groovy-jsr223-2.4.4-indy.jar
>>
>
>
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Jeff Adamson <jw...@gmail.com>.
My non-contributor $0.02 would be exactly the opposite Henrik, for very
similar reasons.
I would agree that the embeddable jar should include version number by
default because that is supposed to be dropped around, but the rest of the
lib is already version locked to a groovy version. If someone is mucking
around with the directory contents of the groovy lib, they already have
made a bad decision causing drift from a traceable setup.
The very argument of the maven-gradle shows how having version numbers adds
a burden to other processes. Now the build-system and non-maven consumers
of the jars need specific specialized (but common) logic to do something
that has negligible benefit. Do you trust your dependency system to have
pulled the correct versions or not? If you don't, then you should look into
other dependency management tools.
The authoritative place for version meta-data is in the jar file, not it's
name; there even is a standard place for this MANIFEST.MF.
I remember the 90's being the era when the fashion was to jam every bit of
meta-data was jammed in file names (see cvs or tarball versioning), not the
other way around. Maven is from 2001, therefore it seems more correct to
attribute version numbering file names to be a hold-over from the 90s.
--Jeff
On Tue, Jul 28, 2015 at 10:38 AM, Henrik Martin <he...@netgate.net> wrote:
> I'm not part of the contributor team, so I can't speak for the Groovy
> team, but I would strongly disagree with you. If you use Maven or Gradle,
> it's easy to maintain dependencies on particular versions of jar files, and
> have your IDE immediately pick up the new version. In fact, the default
> behavior for both Maven and Gradle is to explicitly maintain version
> numbers in artifacts. Removing this would be a step back to the 90s.
> Sometimes jar files have to copied into other directories outside of their
> normal home. An example is when deploying Tomcat. Stuff like jdbc drivers
> etc typically get copied into $CATALINA_BASE/lib. It's worth gold to
> immediately be able to tell which particular version of those jar files are
> in there, vs just seeing "foobar.jar".
>
> I would argue that you should probably change the practice of creating
> symlinks to explicitly versioned jar files as this is obviously a pain when
> new versions are introduced.
>
> Just my $0.02.
>
> -H
>
>
> On 7/28/15 5:26 AM, Steve Amerige wrote:
>
> Hi all,
>
> Every time we take a download of the latest Groovy software, we have to do
> the same task: remove version numbers from file names. As of the 2.4.4
> release, there are 39 files, shown below, that have the version number as
> part of the distribution. So, why is this a problem?
>
> - IDEs cannot silently be updated to use a mandated Groovy version.
> With 2.4.4 dealing with a zero-day vulnerability issue, we want to push
> this out. However, the version numbers in files mean that users must
> participate in the updating. This is not desirable.
> - Links that users might have at the OS level are broken with each new
> release.
> - Version numbers in files makes it more difficult to diff between
> releases.
> - Version numbers as a part of a filename is a specific case of
> metadata as part of file names and, as such, we consider it to be a bad
> practice. This information is better kept in a file, preferably machine
> readable in a format such as JSON or XML, or in manifest files that can be
> consumed by software that might do version number validation as part of
> security efforts in maintaining code.
>
> It is reasonable that the root directory include a version number. But,
> under that directory, we'd expect that the contents are version-less. A
> good example of a version-less Apache project is the HTTP Server
> <http://httpd.apache.org/download.cgi>. The download is presently a file
> named *httpd-2.4.16.tar.gz*, and when extracted produces a top-level
> directory named *httpd-2.4.16*. No file name contains the version number
> string. The two files *CHANGES *and *httpd.spec *contain the version
> number string. I believe that Groovy should follow this example, and
> possibly go one step better by having an explicit manifest file with all
> pertinent metadata for the Groovy release that includes metadata such as
> the version number, license name, checksums of files (for security
> checking), etc.
>
> If you agree, how can we start the process of making this change?
>
> Thanks,
> Steve Amerige
> Principal Software Developer, Fraud and Compliance Solutions Development
> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>
>
>
> ./lib/groovy-sql-2.4.4.jar
> ./lib/groovy-testng-2.4.4.jar
> ./lib/groovy-jsr223-2.4.4.jar
> ./lib/groovy-servlet-2.4.4.jar
> ./lib/groovy-json-2.4.4.jar
> ./lib/groovy-jmx-2.4.4.jar
> ./lib/groovy-test-2.4.4.jar
> ./lib/groovy-bsf-2.4.4.jar
> ./lib/groovy-groovydoc-2.4.4.jar
> ./lib/groovy-nio-2.4.4.jar
> ./lib/groovy-console-2.4.4.jar
> ./lib/groovy-xml-2.4.4.jar
> ./lib/groovy-ant-2.4.4.jar
> ./lib/groovy-docgenerator-2.4.4.jar
> ./lib/groovy-groovysh-2.4.4.jar
> ./lib/groovy-templates-2.4.4.jar
> ./lib/groovy-swing-2.4.4.jar
> ./lib/groovy-2.4.4.jar
> ./apache-groovy-src-2.4.4-incubating.zip
> ./embeddable/groovy-all-2.4.4-indy.jar
> ./embeddable/groovy-all-2.4.4.jar
> ./indy/groovy-json-2.4.4-indy.jar
> ./indy/groovy-console-2.4.4-indy.jar
> ./indy/groovy-2.4.4-indy.jar
> ./indy/groovy-sql-2.4.4-indy.jar
> ./indy/groovy-jmx-2.4.4-indy.jar
> ./indy/groovy-servlet-2.4.4-indy.jar
> ./indy/groovy-xml-2.4.4-indy.jar
> ./indy/groovy-swing-2.4.4-indy.jar
> ./indy/groovy-templates-2.4.4-indy.jar
> ./indy/groovy-ant-2.4.4-indy.jar
> ./indy/groovy-groovydoc-2.4.4-indy.jar
> ./indy/groovy-nio-2.4.4-indy.jar
> ./indy/groovy-test-2.4.4-indy.jar
> ./indy/groovy-testng-2.4.4-indy.jar
> ./indy/groovy-groovysh-2.4.4-indy.jar
> ./indy/groovy-docgenerator-2.4.4-indy.jar
> ./indy/groovy-bsf-2.4.4-indy.jar
> ./indy/groovy-jsr223-2.4.4-indy.jar
>
>
>
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Henrik Martin <he...@netgate.net>.
On 7/28/15 12:08 PM, Steve Amerige wrote:
> Hi Henrik,
>
> In most instances, version numbers aren't part of filenames. Consider
> executables. For example, OS commands such as 'ls' aren't ls-1.1.
> Scripts are written to depend on resources with constant naming. The
> same applies to jar files. Code can be written to use standardized
> filenames and can be depended upon to work even when jar files are
> updated. In Linux, the /alternatives /command is one way of managing
> versioning. And, there are plenty of other approaches to versioning.
> For jars, the manifest.mf is a common (and standard) place to set
> package version information
> <https://docs.oracle.com/javase/tutorial/deployment/jar/packageman.html>.
Hi Steve. Agreed that executables are rarely versioned. But, a jar file
is an archive (more similar to a file/directory bundle), and not much
different from your versioned tar file example in your first email
(httpd-2.4.16.tar.gz). And in order to examine a manifest file, the
archive has to be unzipped first, which adds an extra step. As I
mentioned in my other email, the default behavior of Maven and Gradle is
to version artifacts. Like it or not, but it *is* the industry standard,
and has been for a long time.
> In modern deployment environments such as cloud computing, in
> particular, the notion of version is not as relevant as it used to
> be. Customers do not think of what version software is in the cloud.
> It is the application that is undergoing continuous, agile, modification.
Version numbers are of course of limited interest to the consumers of
cloud services/apps. I for sure don't care which version of Facebook I'm
using. But for the Facebook developers, it's absolutely crucial to know
which version is deployed. A company I've done work for recently had a
system glitch *precisely* because an older version of a jar file
happened to make it into a production deployment. It took hours of
debugging to determine the cause of the glitch, something that would
have been immediately avoided if the jar file name had contained the
version.
> Having version numbers as part of filenames breaks the use of these
> files as reusable components.
I don't see how. foo-1.jar and foo-2.jar aren't the same thing. Sure,
they may only differ slightly, but I don't see how being explicit breaks
things. The common way of dealing with dependencies is to be explicit
about the versioning. I.e. in my build system, I usually express
specifically which version of every single dependency I'm using (in the
case where it's relevant).
> The issue of binary, source code, and behavioral compatibility is
> important. As long as contracts are preserved, all is well. In any
> event, a version number is a very weak indicator and cannot be relied
> on to determine compatibility.
Why is versioning a "weak indicator"?
> Thorough testing before deploying code that relies on updated jar
> files is important.
Of course, but sometimes it's advantageous to have a simple mechanism
for determining this long after deployment. Sure, I can always open the
archive and examine the manifest, but that's pretty tedious, especially
in applications that may depend on hundreds of jar archives.
> And, from a security perspective, we're looking at standardized
> filenames and testing them against various exploits. It is better to
> have a relatively unchanging set of names that we can run checks
> against to determine what they are by checksums, etc. Security is
> becoming a bigger and bigger concern, and efforts should be made to
> have consistent filenames across releases so that changes are more
> readily identifiable.
OK, one additional argument for being explicit and keep the versioning
in the name :-)
> From at IT perspective, changes are pushed out to developers to ensure
> that they're developing into standardized environments. We're
> essentially doing at least the following as a workaround:
>
> VERSION=2.4.4
> cd /usr/local
> rm -rf groovy-$VERSIONgroovy
> unzip -q /network/path/to/apache-groovy-sdk-$VERSION.zip
> ln -s groovy-$VERSION groovy
> cd groovy-$VERSION
> find . -name "*-$VERSION*" | while read FILENAME ; do
> STANDARD_NAME=$(echo "$FILENAME" | sed "s/-$VERSION//")
> mv $FILENAME $STANDARD_NAME
> done
>
This seems like a lot of extra work to me, and if you need portability
across OS flavors/versions, this is certainly a big hassle. Many
companies have developers on multiple platforms (Windows, MacOS, Linux,
...). Managing scripts, symlinks, and renaming files is a sure way to
create a lot of headaches in a heterogeneous environment. Why are you
not relying on the build system to provide the correct versions of your
dependencies? You could set up a local Archiva server or similar to
serve up the jar files that have been approved and security tested.
Maybe your requirements are different than most shops, but it seems like
a lot of extra work and a pretty brittle system to do what you showed
above. Why not drive this through Maven or Gradle, and integrate that
with your SCM system? Most project I've worked on in the last few years
make sure all build dependencies are explicitly expressed in a pom.xml
or build.gradle file. When a developer checks out the latest revision
from the SCM and runs a build, he/she will have the exact same
dependencies as all the other developers. No need for mucking around
with unzipping archives, renaming files, creating symlinks, etc.
> Note that the above isn't perfect as the internal components of the
> zip archive include other jar files with version numbers other than
> the Groovy version number. One could remove version numbers from all
> files. It's merely used for illustration and shouldn't be used in any
> production environment.
>
> Apache's own HTTP <http://httpd.apache.org/download.cgi> or Tomcat
> <http://tomcat.apache.org/download-80.cgi> code that doesn't use
> filenames with embedded version numbers; however, counter examples
> include Hadoop <https://hadoop.apache.org/releases.html> and Lucene
> <https://lucene.apache.org/core/> that do use version numbering in
> filenames. So, it is clear that this is an area that has not been
> standardized with the ASF. Perhaps the Apache folks can chime in on this.
Well, if you pull your Apache jar dependencies from a Maven/Ivy repo,
all artifacts are versioned, so I'm not sure what the benefit is of
doing it the Tomcat way. I think it has more to do with the fact that
Tomcat is a very old product, and it's just legacy packaging. I would
applaud if the jars distributed with Tomcat were versioned just like the
Maven/Ivy artifacts.
> We continue to assert that having version numbers, or other metadata,
> as a part of filenames is a bad practice. We do have workarounds in
> place, but think it would benefit the community to consider this change.
And I continue to assert the opposite :-)
The habit of renaming archives as in your script snippet above is
especially bad practice if you ask me. I'll give you another interesting
use case where versioning info in the file name is handy. Have you ever
happened to be in a situation where an app fails or starts behaving
strange because of a duplicate jar file name with a different version
happen to end up on the classpath? It's a lot quicker to figure this out
when immediately noticing that the jar file name contains two different
versions. Example classpath:
/usr/local/foo/foo.jar:...<gazillion other classpath entries
here>...:/opt/foo/foo.jar:...<gazillion other classpath entries here>...
To figure out the versions of the duplicate foo.jar archives, I have to
unzip them and examine the manifest. If the version is already in the
file name, it would have been immediately obvious.
>
> Enjoy,
> Steve Amerige
> Principal Software Developer, Fraud and Compliance Solutions Development
> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>
>
Cheers,
-H
>
> On 7/28/2015 10:38 AM, Henrik Martin wrote:
>> I'm not part of the contributor team, so I can't speak for the Groovy
>> team, but I would strongly disagree with you. If you use Maven or
>> Gradle, it's easy to maintain dependencies on particular versions of
>> jar files, and have your IDE immediately pick up the new version. In
>> fact, the default behavior for both Maven and Gradle is to explicitly
>> maintain version numbers in artifacts. Removing this would be a step
>> back to the 90s. Sometimes jar files have to copied into other
>> directories outside of their normal home. An example is when
>> deploying Tomcat. Stuff like jdbc drivers etc typically get copied
>> into $CATALINA_BASE/lib. It's worth gold to immediately be able to
>> tell which particular version of those jar files are in there, vs
>> just seeing "foobar.jar".
>>
>> I would argue that you should probably change the practice of
>> creating symlinks to explicitly versioned jar files as this is
>> obviously a pain when new versions are introduced.
>>
>> Just my $0.02.
>>
>> -H
>>
>> On 7/28/15 5:26 AM, Steve Amerige wrote:
>>> Hi all,
>>>
>>> Every time we take a download of the latest Groovy software, we have
>>> to do the same task: remove version numbers from filenames. As of
>>> the 2.4.4 release, there are 39 files, shown below, that have the
>>> version number as part of the distribution. So, why is this a problem?
>>>
>>> * IDEs cannot silently be updated to use a mandated Groovy
>>> version. With 2.4.4 dealing with a zero-day vulnerability
>>> issue, we want to push this out. However, the version numbers
>>> in files mean that users must participate in the updating. This
>>> is not desirable.
>>> * Links that users might have at the OS level are broken with each
>>> new release.
>>> * Version numbers in files makes it more difficult to diff between
>>> releases.
>>> * Version numbers as a part of a filename is a specific case of
>>> metadata as part of filenames and, as such, we consider it to be
>>> a bad practice. This information is better kept in a file,
>>> preferably machine readable in a format such as JSON or XML, or
>>> in manifest files that can be consumed by software that might do
>>> version number validation as part of security efforts in
>>> maintaining code.
>>>
>>> It is reasonable that the root directory include a version number.
>>> But, under that directory, we'd expect that the contents are
>>> version-less. A good example of a version-less Apache project is the
>>> HTTP Server <http://httpd.apache.org/download.cgi>. The download is
>>> presently a file named *httpd-2.4.16.tar.gz*, and when extracted
>>> produces a top-level directory named *httpd-2.4.16*. No file name
>>> contains the version number string. The two files *CHANGES *and
>>> *httpd.spec *contain the version number string. I believe that
>>> Groovy should follow this example, and possibly go one step better
>>> by having an explicit manifest file with all pertinent metadata for
>>> the Groovy release that includes metadata such as the version
>>> number, license name, checksums of files (for security checking), etc.
>>>
>>> If you agree, how can we start the process of making this change?
>>>
>>> Thanks,
>>> Steve Amerige
>>> Principal Software Developer, Fraud and Compliance Solutions Development
>>> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>>>
>>>
>>>
>>> ./lib/groovy-sql-2.4.4.jar
>>> ./lib/groovy-testng-2.4.4.jar
>>> ./lib/groovy-jsr223-2.4.4.jar
>>> ./lib/groovy-servlet-2.4.4.jar
>>> ./lib/groovy-json-2.4.4.jar
>>> ./lib/groovy-jmx-2.4.4.jar
>>> ./lib/groovy-test-2.4.4.jar
>>> ./lib/groovy-bsf-2.4.4.jar
>>> ./lib/groovy-groovydoc-2.4.4.jar
>>> ./lib/groovy-nio-2.4.4.jar
>>> ./lib/groovy-console-2.4.4.jar
>>> ./lib/groovy-xml-2.4.4.jar
>>> ./lib/groovy-ant-2.4.4.jar
>>> ./lib/groovy-docgenerator-2.4.4.jar
>>> ./lib/groovy-groovysh-2.4.4.jar
>>> ./lib/groovy-templates-2.4.4.jar
>>> ./lib/groovy-swing-2.4.4.jar
>>> ./lib/groovy-2.4.4.jar
>>> ./apache-groovy-src-2.4.4-incubating.zip
>>> ./embeddable/groovy-all-2.4.4-indy.jar
>>> ./embeddable/groovy-all-2.4.4.jar
>>> ./indy/groovy-json-2.4.4-indy.jar
>>> ./indy/groovy-console-2.4.4-indy.jar
>>> ./indy/groovy-2.4.4-indy.jar
>>> ./indy/groovy-sql-2.4.4-indy.jar
>>> ./indy/groovy-jmx-2.4.4-indy.jar
>>> ./indy/groovy-servlet-2.4.4-indy.jar
>>> ./indy/groovy-xml-2.4.4-indy.jar
>>> ./indy/groovy-swing-2.4.4-indy.jar
>>> ./indy/groovy-templates-2.4.4-indy.jar
>>> ./indy/groovy-ant-2.4.4-indy.jar
>>> ./indy/groovy-groovydoc-2.4.4-indy.jar
>>> ./indy/groovy-nio-2.4.4-indy.jar
>>> ./indy/groovy-test-2.4.4-indy.jar
>>> ./indy/groovy-testng-2.4.4-indy.jar
>>> ./indy/groovy-groovysh-2.4.4-indy.jar
>>> ./indy/groovy-docgenerator-2.4.4-indy.jar
>>> ./indy/groovy-bsf-2.4.4-indy.jar
>>> ./indy/groovy-jsr223-2.4.4-indy.jar
>>>
>>
>
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Steve Amerige <St...@sas.com>.
Hi Henrik,
In most instances, version numbers aren't part of filenames. Consider
executables. For example, OS commands such as 'ls' aren't ls-1.1.
Scripts are written to depend on resources with constant naming. The
same applies to jar files. Code can be written to use standardized
filenames and can be depended upon to work even when jar files are
updated. In Linux, the /alternatives /command is one way of managing
versioning. And, there are plenty of other approaches to versioning.
For jars, the manifest.mf is a common (and standard) place to set
package version information
<https://docs.oracle.com/javase/tutorial/deployment/jar/packageman.html>.
In modern deployment environments such as cloud computing, in
particular, the notion of version is not as relevant as it used to be.
Customers do not think of what version software is in the cloud. It is
the application that is undergoing continuous, agile, modification.
Having version numbers as part of filenames breaks the use of these
files as reusable components. The issue of binary, source code, and
behavioral compatibility is important. As long as contracts are
preserved, all is well. In any event, a version number is a very weak
indicator and cannot be relied on to determine compatibility. Thorough
testing before deploying code that relies on updated jar files is important.
And, from a security perspective, we're looking at standardized
filenames and testing them against various exploits. It is better to
have a relatively unchanging set of names that we can run checks against
to determine what they are by checksums, etc. Security is becoming a
bigger and bigger concern, and efforts should be made to have consistent
filenames across releases so that changes are more readily identifiable.
From at IT perspective, changes are pushed out to developers to ensure
that they're developing into standardized environments. We're
essentially doing at least the following as a workaround:
VERSION=2.4.4
cd /usr/local
rm -rf groovy-$VERSIONgroovy
unzip -q /network/path/to/apache-groovy-sdk-$VERSION.zip
ln -s groovy-$VERSION groovy
cd groovy-$VERSION
find . -name "*-$VERSION*" | while read FILENAME ; do
STANDARD_NAME=$(echo "$FILENAME" | sed "s/-$VERSION//")
mv $FILENAME $STANDARD_NAME
done
Note that the above isn't perfect as the internal components of the zip
archive include other jar files with version numbers other than the
Groovy version number. One could remove version numbers from all
files. It's merely used for illustration and shouldn't be used in any
production environment.
Apache's own HTTP <http://httpd.apache.org/download.cgi> or Tomcat
<http://tomcat.apache.org/download-80.cgi> code that doesn't use
filenames with embedded version numbers; however, counter examples
include Hadoop <https://hadoop.apache.org/releases.html> and Lucene
<https://lucene.apache.org/core/> that do use version numbering in
filenames. So, it is clear that this is an area that has not been
standardized with the ASF. Perhaps the Apache folks can chime in on this.
We continue to assert that having version numbers, or other metadata, as
a part of filenames is a bad practice. We do have workarounds in place,
but think it would benefit the community to consider this change.
Enjoy,
Steve Amerige
Principal Software Developer, Fraud and Compliance Solutions Development
SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
On 7/28/2015 10:38 AM, Henrik Martin wrote:
> I'm not part of the contributor team, so I can't speak for the Groovy
> team, but I would strongly disagree with you. If you use Maven or
> Gradle, it's easy to maintain dependencies on particular versions of
> jar files, and have your IDE immediately pick up the new version. In
> fact, the default behavior for both Maven and Gradle is to explicitly
> maintain version numbers in artifacts. Removing this would be a step
> back to the 90s. Sometimes jar files have to copied into other
> directories outside of their normal home. An example is when deploying
> Tomcat. Stuff like jdbc drivers etc typically get copied into
> $CATALINA_BASE/lib. It's worth gold to immediately be able to tell
> which particular version of those jar files are in there, vs just
> seeing "foobar.jar".
>
> I would argue that you should probably change the practice of creating
> symlinks to explicitly versioned jar files as this is obviously a pain
> when new versions are introduced.
>
> Just my $0.02.
>
> -H
>
> On 7/28/15 5:26 AM, Steve Amerige wrote:
>> Hi all,
>>
>> Every time we take a download of the latest Groovy software, we have
>> to do the same task: remove version numbers from filenames. As of
>> the 2.4.4 release, there are 39 files, shown below, that have the
>> version number as part of the distribution. So, why is this a problem?
>>
>> * IDEs cannot silently be updated to use a mandated Groovy
>> version. With 2.4.4 dealing with a zero-day vulnerability issue,
>> we want to push this out. However, the version numbers in files
>> mean that users must participate in the updating. This is not
>> desirable.
>> * Links that users might have at the OS level are broken with each
>> new release.
>> * Version numbers in files makes it more difficult to diff between
>> releases.
>> * Version numbers as a part of a filename is a specific case of
>> metadata as part of filenames and, as such, we consider it to be
>> a bad practice. This information is better kept in a file,
>> preferably machine readable in a format such as JSON or XML, or
>> in manifest files that can be consumed by software that might do
>> version number validation as part of security efforts in
>> maintaining code.
>>
>> It is reasonable that the root directory include a version number.
>> But, under that directory, we'd expect that the contents are
>> version-less. A good example of a version-less Apache project is the
>> HTTP Server <http://httpd.apache.org/download.cgi>. The download is
>> presently a file named *httpd-2.4.16.tar.gz*, and when extracted
>> produces a top-level directory named *httpd-2.4.16*. No file name
>> contains the version number string. The two files *CHANGES *and
>> *httpd.spec *contain the version number string. I believe that
>> Groovy should follow this example, and possibly go one step better by
>> having an explicit manifest file with all pertinent metadata for the
>> Groovy release that includes metadata such as the version number,
>> license name, checksums of files (for security checking), etc.
>>
>> If you agree, how can we start the process of making this change?
>>
>> Thanks,
>> Steve Amerige
>> Principal Software Developer, Fraud and Compliance Solutions Development
>> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>>
>>
>>
>> ./lib/groovy-sql-2.4.4.jar
>> ./lib/groovy-testng-2.4.4.jar
>> ./lib/groovy-jsr223-2.4.4.jar
>> ./lib/groovy-servlet-2.4.4.jar
>> ./lib/groovy-json-2.4.4.jar
>> ./lib/groovy-jmx-2.4.4.jar
>> ./lib/groovy-test-2.4.4.jar
>> ./lib/groovy-bsf-2.4.4.jar
>> ./lib/groovy-groovydoc-2.4.4.jar
>> ./lib/groovy-nio-2.4.4.jar
>> ./lib/groovy-console-2.4.4.jar
>> ./lib/groovy-xml-2.4.4.jar
>> ./lib/groovy-ant-2.4.4.jar
>> ./lib/groovy-docgenerator-2.4.4.jar
>> ./lib/groovy-groovysh-2.4.4.jar
>> ./lib/groovy-templates-2.4.4.jar
>> ./lib/groovy-swing-2.4.4.jar
>> ./lib/groovy-2.4.4.jar
>> ./apache-groovy-src-2.4.4-incubating.zip
>> ./embeddable/groovy-all-2.4.4-indy.jar
>> ./embeddable/groovy-all-2.4.4.jar
>> ./indy/groovy-json-2.4.4-indy.jar
>> ./indy/groovy-console-2.4.4-indy.jar
>> ./indy/groovy-2.4.4-indy.jar
>> ./indy/groovy-sql-2.4.4-indy.jar
>> ./indy/groovy-jmx-2.4.4-indy.jar
>> ./indy/groovy-servlet-2.4.4-indy.jar
>> ./indy/groovy-xml-2.4.4-indy.jar
>> ./indy/groovy-swing-2.4.4-indy.jar
>> ./indy/groovy-templates-2.4.4-indy.jar
>> ./indy/groovy-ant-2.4.4-indy.jar
>> ./indy/groovy-groovydoc-2.4.4-indy.jar
>> ./indy/groovy-nio-2.4.4-indy.jar
>> ./indy/groovy-test-2.4.4-indy.jar
>> ./indy/groovy-testng-2.4.4-indy.jar
>> ./indy/groovy-groovysh-2.4.4-indy.jar
>> ./indy/groovy-docgenerator-2.4.4-indy.jar
>> ./indy/groovy-bsf-2.4.4-indy.jar
>> ./indy/groovy-jsr223-2.4.4-indy.jar
>>
>
Re: Remove Version from File Names in Distributions; Add Manifest
Posted by Henrik Martin <he...@netgate.net>.
I'm not part of the contributor team, so I can't speak for the Groovy
team, but I would strongly disagree with you. If you use Maven or
Gradle, it's easy to maintain dependencies on particular versions of jar
files, and have your IDE immediately pick up the new version. In fact,
the default behavior for both Maven and Gradle is to explicitly maintain
version numbers in artifacts. Removing this would be a step back to the
90s. Sometimes jar files have to copied into other directories outside
of their normal home. An example is when deploying Tomcat. Stuff like
jdbc drivers etc typically get copied into $CATALINA_BASE/lib. It's
worth gold to immediately be able to tell which particular version of
those jar files are in there, vs just seeing "foobar.jar".
I would argue that you should probably change the practice of creating
symlinks to explicitly versioned jar files as this is obviously a pain
when new versions are introduced.
Just my $0.02.
-H
On 7/28/15 5:26 AM, Steve Amerige wrote:
> Hi all,
>
> Every time we take a download of the latest Groovy software, we have
> to do the same task: remove version numbers from file names. As of the
> 2.4.4 release, there are 39 files, shown below, that have the version
> number as part of the distribution. So, why is this a problem?
>
> * IDEs cannot silently be updated to use a mandated Groovy version.
> With 2.4.4 dealing with a zero-day vulnerability issue, we want to
> push this out. However, the version numbers in files mean that
> users must participate in the updating. This is not desirable.
> * Links that users might have at the OS level are broken with each
> new release.
> * Version numbers in files makes it more difficult to diff between
> releases.
> * Version numbers as a part of a filename is a specific case of
> metadata as part of file names and, as such, we consider it to be
> a bad practice. This information is better kept in a file,
> preferably machine readable in a format such as JSON or XML, or in
> manifest files that can be consumed by software that might do
> version number validation as part of security efforts in
> maintaining code.
>
> It is reasonable that the root directory include a version number.
> But, under that directory, we'd expect that the contents are
> version-less. A good example of a version-less Apache project is the
> HTTP Server <http://httpd.apache.org/download.cgi>. The download is
> presently a file named *httpd-2.4.16.tar.gz*, and when extracted
> produces a top-level directory named *httpd-2.4.16*. No file name
> contains the version number string. The two files *CHANGES *and
> *httpd.spec *contain the version number string. I believe that Groovy
> should follow this example, and possibly go one step better by having
> an explicit manifest file with all pertinent metadata for the Groovy
> release that includes metadata such as the version number, license
> name, checksums of files (for security checking), etc.
>
> If you agree, how can we start the process of making this change?
>
> Thanks,
> Steve Amerige
> Principal Software Developer, Fraud and Compliance Solutions Development
> SAS Institute, 100 SAS Campus Dr, Room U3050, Cary, NC 27513-8617
>
>
>
> ./lib/groovy-sql-2.4.4.jar
> ./lib/groovy-testng-2.4.4.jar
> ./lib/groovy-jsr223-2.4.4.jar
> ./lib/groovy-servlet-2.4.4.jar
> ./lib/groovy-json-2.4.4.jar
> ./lib/groovy-jmx-2.4.4.jar
> ./lib/groovy-test-2.4.4.jar
> ./lib/groovy-bsf-2.4.4.jar
> ./lib/groovy-groovydoc-2.4.4.jar
> ./lib/groovy-nio-2.4.4.jar
> ./lib/groovy-console-2.4.4.jar
> ./lib/groovy-xml-2.4.4.jar
> ./lib/groovy-ant-2.4.4.jar
> ./lib/groovy-docgenerator-2.4.4.jar
> ./lib/groovy-groovysh-2.4.4.jar
> ./lib/groovy-templates-2.4.4.jar
> ./lib/groovy-swing-2.4.4.jar
> ./lib/groovy-2.4.4.jar
> ./apache-groovy-src-2.4.4-incubating.zip
> ./embeddable/groovy-all-2.4.4-indy.jar
> ./embeddable/groovy-all-2.4.4.jar
> ./indy/groovy-json-2.4.4-indy.jar
> ./indy/groovy-console-2.4.4-indy.jar
> ./indy/groovy-2.4.4-indy.jar
> ./indy/groovy-sql-2.4.4-indy.jar
> ./indy/groovy-jmx-2.4.4-indy.jar
> ./indy/groovy-servlet-2.4.4-indy.jar
> ./indy/groovy-xml-2.4.4-indy.jar
> ./indy/groovy-swing-2.4.4-indy.jar
> ./indy/groovy-templates-2.4.4-indy.jar
> ./indy/groovy-ant-2.4.4-indy.jar
> ./indy/groovy-groovydoc-2.4.4-indy.jar
> ./indy/groovy-nio-2.4.4-indy.jar
> ./indy/groovy-test-2.4.4-indy.jar
> ./indy/groovy-testng-2.4.4-indy.jar
> ./indy/groovy-groovysh-2.4.4-indy.jar
> ./indy/groovy-docgenerator-2.4.4-indy.jar
> ./indy/groovy-bsf-2.4.4-indy.jar
> ./indy/groovy-jsr223-2.4.4-indy.jar
>