How to sort Manifest entries in a jar?

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

Does anyone have a good trick for doing that? So far my best solution
is reimplementing Manifest.write(..) which is kind of ugly.

The use case is to have reproducible builds when our slingstart plugin
rewrites a manifest using our BSNRenamer[1] - currently multiple
builds (on different platforms maybe) can lead to jars with
MANIFEST.MF entries in a different order, making the jars different
from a binary/digest point of view. Not the end of the world but
inconvenient.

-Bertrand

[1] https://svn.apache.org/repos/asf/sling/trunk/bundles/commons/osgi/src/main/java/org/apache/sling/commons/osgi/BSNRenamer.java

Re: How to sort Manifest entries in a jar?

[Bertrand Delacretaz <bd...@apache.org>]

Hi Julian,

On Fri, Apr 28, 2017 at 11:49 AM, Julian Sedding <js...@gmail.com> wrote:
> ...You could implement a FilterOutputStream that buffers the lines and on
> close sorts them, writes them to the delegate and flushes the
> delegate....

Good idea but it's a bit tricky though because of the 72 chars output
width limit that Manifest implements, you cannot just sort the
manifest lines.

I suppose this would mean reparsing and rebuilding the generated manifest.

At this point I'm inclined to do nothing and just keep this as a known
issue - it's not dramatic, just inconvenient, and the sorting
solutions are all somewhat ugly so far or involve too much duplicated
code for my taste.

-Bertrand

Re: How to sort Manifest entries in a jar?

[Julian Sedding <js...@gmail.com>]

Hi Bertrand

You could implement a FilterOutputStream that buffers the lines and on
close sorts them, writes them to the delegate and flushes the
delegate.

Then you could overwrite Manifest.write to decorate the provided OutputStream.

Not really much different from your initial suggestion, but it avoids
rewriting the Manifest.write logic.

Regards
Julian

On Fri, Apr 28, 2017 at 10:57 AM, Bertrand Delacretaz
<bd...@apache.org> wrote:
> Hi Robert,
>
> On Fri, Apr 28, 2017 at 10:04 AM, Robert Munteanu <ro...@apache.org> wrote:
>> ...My hunch is that this ordering change happens because of it being ran
>> with different Java versions...
>
> Yeah that's what I suspect as well so an obvious workaround is
> sticking with a single Java version for builds.
>
> -Bertrand

Re: How to sort Manifest entries in a jar?

[Bertrand Delacretaz <bd...@apache.org>]

Hi Robert,

On Fri, Apr 28, 2017 at 10:04 AM, Robert Munteanu <ro...@apache.org> wrote:
> ...My hunch is that this ordering change happens because of it being ran
> with different Java versions...

Yeah that's what I suspect as well so an obvious workaround is
sticking with a single Java version for builds.

-Bertrand

Re: How to sort Manifest entries in a jar?

[Robert Munteanu <ro...@apache.org>]

Hi Bertrand,

On Thu, 2017-04-27 at 18:33 +0200, Bertrand Delacretaz wrote:
> Hi,
> 
> Does anyone have a good trick for doing that? So far my best solution
> is reimplementing Manifest.write(..) which is kind of ugly.

No tricks here. I only have two ideas, and they are probably not as
good as overwriting Manifest#write:

1. Use reflection to change the inner HashMap fields to LinkedHashMap
2. Intercept the OutputStream after it has been written and apply the
sorting in-place


> The use case is to have reproducible builds when our slingstart
> plugin
> rewrites a manifest using our BSNRenamer[1] - currently multiple
> builds (on different platforms maybe) can lead to jars with
> MANIFEST.MF entries in a different order, making the jars different
> from a binary/digest point of view. Not the end of the world but
> inconvenient.

My hunch is that this ordering change happens because of it being ran
with different Java versions. There were some hash algorithm changes 
[2] with Java 8 which lead to Hash{Map,Set} getting a different
iteration order.

Robert

> 
> -Bertrand
> 
> [1] https://svn.apache.org/repos/asf/sling/trunk/bundles/commons/osgi
> /src/main/java/org/apache/sling/commons/osgi/BSNRenamer.java

[2]: https://docs.oracle.com/javase/8/docs/technotes/guides/collections
/changes8.html