You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mime4j-dev@james.apache.org by Andreas Veithen <an...@gmail.com> on 2013/03/23 10:51:56 UTC
Issues in Axiom after recent changes in MIME4J
All,
As you may know, Apache Axiom uses MIME4J (actually mime4j-core). We
switched from our own custom MIME processing code to MIME4J more than
a year and a half ago and we are very happy with the results: it
allows Axiom to do some extra optimizations and since the switch no
issues have ever been reported.
Our builds are configured to use the latest snapshot (now
0.8.0-SNAPSHOT). A couple of days ago, there have been a bunch of
changes in MIME4J and we detected a couple of issues with recent
snapshots:
1. Our OSGi tests sometimes fail with the following error:
org.ops4j.pax.runner.platform.PlatformException:
[file:/home/jenkins/jenkins-slave/workspace/ws-axiom-trunk/.repository/org/apache/james/apache-mime4j-core/0.8.0-SNAPSHOT/apache-mime4j-core-0.8.0-SNAPSHOT.jar]
is not a valid bundle
It turns out that this error occurs only if the last mime4-core
snapshot has been built on Solaris. If the snapshot has been built on
Ubuntu, the problem doesn't occur. It looks like there is a platform
specific issue in the MIME4J build, more precisely in the generation
of the OSGi bundle manifest.
2. mime4j-core is now compiled for Java 1.6, but the target version
for Axiom is still Java 1.5. The discussion about the Java version
already came up some time ago [1], and I think that the conclusions
are still valid. Quoting Oleg: "We should upgrade to Java 1.6 only if
there is a compelling reason to do so." Can you please configure at
least mime4j-core for Java 1.5?
In the meantime we will switch back to the latest MIME4J release.
Andreas
[1] http://markmail.org/thread/p5ojyzlns4c45cc6
[PATCH] 1.5 compatibility; was Re: Issues in Axiom after recent
changes in MIME4J
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2013-04-24 at 09:47 +0300, Ioan Eugen Stan wrote:
> Hi Oleg,
>
> I re-factored the code and switched to Java 1.6 language (for-each,
> @Overrides, etc). I didn't expect to cause a mess. I didn't expect
> people to still require Java 1.5 since it's EOL for such a long time.
> There is no right answer to your question, I don't particularly like
> having different requirements. On the other hand, I favor moving
> forward, although I also recognize the need to maintain compatibility
> to older platforms. Ideally we can solve that problem if we ensure
> that some branches (like 0.7.x) require Java 1.5, while in new
> releases we can require a different Java version if needed. This has
> the disadvantage that it will put a harder burden on maintenance.
>
> In our case we can require Java 1.5 for mime4j-core. For dom we can
> keep 1.6 or do some work to revert back.
>
> Cheers,
It is not about expecting people to use older versions of JRE but rather
providing a reasonable level of compatibility. We can build against 1.5
and at the same time to recommend users to run 1.6 or newer. Upgrading
to 1.6 buys us almost nothing (the only nice to have functionality I can
think of is String constructors taking charset as java.nio.Charset
instead of String).
I am attaching a patch for your review that restores full 1.5
compatibility. The changes, as you can see, are very minor and affect
only three classes. I also had to downgrade Commons IO to version 2.2
but as it turned out we are not using any new functionality at all. So,
I do not see this as a loss.
If I hear no objections I'll commit the patch in a few days. So, please
complain loudly if you find anything disagreeable.
Oleg
Re: Issues in Axiom after recent changes in MIME4J
Posted by Ioan Eugen Stan <st...@gmail.com>.
Hi Oleg,
I re-factored the code and switched to Java 1.6 language (for-each,
@Overrides, etc). I didn't expect to cause a mess. I didn't expect
people to still require Java 1.5 since it's EOL for such a long time.
There is no right answer to your question, I don't particularly like
having different requirements. On the other hand, I favor moving
forward, although I also recognize the need to maintain compatibility
to older platforms. Ideally we can solve that problem if we ensure
that some branches (like 0.7.x) require Java 1.5, while in new
releases we can require a different Java version if needed. This has
the disadvantage that it will put a harder burden on maintenance.
In our case we can require Java 1.5 for mime4j-core. For dom we can
keep 1.6 or do some work to revert back.
Cheers,
Re: Issues in Axiom after recent changes in MIME4J
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2013-03-24 at 11:23 +0200, Ioan Eugen Stan wrote:
> Hello Andreas,
>
> Sorry for breaking your code. I didn't expect people to use SNAPSHOT
> versions as a dependency in their code. I've looked at Axiom. Nice
> project, didn't know about it.
>
> [1] https://builds.apache.org/job/ws-axiom-trunk/
>
> Regards,
>
Hi Ioan
Sorry for coming late to the party.
I just noticed that mime4j core requires Java 1.5 while mime4j dom does
Java 1.6. Is there a particular reason for requiring Java 1.6 for DOM
APIs? Do we really want to have different minimal JRE levels per
individual modules?
Oleg
> On Sat, Mar 23, 2013 at 11:51 AM, Andreas Veithen
> <an...@gmail.com> wrote:
> > All,
> >
> > As you may know, Apache Axiom uses MIME4J (actually mime4j-core). We
> > switched from our own custom MIME processing code to MIME4J more than
> > a year and a half ago and we are very happy with the results: it
> > allows Axiom to do some extra optimizations and since the switch no
> > issues have ever been reported.
>
> Great to hear!
>
> > Our builds are configured to use the latest snapshot (now
> > 0.8.0-SNAPSHOT). A couple of days ago, there have been a bunch of
> > changes in MIME4J and we detected a couple of issues with recent
> > snapshots:
> >
> > 1. Our OSGi tests sometimes fail with the following error:
> >
> > org.ops4j.pax.runner.platform.PlatformException:
> > [file:/home/jenkins/jenkins-slave/workspace/ws-axiom-trunk/.repository/org/apache/james/apache-mime4j-core/0.8.0-SNAPSHOT/apache-mime4j-core-0.8.0-SNAPSHOT.jar]
> > is not a valid bundle
> >
> > It turns out that this error occurs only if the last mime4-core
> > snapshot has been built on Solaris. If the snapshot has been built on
> > Ubuntu, the problem doesn't occur. It looks like there is a platform
> > specific issue in the MIME4J build, more precisely in the generation
> > of the OSGi bundle manifest.
>
> Thanks for reporting this issue. It bugged me a lot since it happened
> quite "random". I've fixed the build and restricted it to Ubuntu.
>
> >
> > 2. mime4j-core is now compiled for Java 1.6, but the target version
> > for Axiom is still Java 1.5. The discussion about the Java version
> > already came up some time ago [1], and I think that the conclusions
> > are still valid. Quoting Oleg: "We should upgrade to Java 1.6 only if
> > there is a compelling reason to do so." Can you please configure at
> > least mime4j-core for Java 1.5?
>
> Changed compiler plugin settings for core to build 1.5. Give it a spin
> and comment on [1] . We'll keep core at 1.5 as much as possible. I'm
> planning some API changes in the future for mime4j. It will be nice if
> I would get some feedback then.
>
> [1] https://issues.apache.org/jira/browse/MIME4J-226
> [2] http://svn.apache.org/r1460288
>
> Cheers,
>
> >
> > In the meantime we will switch back to the latest MIME4J release.
> >
> > Andreas
> >
> > [1] http://markmail.org/thread/p5ojyzlns4c45cc6
>
>
>
Re: Issues in Axiom after recent changes in MIME4J
Posted by Andreas Veithen <an...@gmail.com>.
The problem still exists. I've opened a JIRA issue for that: MIME4J-231.
Andreas
On Sun, Apr 28, 2013 at 5:05 PM, Andreas Veithen
<an...@gmail.com> wrote:
> Did somebody take a look at the issue described below?
>
> Andreas
>
> On Wed, Apr 10, 2013 at 10:22 PM, Andreas Veithen
> <an...@gmail.com> wrote:
>> The issue still occurs randomly, even on Ubuntu. I investigated this a
>> bit further, and I noticed the following.
>>
>> I inspected the logs of a build that produced an artifact without OSGi
>> manifest and I saw this:
>>
>> [INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @
>> apache-mime4j-core ---
>> [INFO]
>> [INFO] --- maven-jar-plugin:2.4:jar (jar) @ apache-mime4j-core ---
>> [INFO] Building jar:
>> /home/jenkins/jenkins-slave/workspace/mime4j-trunk/trunk/core/target/apache-mime4j-core-0.8.0-SNAPSHOT.jar
>> [INFO]
>>
>> On the other hand, for one of the builds that was successful, the
>> output is different:
>>
>> [INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @
>> apache-mime4j-core ---
>> [INFO]
>> [INFO] --- maven-jar-plugin:2.4:jar (jar) @ apache-mime4j-core ---
>> [INFO]
>>
>> Normally, there is an up-to-date check that should cause the
>> maven-jar-plugin to skip the execution of the jar goal because
>> maven-bundle-plugin already generated the JAR. It looks like for some
>> unknown reason that up-to-date check is not deterministic, and
>> sometimes maven-jar-plugin replaces the JAR generated by
>> maven-bundle-plugin. In that case maven-jar-plugin generates its own
>> MANIFEST.MF (which is expected) instead of using the manifest
>> generated by maven-bundle-plugin.
>>
>> That being said, since the packaging of the project is "bundle", there
>> actually shouldn't be any execution of the jar goal at all. It turns
>> out that the execution is configured by the
>> org.apache.james:james-project parent POM and that that configuration
>> was added between 1.8.1 and 1.8.2. You switched that version in
>> r1457430 (Sun Mar 17 12:08:05 2013), and this is indeed when the
>> problems started.
>>
>> Can you temporarily revert that change so that we can confirm the
>> analysis and check if it stabilizes the build? Once we confirm that
>> the build becomes deterministic again with
>> org.apache.james:james-project:1.8.1 you can check what modifications
>> need to be done on the parent POM to avoid the issue.
>>
>> Andreas
>>
>>
>> On Mon, Apr 1, 2013 at 2:05 PM, Ioan Eugen Stan <st...@gmail.com> wrote:
>>> Thank you Andreas for investigating the issue and reporting it. If
>>> your changes do not fix the issue, maybe it would be wise to report
>>> this to infra.
>>> For some reason Jenkins builds are very problematic - they fail for a
>>> lot of reasons unrelated to code.
>>>
>>> Please keep us posted if you get updates on this issue.
>>>
>>> Cheers,
Re: Issues in Axiom after recent changes in MIME4J
Posted by Andreas Veithen <an...@gmail.com>.
Did somebody take a look at the issue described below?
Andreas
On Wed, Apr 10, 2013 at 10:22 PM, Andreas Veithen
<an...@gmail.com> wrote:
> The issue still occurs randomly, even on Ubuntu. I investigated this a
> bit further, and I noticed the following.
>
> I inspected the logs of a build that produced an artifact without OSGi
> manifest and I saw this:
>
> [INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @
> apache-mime4j-core ---
> [INFO]
> [INFO] --- maven-jar-plugin:2.4:jar (jar) @ apache-mime4j-core ---
> [INFO] Building jar:
> /home/jenkins/jenkins-slave/workspace/mime4j-trunk/trunk/core/target/apache-mime4j-core-0.8.0-SNAPSHOT.jar
> [INFO]
>
> On the other hand, for one of the builds that was successful, the
> output is different:
>
> [INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @
> apache-mime4j-core ---
> [INFO]
> [INFO] --- maven-jar-plugin:2.4:jar (jar) @ apache-mime4j-core ---
> [INFO]
>
> Normally, there is an up-to-date check that should cause the
> maven-jar-plugin to skip the execution of the jar goal because
> maven-bundle-plugin already generated the JAR. It looks like for some
> unknown reason that up-to-date check is not deterministic, and
> sometimes maven-jar-plugin replaces the JAR generated by
> maven-bundle-plugin. In that case maven-jar-plugin generates its own
> MANIFEST.MF (which is expected) instead of using the manifest
> generated by maven-bundle-plugin.
>
> That being said, since the packaging of the project is "bundle", there
> actually shouldn't be any execution of the jar goal at all. It turns
> out that the execution is configured by the
> org.apache.james:james-project parent POM and that that configuration
> was added between 1.8.1 and 1.8.2. You switched that version in
> r1457430 (Sun Mar 17 12:08:05 2013), and this is indeed when the
> problems started.
>
> Can you temporarily revert that change so that we can confirm the
> analysis and check if it stabilizes the build? Once we confirm that
> the build becomes deterministic again with
> org.apache.james:james-project:1.8.1 you can check what modifications
> need to be done on the parent POM to avoid the issue.
>
> Andreas
>
>
> On Mon, Apr 1, 2013 at 2:05 PM, Ioan Eugen Stan <st...@gmail.com> wrote:
>> Thank you Andreas for investigating the issue and reporting it. If
>> your changes do not fix the issue, maybe it would be wise to report
>> this to infra.
>> For some reason Jenkins builds are very problematic - they fail for a
>> lot of reasons unrelated to code.
>>
>> Please keep us posted if you get updates on this issue.
>>
>> Cheers,
Re: Issues in Axiom after recent changes in MIME4J
Posted by Andreas Veithen <an...@gmail.com>.
The issue still occurs randomly, even on Ubuntu. I investigated this a
bit further, and I noticed the following.
I inspected the logs of a build that produced an artifact without OSGi
manifest and I saw this:
[INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @
apache-mime4j-core ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (jar) @ apache-mime4j-core ---
[INFO] Building jar:
/home/jenkins/jenkins-slave/workspace/mime4j-trunk/trunk/core/target/apache-mime4j-core-0.8.0-SNAPSHOT.jar
[INFO]
On the other hand, for one of the builds that was successful, the
output is different:
[INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @
apache-mime4j-core ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (jar) @ apache-mime4j-core ---
[INFO]
Normally, there is an up-to-date check that should cause the
maven-jar-plugin to skip the execution of the jar goal because
maven-bundle-plugin already generated the JAR. It looks like for some
unknown reason that up-to-date check is not deterministic, and
sometimes maven-jar-plugin replaces the JAR generated by
maven-bundle-plugin. In that case maven-jar-plugin generates its own
MANIFEST.MF (which is expected) instead of using the manifest
generated by maven-bundle-plugin.
That being said, since the packaging of the project is "bundle", there
actually shouldn't be any execution of the jar goal at all. It turns
out that the execution is configured by the
org.apache.james:james-project parent POM and that that configuration
was added between 1.8.1 and 1.8.2. You switched that version in
r1457430 (Sun Mar 17 12:08:05 2013), and this is indeed when the
problems started.
Can you temporarily revert that change so that we can confirm the
analysis and check if it stabilizes the build? Once we confirm that
the build becomes deterministic again with
org.apache.james:james-project:1.8.1 you can check what modifications
need to be done on the parent POM to avoid the issue.
Andreas
On Mon, Apr 1, 2013 at 2:05 PM, Ioan Eugen Stan <st...@gmail.com> wrote:
> Thank you Andreas for investigating the issue and reporting it. If
> your changes do not fix the issue, maybe it would be wise to report
> this to infra.
> For some reason Jenkins builds are very problematic - they fail for a
> lot of reasons unrelated to code.
>
> Please keep us posted if you get updates on this issue.
>
> Cheers,
Re: Issues in Axiom after recent changes in MIME4J
Posted by Ioan Eugen Stan <st...@gmail.com>.
Thank you Andreas for investigating the issue and reporting it. If
your changes do not fix the issue, maybe it would be wise to report
this to infra.
For some reason Jenkins builds are very problematic - they fail for a
lot of reasons unrelated to code.
Please keep us posted if you get updates on this issue.
Cheers,
Re: Issues in Axiom after recent changes in MIME4J
Posted by Andreas Veithen <an...@gmail.com>.
We actually don't depend on any feature in the snapshot version and
Axiom works fine with mime4j 0.7.2. The build is configured to use the
snapshot version to detect regression issues early.
Thanks for switching the Java version back to 1.5. I changed the
dependency back to 0.8.0-SNAPSHOT and I noticed that we get the OSGi
test failure also with certain snapshots built on Ubuntu. I had a
closer look at these builds and I noticed the following error in the
logs:
[INFO] --- maven-bundle-plugin:2.3.7:install (default-install) @
apache-mime4j-core ---
[WARNING] Exception while updating local OBR: Unable to read
repository xml:
file:/home/jenkins/jenkins-slave/maven-repositories/1/repository.xml
org.apache.maven.plugin.MojoExecutionException: Unable to read
repository xml:
file:/home/jenkins/jenkins-slave/maven-repositories/1/repository.xml
at org.apache.felix.obrplugin.ObrUpdate.parseRepositoryXml(ObrUpdate.java:324)
at org.apache.felix.obrplugin.ObrInstall.execute(ObrInstall.java:141)
[...]
Caused by: org.xmlpull.v1.XmlPullParserException: unexpected type
(position:END_DOCUMENT null@3:1 in java.io.InputStreamReader@95d728)
[...]
It looks like the local Maven repository is corrupted. This suggests
that the issue actually doesn't depend on the platform of the executor
on which the build runs, but instead on the state of the local Maven
repository of the executor.
To avoid that issue I changed the configuration of the mime4j Jenkins
job to use a private Maven repository local to the workspace.
Hopefully this will solve the issue.
Andreas
On Sun, Mar 24, 2013 at 10:23 AM, Ioan Eugen Stan <st...@gmail.com> wrote:
> Hello Andreas,
>
> Sorry for breaking your code. I didn't expect people to use SNAPSHOT
> versions as a dependency in their code. I've looked at Axiom. Nice
> project, didn't know about it.
>
> [1] https://builds.apache.org/job/ws-axiom-trunk/
>
> Regards,
>
> On Sat, Mar 23, 2013 at 11:51 AM, Andreas Veithen
> <an...@gmail.com> wrote:
>> All,
>>
>> As you may know, Apache Axiom uses MIME4J (actually mime4j-core). We
>> switched from our own custom MIME processing code to MIME4J more than
>> a year and a half ago and we are very happy with the results: it
>> allows Axiom to do some extra optimizations and since the switch no
>> issues have ever been reported.
>
> Great to hear!
>
>> Our builds are configured to use the latest snapshot (now
>> 0.8.0-SNAPSHOT). A couple of days ago, there have been a bunch of
>> changes in MIME4J and we detected a couple of issues with recent
>> snapshots:
>>
>> 1. Our OSGi tests sometimes fail with the following error:
>>
>> org.ops4j.pax.runner.platform.PlatformException:
>> [file:/home/jenkins/jenkins-slave/workspace/ws-axiom-trunk/.repository/org/apache/james/apache-mime4j-core/0.8.0-SNAPSHOT/apache-mime4j-core-0.8.0-SNAPSHOT.jar]
>> is not a valid bundle
>>
>> It turns out that this error occurs only if the last mime4-core
>> snapshot has been built on Solaris. If the snapshot has been built on
>> Ubuntu, the problem doesn't occur. It looks like there is a platform
>> specific issue in the MIME4J build, more precisely in the generation
>> of the OSGi bundle manifest.
>
> Thanks for reporting this issue. It bugged me a lot since it happened
> quite "random". I've fixed the build and restricted it to Ubuntu.
>
>>
>> 2. mime4j-core is now compiled for Java 1.6, but the target version
>> for Axiom is still Java 1.5. The discussion about the Java version
>> already came up some time ago [1], and I think that the conclusions
>> are still valid. Quoting Oleg: "We should upgrade to Java 1.6 only if
>> there is a compelling reason to do so." Can you please configure at
>> least mime4j-core for Java 1.5?
>
> Changed compiler plugin settings for core to build 1.5. Give it a spin
> and comment on [1] . We'll keep core at 1.5 as much as possible. I'm
> planning some API changes in the future for mime4j. It will be nice if
> I would get some feedback then.
>
> [1] https://issues.apache.org/jira/browse/MIME4J-226
> [2] http://svn.apache.org/r1460288
>
> Cheers,
>
>>
>> In the meantime we will switch back to the latest MIME4J release.
>>
>> Andreas
>>
>> [1] http://markmail.org/thread/p5ojyzlns4c45cc6
>
>
>
> --
> Ioan Eugen Stan
Re: Issues in Axiom after recent changes in MIME4J
Posted by Ioan Eugen Stan <st...@gmail.com>.
Hello Andreas,
Sorry for breaking your code. I didn't expect people to use SNAPSHOT
versions as a dependency in their code. I've looked at Axiom. Nice
project, didn't know about it.
[1] https://builds.apache.org/job/ws-axiom-trunk/
Regards,
On Sat, Mar 23, 2013 at 11:51 AM, Andreas Veithen
<an...@gmail.com> wrote:
> All,
>
> As you may know, Apache Axiom uses MIME4J (actually mime4j-core). We
> switched from our own custom MIME processing code to MIME4J more than
> a year and a half ago and we are very happy with the results: it
> allows Axiom to do some extra optimizations and since the switch no
> issues have ever been reported.
Great to hear!
> Our builds are configured to use the latest snapshot (now
> 0.8.0-SNAPSHOT). A couple of days ago, there have been a bunch of
> changes in MIME4J and we detected a couple of issues with recent
> snapshots:
>
> 1. Our OSGi tests sometimes fail with the following error:
>
> org.ops4j.pax.runner.platform.PlatformException:
> [file:/home/jenkins/jenkins-slave/workspace/ws-axiom-trunk/.repository/org/apache/james/apache-mime4j-core/0.8.0-SNAPSHOT/apache-mime4j-core-0.8.0-SNAPSHOT.jar]
> is not a valid bundle
>
> It turns out that this error occurs only if the last mime4-core
> snapshot has been built on Solaris. If the snapshot has been built on
> Ubuntu, the problem doesn't occur. It looks like there is a platform
> specific issue in the MIME4J build, more precisely in the generation
> of the OSGi bundle manifest.
Thanks for reporting this issue. It bugged me a lot since it happened
quite "random". I've fixed the build and restricted it to Ubuntu.
>
> 2. mime4j-core is now compiled for Java 1.6, but the target version
> for Axiom is still Java 1.5. The discussion about the Java version
> already came up some time ago [1], and I think that the conclusions
> are still valid. Quoting Oleg: "We should upgrade to Java 1.6 only if
> there is a compelling reason to do so." Can you please configure at
> least mime4j-core for Java 1.5?
Changed compiler plugin settings for core to build 1.5. Give it a spin
and comment on [1] . We'll keep core at 1.5 as much as possible. I'm
planning some API changes in the future for mime4j. It will be nice if
I would get some feedback then.
[1] https://issues.apache.org/jira/browse/MIME4J-226
[2] http://svn.apache.org/r1460288
Cheers,
>
> In the meantime we will switch back to the latest MIME4J release.
>
> Andreas
>
> [1] http://markmail.org/thread/p5ojyzlns4c45cc6
--
Ioan Eugen Stan