You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@buildr.apache.org by Ingo Schmidt <an...@der-ingo.de> on 2013/05/21 14:25:44 UTC
Problems with merging
Hi,
once again I have a problem with merging.
Consider the following extremely simple project layout:
/config/images/logo.png
/webshop-war/src/main/webapp/images/logo.png
And the buildfile looks like this:
define "merge-fail" do
project.version = "1.0.0"
define "webshop-war" do
images = zip(_(:target, images.zip)).include(project.parent)
war = package(:war)
war.enhance([images]) # workaround for BUILDR-605
war.path("images").merge(images)
end
end
When running "buildr clean package" I can observe the following behaviour:
Sometimes the logo from the config folder is included in the WAR file,
sometimes the one from webshop-war/images.
I have not found out as to why this happens. Sometimes you get the same
result 20 times in a row, so do run as often until you finally see this
behaviour.
Could someone explain me what this is?
Any ideas for a workaround?
What I would like to achieve is the following:
If there is a picture in the config folder, use that one, otherwise use
the default one inside webshop-war/images. What is the correct way of
doing this? I am really stuck now...
Cheers, Ingo =;->
Re: Problems with merging
Posted by Ingo Schmidt <an...@der-ingo.de>.
Hi Alex,
> I haven't had time to look into this today
No problem, the buildfile had a bug anyway, here is the corrected version:
define "merge-fail" do
project.version = "1.0.0"
define "webshop-war" do
images = zip(_(:target,
"images.zip")).include(project.parent._("config","images", "**"))
war = package(:war)
war.enhance([images]) # workaround for BUILDR-605
war.path("images").merge(images)
end
end
I should say, that I ran the tests under Linux (Ubuntu 12.04 LTS) and
with buildr version 1.4.7.
Under Windows 7 with the same buildr version I could not reproduce the
problem.
> ... I suspect the
> non-deterministic behavior is due to putting things in a hash map. Will
> see if we can fix or if there's a workaround. Stay tuned.
Yes, I think you are quite right with that. I did a trace of these
builds and there is indeed some variation between them. If you want, I
can paste the trace logs here, too. Maybe that already is enough for you
to see what is going on.
Cheers, Ingo =;->
Re: Problems with merging
Posted by Ingo Schmidt <an...@der-ingo.de>.
Hi Alex,
> Ok, I committed a fix to trunk. Can you try it?
>
> (I've also shared pre-built gem binaries via dropbox directly with you)
I used the version from dropbox and it works fine indeed. I made 1000
builds and all are correct.
I have also tested it about 20 times with my actual buildfile and that
works, too!
Thanks a lot for this fix! This was really very critical, because in the
actual build it was not only a picture, that was wrong, it was some
JSPs, too, making the build useless. Now the builds are fine again :)
Cheers, Ingo =;->
Re: Problems with merging
Posted by Alex Boisvert <al...@gmail.com>.
Ok, I committed a fix to trunk. Can you try it?
(I've also shared pre-built gem binaries via dropbox directly with you)
On Sun, Jun 2, 2013 at 9:16 AM, Alex Boisvert <al...@gmail.com>wrote:
> Hi Ingo, just a quick update that I can reproduce the problem. Thanks for
> providing this sample project. Looking into it ...
>
>
> On Wed, May 29, 2013 at 9:59 AM, Ingo Schmidt <an...@der-ingo.de>wrote:
>
>> Hi Alex,
>>
>> ok, bad news, the problem persists. But now I have steps for you to
>> easily reproduce this:
>> - get Ubuntu 12.04 LTS Server 64 bit and install on some VM (VirtualBox)
>> - run the following commands:
>> sudo aptitude update
>> sudo aptitude safe-upgrade
>> sudo aptitude install openjdk-7-jdk unzip
>> wget http://buildr.apache.org/**scripts/install-linux.sh<http://buildr.apache.org/scripts/install-linux.sh>
>> export JAVA_HOME=/usr/lib/jvm/java-7-**openjdk-amd64
>> sudo env JAVA_HOME=$JAVA_HOME ./install.sh # fails in Ubuntu 12.04 LTS,
>> need to manually "gem update --system"
>> wget http://www.der-ingo.de/buildr-**fail.tgz<http://www.der-ingo.de/buildr-fail.tgz># get demo project
>> tar -xzf buildr-fail.tgz
>> cd buildr-fail
>> ./build.sh
>>
>> Things to note:
>> - www.der-ingo.de is my personal homepage, I am not a fan of file
>> sharing sites ;)
>> - The tgz contains pretty much the same project which I described in my
>> OP.
>> - The build.sh script runs buildr 20 times, extracts a PNG from the WAR,
>> logs the PNG file size to a file and then display all different lines of
>> the logfile
>> - You can enabled a traced build in the build.sh. Maybe you should put a
>> sleep 1 in the loop.
>>
>> If you run all that on Ubuntu Server 13.04, then everything works as
>> expected!
>>
>> So I hope this helps you track down the problem. Looks like it's hidden
>> somewhere in Ruby and not buildr.
>>
>> Any hint as to how to work around this problem is highly appreciated!
>>
>> Cheers, Ingo =;->
>>
>
>
Re: Problems with merging
Posted by Alex Boisvert <al...@gmail.com>.
Hi Ingo, just a quick update that I can reproduce the problem. Thanks for
providing this sample project. Looking into it ...
On Wed, May 29, 2013 at 9:59 AM, Ingo Schmidt <an...@der-ingo.de> wrote:
> Hi Alex,
>
> ok, bad news, the problem persists. But now I have steps for you to easily
> reproduce this:
> - get Ubuntu 12.04 LTS Server 64 bit and install on some VM (VirtualBox)
> - run the following commands:
> sudo aptitude update
> sudo aptitude safe-upgrade
> sudo aptitude install openjdk-7-jdk unzip
> wget http://buildr.apache.org/**scripts/install-linux.sh<http://buildr.apache.org/scripts/install-linux.sh>
> export JAVA_HOME=/usr/lib/jvm/java-7-**openjdk-amd64
> sudo env JAVA_HOME=$JAVA_HOME ./install.sh # fails in Ubuntu 12.04 LTS,
> need to manually "gem update --system"
> wget http://www.der-ingo.de/buildr-**fail.tgz<http://www.der-ingo.de/buildr-fail.tgz># get demo project
> tar -xzf buildr-fail.tgz
> cd buildr-fail
> ./build.sh
>
> Things to note:
> - www.der-ingo.de is my personal homepage, I am not a fan of file sharing
> sites ;)
> - The tgz contains pretty much the same project which I described in my OP.
> - The build.sh script runs buildr 20 times, extracts a PNG from the WAR,
> logs the PNG file size to a file and then display all different lines of
> the logfile
> - You can enabled a traced build in the build.sh. Maybe you should put a
> sleep 1 in the loop.
>
> If you run all that on Ubuntu Server 13.04, then everything works as
> expected!
>
> So I hope this helps you track down the problem. Looks like it's hidden
> somewhere in Ruby and not buildr.
>
> Any hint as to how to work around this problem is highly appreciated!
>
> Cheers, Ingo =;->
>
Re: Problems with merging
Posted by Ingo Schmidt <an...@der-ingo.de>.
Hi Alex,
ok, bad news, the problem persists. But now I have steps for you to
easily reproduce this:
- get Ubuntu 12.04 LTS Server 64 bit and install on some VM (VirtualBox)
- run the following commands:
sudo aptitude update
sudo aptitude safe-upgrade
sudo aptitude install openjdk-7-jdk unzip
wget http://buildr.apache.org/scripts/install-linux.sh
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
sudo env JAVA_HOME=$JAVA_HOME ./install.sh # fails in Ubuntu 12.04 LTS,
need to manually "gem update --system"
wget http://www.der-ingo.de/buildr-fail.tgz # get demo project
tar -xzf buildr-fail.tgz
cd buildr-fail
./build.sh
Things to note:
- www.der-ingo.de is my personal homepage, I am not a fan of file
sharing sites ;)
- The tgz contains pretty much the same project which I described in my OP.
- The build.sh script runs buildr 20 times, extracts a PNG from the WAR,
logs the PNG file size to a file and then display all different lines of
the logfile
- You can enabled a traced build in the build.sh. Maybe you should put a
sleep 1 in the loop.
If you run all that on Ubuntu Server 13.04, then everything works as
expected!
So I hope this helps you track down the problem. Looks like it's hidden
somewhere in Ruby and not buildr.
Any hint as to how to work around this problem is highly appreciated!
Cheers, Ingo =;->
Re: Problems with merging
Posted by Alex Boisvert <al...@gmail.com>.
Ah, thanks for letting me know. I was still at the stage of trying to
reproduce the issue (and reviewing the code to understand where it could
possibly happen).
alex
On Wed, May 29, 2013 at 7:59 AM, Ingo Schmidt <an...@der-ingo.de> wrote:
> Hi Alex,
>
> it looks like this is fixed with an update of rubygems. However, this
> introduces a lot of new problems for me. I will write a new mail for this.
>
> Anyway, with new rubygems, the problem is gone. All builds are always
> fine, no matter which version of buildr I use (1.4.7 vs. 1.4.12)
>
> So let's consider this problem as solved.
>
> Cheers, Ingo =;->
>
Re: Problems with merging
Posted by Ingo Schmidt <an...@der-ingo.de>.
Hi Alex,
it looks like this is fixed with an update of rubygems. However, this
introduces a lot of new problems for me. I will write a new mail for this.
Anyway, with new rubygems, the problem is gone. All builds are always
fine, no matter which version of buildr I use (1.4.7 vs. 1.4.12)
So let's consider this problem as solved.
Cheers, Ingo =;->
Re: Problems with merging
Posted by Alex Boisvert <al...@gmail.com>.
Hi Ingo,
I haven't had time to look into this today ... I suspect the
non-deterministic behavior is due to putting things in a hash map. Will
see if we can fix or if there's a workaround. Stay tuned.
alex
On Tue, May 21, 2013 at 5:25 AM, Ingo Schmidt <an...@der-ingo.de> wrote:
> Hi,
>
> once again I have a problem with merging.
>
> Consider the following extremely simple project layout:
> /config/images/logo.png
> /webshop-war/src/main/webapp/**images/logo.png
>
> And the buildfile looks like this:
> define "merge-fail" do
> project.version = "1.0.0"
> define "webshop-war" do
> images = zip(_(:target, images.zip)).include(project.**parent)
> war = package(:war)
> war.enhance([images]) # workaround for BUILDR-605
> war.path("images").merge(**images)
> end
> end
>
> When running "buildr clean package" I can observe the following behaviour:
> Sometimes the logo from the config folder is included in the WAR file,
> sometimes the one from webshop-war/images.
>
> I have not found out as to why this happens. Sometimes you get the same
> result 20 times in a row, so do run as often until you finally see this
> behaviour.
>
> Could someone explain me what this is?
>
> Any ideas for a workaround?
> What I would like to achieve is the following:
> If there is a picture in the config folder, use that one, otherwise use
> the default one inside webshop-war/images. What is the correct way of doing
> this? I am really stuck now...
>
>
>
> Cheers, Ingo =;->
>