Ant/Crosswalk woes

[Ian Clelland <ic...@chromium.org>]

tl;dr: I'm running into an issue with the Crosswalk webview project that, I
think, means we have to abandon our custom ant-build and ant-gen
directories.

Some time ago, we put in some override rules into cordova-android's Ant
build config. This was done because the defaults, "bin" and "gen", are also
used by Eclipse, and if you ran Eclipse and the command line tools on the
same project, they would step all over each others' build artifacts. The
only solution at the time was to do a full clean before each build, making
incremental (fast) builds impossible.

There's a problem, though -- it turns out that when the Android AAPT
packager tool collects all of the resources from the /res/ directories, it
is *hard coded* to put its artifacts in /bin/.

(Here's a fun line of code:
https://android.googlesource.com/platform/tools/base/+/master/legacy/ant-tasks/src/main/java/com/android/ant/ComputeDependencyTask.java#203
)

The result of this is that the project compiles, but without all of the
resources needed for a XWalk WebView. The app runs, but fails at run-time
to construct the view, and falls back to the system WebView.

This would also be a problem for any other library projects that use /res/
folders.

Turning off our custom build rules fixes this, but now we're back to the
old situation. I don't see another way to handle this, though.

I think I'm going to have to revert the commit that changed the build
directories, on the 4.0.x branch at least. I'll add an issue for supporting
Gradle as well; it shouldn't have this particular issue; and then at some
point in the future we can go back to supporting fast builds again.

Ian

Re: Ant/Crosswalk woes

[Ian Clelland <ic...@chromium.org>]

On Thu, Jun 5, 2014 at 1:02 PM, Michal Mocny <mm...@chromium.org> wrote:

> Dumb questions from someone who knows nothing about this:
>
> (a) Is it possible to share the same directories and still get incremental
> builds so long as we guard for previous build being done via the same
> method?  aka, could build steps be:
>
> if (!exists_file(last_build_was_$TYPE))
>   clean();
> build();
> create_file(last_build_was_$TYPE)
>
> I don't know enough about the Eclipse build process to say whether this is
possible. (Also, Eclipse has the nasty habit of building continuously if
you have it running)

I was considering just leaving out the automatic clean step entirely, and
telling people "If you are using Eclipse as well as the command line tools,
then you may experience problems building. If you do, then run a clean
before you build after you switch environments."


> (b) Is it possible to leave the separate paths, but change the build steps
> to swap directory locations just for the duration of the offending AAPT
> steps?
>

I don't think we can reliably do this; there could be multiple libraries in
an arbitrary project, each with an bin|ant-build and gen|ant-gen directory,
and the build steps are complex. I think that the aapt steps (there are a
couple) may use the right directories, but theres a 'compute dependencies'
task which gets it wrong. (It's a legit bug, I think -- there's a big FIXME
in the code there).

I'm open to anyone trying it, though ;)


> -Michal
>
>
> On Thu, Jun 5, 2014 at 11:51 AM, Ian Clelland <ic...@chromium.org>
> wrote:
>
> > tl;dr: I'm running into an issue with the Crosswalk webview project
> that, I
> > think, means we have to abandon our custom ant-build and ant-gen
> > directories.
> >
> > Some time ago, we put in some override rules into cordova-android's Ant
> > build config. This was done because the defaults, "bin" and "gen", are
> also
> > used by Eclipse, and if you ran Eclipse and the command line tools on the
> > same project, they would step all over each others' build artifacts. The
> > only solution at the time was to do a full clean before each build,
> making
> > incremental (fast) builds impossible.
> >
> > There's a problem, though -- it turns out that when the Android AAPT
> > packager tool collects all of the resources from the /res/ directories,
> it
> > is *hard coded* to put its artifacts in /bin/.
> >
> > (Here's a fun line of code:
> >
> >
> https://android.googlesource.com/platform/tools/base/+/master/legacy/ant-tasks/src/main/java/com/android/ant/ComputeDependencyTask.java#203
> > )
> >
> > The result of this is that the project compiles, but without all of the
> > resources needed for a XWalk WebView. The app runs, but fails at run-time
> > to construct the view, and falls back to the system WebView.
> >
> > This would also be a problem for any other library projects that use
> /res/
> > folders.
> >
> > Turning off our custom build rules fixes this, but now we're back to the
> > old situation. I don't see another way to handle this, though.
> >
> > I think I'm going to have to revert the commit that changed the build
> > directories, on the 4.0.x branch at least. I'll add an issue for
> supporting
> > Gradle as well; it shouldn't have this particular issue; and then at some
> > point in the future we can go back to supporting fast builds again.
> >
> > Ian
> >
>

Re: Ant/Crosswalk woes

[Michal Mocny <mm...@chromium.org>]

Dumb questions from someone who knows nothing about this:

(a) Is it possible to share the same directories and still get incremental
builds so long as we guard for previous build being done via the same
method?  aka, could build steps be:

if (!exists_file(last_build_was_$TYPE))
  clean();
build();
create_file(last_build_was_$TYPE)

(b) Is it possible to leave the separate paths, but change the build steps
to swap directory locations just for the duration of the offending AAPT
steps?

-Michal


On Thu, Jun 5, 2014 at 11:51 AM, Ian Clelland <ic...@chromium.org>
wrote:

> tl;dr: I'm running into an issue with the Crosswalk webview project that, I
> think, means we have to abandon our custom ant-build and ant-gen
> directories.
>
> Some time ago, we put in some override rules into cordova-android's Ant
> build config. This was done because the defaults, "bin" and "gen", are also
> used by Eclipse, and if you ran Eclipse and the command line tools on the
> same project, they would step all over each others' build artifacts. The
> only solution at the time was to do a full clean before each build, making
> incremental (fast) builds impossible.
>
> There's a problem, though -- it turns out that when the Android AAPT
> packager tool collects all of the resources from the /res/ directories, it
> is *hard coded* to put its artifacts in /bin/.
>
> (Here's a fun line of code:
>
> https://android.googlesource.com/platform/tools/base/+/master/legacy/ant-tasks/src/main/java/com/android/ant/ComputeDependencyTask.java#203
> )
>
> The result of this is that the project compiles, but without all of the
> resources needed for a XWalk WebView. The app runs, but fails at run-time
> to construct the view, and falls back to the system WebView.
>
> This would also be a problem for any other library projects that use /res/
> folders.
>
> Turning off our custom build rules fixes this, but now we're back to the
> old situation. I don't see another way to handle this, though.
>
> I think I'm going to have to revert the commit that changed the build
> directories, on the 4.0.x branch at least. I'll add an issue for supporting
> Gradle as well; it shouldn't have this particular issue; and then at some
> point in the future we can go back to supporting fast builds again.
>
> Ian
>

Re: Ant/Crosswalk woes

[Frederico Galvão <fr...@pontoget.com.br>]

If I may have a word on this:

I fought for quite a few weeks before I could get my project to a working
state for both CLI and my IDE (Intellij IDEA).
Considering what my final solution was and development workflow has been
since then, I could say that for me the CLI working is much more important
right now, only because no IDE can handle the whole cordova development
workflow on it's own right now, which makes me have to deal with CLI sooner
or later. With that said, I'd rather have all my dev workflow based on the
CLI right now than have to deal with any slow solution on either path.

Also, considering my case with IntellijIDEA, I was able to bring almost
everything from the CLI I need into the IDE itself, even more now that they
have an embeded terminal tab which solves everything I eventually couldn't
solve directly on the IDE. In the end, I use CLI for "prepare" and Intellij
IDEA to build, run to a device from the IDE, be able to change native code
(xmls and drawables mainly), and watch it on git (this brings us into
another conversation about the official recomendation of what to add and
what to ignore on git for a cordova project, but I'd leave it for another
day). This has been the best workflow so far, and incremental build is a
key piece in all of this. Actually, as a side note, I don't know why but
Intellij IDEA "run" process on top of the android shell is much much faster
than cordova's.

I'd disconsider every option that throws incremental build away unless it's
the last solution on earth. Having to wait more than 15 seconds everytime I
have to test a new javascript function or css rule is a killer, and doesn't
fit on the cordova way of life. Add at least the FacebookLib into your
project and this jumps to 25 seconds easily on some machines.

I can't say much about Eclipse and how much you could integrate node
scripts or terminal commands into it, but I'd say just dump support for
build on both paths at the same time (which was the root issue that created
the ant-gen and ant-bin) and focus on making the CLI way perfect and fast,
which would make a "simple/clean/*fast*" solution, instead of the
"simple/clean/*slow*" mentioned. Even then, if anyone want to keep on the
IDE itself (like me, using IDE to build and run after a cordova prepare),
it'll still work as long as they don't mix CLI and IDE builds together.

And @JoeBowser: How often do people switch tools? I don't, anymore, since I
realized I could bring the CLI into my IDE. I use both instead, but not at
the same time in the same step. My whole process is one of either:

1- change web code (Any text editor) -> gulp trigger resource preprocessing
on to the www folder (CLI) -> gulp trigger "cordova prepare" (CLI) ->
android compilation and apk build using the default gen/bin folders
(IntellijIDEA) -> run to device (IntellijIDEA).
2- change web code (Any text editor) -> gulp trigger resource preprocessing
on to the www folder (CLI) -> gulp trigger "cordova prepare" (CLI) -> F5 on
browser ripple emulator.


And to make sure it's clear, the only reason I still use Intellij IDEA to
build and run the APK is because it's faster than cordova CLI.


2014-06-06 12:04 GMT-03:00 Marcel Kinard <cm...@gmail.com>:

> Perhaps this is an oversimplification, but if the choice is between
> simple/clean/slow versus complex/fragile/fast, I’d prefer the former.




-- 

*Frederico Galvão*

Diretor de Tecnologia

PontoGet Inovação Web


( +55(62) 8131-5720

* www.pontoget.com.br <http://www.pontoget.com/>

Re: Ant/Crosswalk woes

[Marcel Kinard <cm...@gmail.com>]

Perhaps this is an oversimplification, but if the choice is between simple/clean/slow versus complex/fragile/fast, I’d prefer the former.

Re: Ant/Crosswalk woes

[Joe Bowser <bo...@gmail.com>]

On Thu, Jun 5, 2014 at 10:37 AM, Ian Clelland <ic...@chromium.org> wrote:
> On Thu, Jun 5, 2014 at 1:19 PM, Joe Bowser <bo...@gmail.com> wrote:
>
>> On Thu, Jun 5, 2014 at 8:51 AM, Ian Clelland <ic...@chromium.org>
>> wrote:
>> > tl;dr: I'm running into an issue with the Crosswalk webview project
>> that, I
>> > think, means we have to abandon our custom ant-build and ant-gen
>> > directories.
>> >
>>
>> I'm totally cool with this.
>>
>>
>> > Some time ago, we put in some override rules into cordova-android's Ant
>> > build config. This was done because the defaults, "bin" and "gen", are
>> also
>> > used by Eclipse, and if you ran Eclipse and the command line tools on the
>> > same project, they would step all over each others' build artifacts. The
>> > only solution at the time was to do a full clean before each build,
>> making
>> > incremental (fast) builds impossible.
>> >
>>
>> So, this feature existed because people want Eclipse to auto-build?
>> I've turned off that "feature" in Eclipse years ago, and I've never
>> been happier.  Sure, it means that I have to actually decide when I
>> want to build in Eclipse to see my errors, but I actually LIKE that,
>> because it's useless to see that I broke things when a change is not
>> even halfway written.
>>
>
> Auto builds aren't the root of the problem, but they certainly don't help :)
>
> I think that the problem was that ant writes its output to bin/ and gen/,
> and Eclipse does the same (whether it's an auto build or not), and if
> there's something in those directories, they'll do an incremental build,
> which will be broken. When you switch tools, you have to do a clean before
> you build.
>

That sounds like practicing good hygiene.  How often do people switch
tools?  I tend to stay in Eclipse once I'm in Eclipse, and stay using
Ant if I'm lucky enough to not need to go into Eclipse (almost all my
demo apps aren't Eclipse based, because it makes for a better demo).

>
>>
>> > There's a problem, though -- it turns out that when the Android AAPT
>> > packager tool collects all of the resources from the /res/ directories,
>> it
>> > is *hard coded* to put its artifacts in /bin/.
>> >
>> > (Here's a fun line of code:
>> >
>> https://android.googlesource.com/platform/tools/base/+/master/legacy/ant-tasks/src/main/java/com/android/ant/ComputeDependencyTask.java#203
>> > )
>> >
>> > The result of this is that the project compiles, but without all of the
>> > resources needed for a XWalk WebView. The app runs, but fails at run-time
>> > to construct the view, and falls back to the system WebView.
>> >
>> > This would also be a problem for any other library projects that use
>> /res/
>> > folders.
>> >
>>
>> So, pretty much any modern Android project.
>>
>> > Turning off our custom build rules fixes this, but now we're back to the
>> > old situation. I don't see another way to handle this, though.
>> >
>> > I think I'm going to have to revert the commit that changed the build
>> > directories, on the 4.0.x branch at least. I'll add an issue for
>> supporting
>> > Gradle as well; it shouldn't have this particular issue; and then at some
>> > point in the future we can go back to supporting fast builds again.
>> >
>>
>> 1. Can't you can do incremental builds in bin and gen by unchecking
>> "Build Automatically",
>
> Nope (I don't think so, at least)
>
> or by not using Eclipse and only using Ant?
>>
> Yes. Or vice versa.
>
>
>> 2. We may want to move to Gradle
>>
> Yes. Looks like you filed CB-3445 back when 2.8.0 was the new thing :) I
> think we still want to do it.
>
> 3. Ant vs Gradle is an either/or thing, and I'm not looking forward to
>> the move, but I feel like it's closer than we think.
>>
> Agreed.
>
> Hopefully we can make it mostly transparent to users, and "cordova build
> android" can hide most of the changes.

Well, Android just allowed for command-line Gradle projects in their
tools recently.  The problem is that an Android project either uses
Gradle or Ant, and there's no middle ground.  Honestly, I really think
we need both qualitiative (user stories) and quantitative (survey)
feedback with how people do dev with iOS and Android and the CLI.  The
promise of the CLI was that there'd be no IDE, but that's obviously
bullshit, since people are using IDEs with the damn thing, and on
Android we have to pick which one makes more sense.

I didn't want to hijack the thread with Gradle vs Ant, other than to
point out that it might make this less relevant in the near future. :S

Re: Ant/Crosswalk woes

[Ian Clelland <ic...@chromium.org>]

On Thu, Jun 5, 2014 at 1:19 PM, Joe Bowser <bo...@gmail.com> wrote:

> On Thu, Jun 5, 2014 at 8:51 AM, Ian Clelland <ic...@chromium.org>
> wrote:
> > tl;dr: I'm running into an issue with the Crosswalk webview project
> that, I
> > think, means we have to abandon our custom ant-build and ant-gen
> > directories.
> >
>
> I'm totally cool with this.
>
>
> > Some time ago, we put in some override rules into cordova-android's Ant
> > build config. This was done because the defaults, "bin" and "gen", are
> also
> > used by Eclipse, and if you ran Eclipse and the command line tools on the
> > same project, they would step all over each others' build artifacts. The
> > only solution at the time was to do a full clean before each build,
> making
> > incremental (fast) builds impossible.
> >
>
> So, this feature existed because people want Eclipse to auto-build?
> I've turned off that "feature" in Eclipse years ago, and I've never
> been happier.  Sure, it means that I have to actually decide when I
> want to build in Eclipse to see my errors, but I actually LIKE that,
> because it's useless to see that I broke things when a change is not
> even halfway written.
>

Auto builds aren't the root of the problem, but they certainly don't help :)

I think that the problem was that ant writes its output to bin/ and gen/,
and Eclipse does the same (whether it's an auto build or not), and if
there's something in those directories, they'll do an incremental build,
which will be broken. When you switch tools, you have to do a clean before
you build.


>
> > There's a problem, though -- it turns out that when the Android AAPT
> > packager tool collects all of the resources from the /res/ directories,
> it
> > is *hard coded* to put its artifacts in /bin/.
> >
> > (Here's a fun line of code:
> >
> https://android.googlesource.com/platform/tools/base/+/master/legacy/ant-tasks/src/main/java/com/android/ant/ComputeDependencyTask.java#203
> > )
> >
> > The result of this is that the project compiles, but without all of the
> > resources needed for a XWalk WebView. The app runs, but fails at run-time
> > to construct the view, and falls back to the system WebView.
> >
> > This would also be a problem for any other library projects that use
> /res/
> > folders.
> >
>
> So, pretty much any modern Android project.
>
> > Turning off our custom build rules fixes this, but now we're back to the
> > old situation. I don't see another way to handle this, though.
> >
> > I think I'm going to have to revert the commit that changed the build
> > directories, on the 4.0.x branch at least. I'll add an issue for
> supporting
> > Gradle as well; it shouldn't have this particular issue; and then at some
> > point in the future we can go back to supporting fast builds again.
> >
>
> 1. Can't you can do incremental builds in bin and gen by unchecking
> "Build Automatically",

Nope (I don't think so, at least)

or by not using Eclipse and only using Ant?
>
Yes. Or vice versa.


> 2. We may want to move to Gradle
>
Yes. Looks like you filed CB-3445 back when 2.8.0 was the new thing :) I
think we still want to do it.

3. Ant vs Gradle is an either/or thing, and I'm not looking forward to
> the move, but I feel like it's closer than we think.
>
Agreed.

Hopefully we can make it mostly transparent to users, and "cordova build
android" can hide most of the changes.

Re: Ant/Crosswalk woes

[Joe Bowser <bo...@gmail.com>]

On Thu, Jun 5, 2014 at 8:51 AM, Ian Clelland <ic...@chromium.org> wrote:
> tl;dr: I'm running into an issue with the Crosswalk webview project that, I
> think, means we have to abandon our custom ant-build and ant-gen
> directories.
>

I'm totally cool with this.


> Some time ago, we put in some override rules into cordova-android's Ant
> build config. This was done because the defaults, "bin" and "gen", are also
> used by Eclipse, and if you ran Eclipse and the command line tools on the
> same project, they would step all over each others' build artifacts. The
> only solution at the time was to do a full clean before each build, making
> incremental (fast) builds impossible.
>

So, this feature existed because people want Eclipse to auto-build?
I've turned off that "feature" in Eclipse years ago, and I've never
been happier.  Sure, it means that I have to actually decide when I
want to build in Eclipse to see my errors, but I actually LIKE that,
because it's useless to see that I broke things when a change is not
even halfway written.

> There's a problem, though -- it turns out that when the Android AAPT
> packager tool collects all of the resources from the /res/ directories, it
> is *hard coded* to put its artifacts in /bin/.
>
> (Here's a fun line of code:
> https://android.googlesource.com/platform/tools/base/+/master/legacy/ant-tasks/src/main/java/com/android/ant/ComputeDependencyTask.java#203
> )
>
> The result of this is that the project compiles, but without all of the
> resources needed for a XWalk WebView. The app runs, but fails at run-time
> to construct the view, and falls back to the system WebView.
>
> This would also be a problem for any other library projects that use /res/
> folders.
>

So, pretty much any modern Android project.

> Turning off our custom build rules fixes this, but now we're back to the
> old situation. I don't see another way to handle this, though.
>
> I think I'm going to have to revert the commit that changed the build
> directories, on the 4.0.x branch at least. I'll add an issue for supporting
> Gradle as well; it shouldn't have this particular issue; and then at some
> point in the future we can go back to supporting fast builds again.
>

1. Can't you can do incremental builds in bin and gen by unchecking
"Build Automatically", or by not using Eclipse and only using Ant?
2. We may want to move to Gradle
3. Ant vs Gradle is an either/or thing, and I'm not looking forward to
the move, but I feel like it's closer than we think.