You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Stefan Bodewig <bo...@apache.org> on 2001/05/02 14:23:50 UTC
Filter and Filtersets (was SQLExec and Properties)
Michael McCallum <gh...@xtra.co.nz> wrote:
> On Wednesday 02 May 2001 08:31, you wrote:
>> Michael McCallum <gh...@xtra.co.nz> wrote:
>> > The sql patch does not do anything execpt add a call to
>> > filterset.replaceTokens( sql )
>>
>> Why use filters instead of property substitution for this?
>
> Because the substitution occurs for all sql statements even those
> coming from file.
Why couldn't you do this with properties? I think we are consistently
using property expansions except when copying files and I'd like to
keep it this way.
When you invoke filterset.replaceTokens(sql), how is this different
from ProjectHelper.replaceProperties(project, sql, project.getProperties())
- from the point of functionality, you are replacing one pattern with another.
I guess, I'm missing something obvious.
> I do not believe that property replacement should be carried out on
> file loaded sql.
Why not? To me filters are restricted to the filtered copying of
files - and I'm even thinking about removing them for this purpose as
well, like
<copy ...>
<filterset>
<property name="prop1" />
<property name="prop2" />
</filterset>
</copy>
where each occurrence of @prop1@ would be replaced with the expanded
${prop1}. This is just a random thought.
Here we are getting at the point where my main concern with your
filterset patch originates from. We have decided to give people more
control over the filtering that happens when copying files - we've not
decided how we would do that yet.
If we add filtersets in Ant 1.4 but decide to drop filters completely,
change the concept in some other way, we are going to confuse users
more than we need to do. I'd simply prefer to see how we are going to
change it in Ant2 - and I'd like to take your filterset concept as one
proposal to achieve it.
>> > I could split it all up into patches for Project, copy,
>> > DefaultCompilerAdapter, Filter, SQLExec etc if you would like.
I've just browsed over the original patch to find out what you've been
doing with <javac> here (as filtering has gone away with the copying
of support files in Ant 1.2).
You want to apply filtering to the output from javac? Why? I'd
prefer to see what an error message really looks like 8-)
Stefan
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Daniel Barclay <Da...@digitalfocus.com>.
Michael McCallum wrote:
>
....
> What I meant to say is that the complexity is not increased significantly by allowing the setting
> of the token symbols. :)
>
> Its not multiple kinds of token replacement unless you allow chained filtersets.
> Otherwise its only one but you can set the token symbols.
Oh. I though you were opening the door to more complex token parsing
(more complex than just setting the start and end strings).
Daniel
--
Daniel Barclay
Digital Focus
Daniel.Barclay@digitalfocus.com
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Michael McCallum <mi...@spinsoftware.com>.
On 8 May 2001, at 18:02, Daniel Barclay wrote:
> Michael McCallum wrote:
> > ...
> > And there is no reason to restrict the token replacement to being between @ symbols.
> > It does not simplify anything is just restricts flexiblity.
>
> You can't really be saying that restricting Ant's token replacement
> doesn't simplify anything, because an Ant that does one kind of token
> replacement is clearly simpler than an Ant that does multiple kinds
> of token replacement.
>
> What did you mean to say?
What I meant to say is that the complexity is not increased significantly by allowing the setting
of the token symbols. :)
Its not multiple kinds of token replacement unless you allow chained filtersets.
Otherwise its only one but you can set the token symbols.
Michael
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Daniel Barclay <Da...@digitalfocus.com>.
Michael McCallum wrote:
> ...
> And there is no reason to restrict the token replacement to being between @ symbols.
> It does not simplify anything is just restricts flexiblity.
You can't really be saying that restricting Ant's token replacement
doesn't simplify anything, because an Ant that does one kind of token
replacement is clearly simpler than an Ant that does multiple kinds
of token replacement.
What did you mean to say?
Daniel
--
Daniel Barclay
Digital Focus
Daniel.Barclay@digitalfocus.com
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Michael McCallum <mi...@spinsoftware.com>.
Ant is a build tool. As such it brings together all the components of a software project readying it
for deployment.
For ease of development many tools will be used for different components of the project.
Where possible Ant should be able to work alongside these so that we can use tool X and tool Y
to development pieces of a project and then tie them all together with Ant.
Here is a use case:
I need to create database schema for oracle. So I have a bunch of sql scripts which are run with
sqlplus when developing. These often require parameters so there are sqlplus tokens in the files.
For these to be run in an automated manner for deployment it is necessary to do the token
replacement.
IMO it is Ants role to be flexible enough to fit in with tools such as sql plus.
And there is no reason to restrict the token replacement to being between @ symbols.
It does not simplify anything is just restricts flexiblity.
Michael
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Daniel Barclay <Da...@digitalfocus.com>.
Michael McCallum wrote:
>
...
> Yes. Ant needs to be flexible so it can complement tools that are out there.
> So if tool X needs to do token replacement with Tokens wrapped in @, $, % or even
> START/END then it should be supported in some way.
Why would Ant need to do token replacement in the style of tool X?
Wouldn't you put Ant-style tokens in a file, replace them using Ant,
and then have a X-style file to be processed by tool X?
Or do you really want Ant to be able to process tokens that tool X
could have processed?
Daniel
--
Daniel Barclay
Digital Focus
Daniel.Barclay@digitalfocus.com
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Michael McCallum <mi...@spinsoftware.com>.
On 3 May 2001, at 13:03, Stefan Bodewig wrote:
> Michael McCallum <mi...@spinsoftware.com> wrote:
>
> > It is not enough either to just use expansion of ${foobar} it has to
> > be flexible. ie need to be able to replace @foobar@ and #foobar# and
> > anything else you might consider.
>
> Where do these @foobar@ things come from? I.e. why can't they be
> ${foobar}?
>
> >> > I do not believe that property replacement should be carried out
> >> > on file loaded sql.
> > Yes it should it is absolutely necessary when running the scripts
> > using other tools like sqlplus they have their own file substitution
> > patterns that ant needs to be flexible enough to work with.
>
> I don't understand this, sorry. You say, you have a tool that
> performs some substitution that follows either @foobar@ or #foobar#
> convention (but not consistently) and want Ant to adjust to this tool,
> right? Excuse me, if this is a dumb question, I've never used sqlplus
sqlplus uses &foobar. and &&foobar.
> - I'm a DB2 guy with some occasional project using Interbase or MS SQL
> Server.
Yes. Ant needs to be flexible so it can complement tools that are out there.
So if tool X needs to do token replacement with Tokens wrapped in @, $, % or even
START/END then it should be supported in some way.
>
> > I guess what Im saying is the the use of a token replacement
> > mechanism should be standard. Which we could use properties for (I
> > just thought that the filter task was hackish so improved it).
>
> Agreed on everything you've said (including the "filter was hackish"
> part 8-).
>
> >> If we add filtersets in Ant 1.4 but decide to drop filters
> >> completely, change the concept in some other way, we are going to
> >> confuse users more than we need to do. I'd simply prefer to see
> >> how we are going to change it in Ant2 - and I'd like to take your
> >> filterset concept as one proposal to achieve it.
>
> > One thing I cant stand is spoon feeding.
>
> Sorry, my English language parser has left me - "untranslatable metaphor"
> 8-)
When you provide all the solutions to someone and they dont have to think. And usually as
a result they dont think in the future instead try to regurgitate what they were spoon fed and
funnily enough its not the right answer to the problem they now face.
Clear as mud? :)
> > Plus you have a solution that may not be perfect but people will get
> > to try it out and give feedback. So you can either improve the
> > concept or discard it for Ant2. But sitting the code on the fence
> > does not achieve anything except we sit around and discuss it merits
> > and then the users use it in completely unexpected way later.
>
> I guess I see us much closer to agree on how we want something to work
> in Ant2 than you do. I expect this to happen in a few weeks (prior to
> an Ant 1.4 release), but I may be wrong. I'll be pushing for it.
What im saying is that the filtering was a hack and better solution has not been trialed
so all the agreement may mean nothing if the users dont see it in the same way.
regards
Michael
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Diane Holt <ho...@yahoo.com>.
--- Stefan Bodewig <bo...@apache.org> wrote:
> Michael McCallum <mi...@spinsoftware.com> wrote:
> > One thing I cant stand is spoon feeding.
>
> Sorry, my English language parser has left me - "untranslatable
> metaphor"
> 8-)
spoon-feed:
TRANSITIVE VERB : Inflected forms: �fed (-fd��) , �feed�ing, �feeds
1. To feed (another) with a spoon. 2. To treat (another) in a way that
discourages independent thought or action, as by overindulgence. 3. a. To
provide (another) with knowledge or information in an oversimplified way.
b. To provide (knowledge or information) in an oversimplified way.
Diane
=====
(holtdl@yahoo.com)
__________________________________________________
Do You Yahoo!?
Yahoo! Auctions - buy the things you want at great prices
http://auctions.yahoo.com/
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Michael McCallum <mi...@spinsoftware.com>.
Should have read all my mail before I started responding. :)
On 3 May 2001, at 15:47, Stefan Bodewig wrote:
> Jose Alberto Fernandez <j_...@yahoo.com> wrote:
>
> > Here we are starting with the premise that you have SQL files for a
> > particular 3rd party tool, that you want to incorporate in your
> > build. Some of them may be yours to modify, but others may have
> > been provided to you by others.
>
> And you need to replace @foo@ or &&foo with something meaningful
> inside Ant as the scripting tool of your RDBMS would do, right? OK.
>
> The filter mechanism wouldn't work for &&foo anyway, would it?
With my changes it does.
>
> OK, I see that <sql> could and should be improved to support something
> like this, but don't think we should couple it to filters at all.
The way I see its the filters are a set of name/value pairs (just likeproperties as you pointed out)
that you use for substitution.
When doing a copy you use a the name values pairs to replace tokens.
When you read the sql file you use the name value pairs to replace tokens I dont see a
difference.
> I agree that simple ${} expansion is not enough, thank you and Michael
> for explaining it to me.
You are welcome. :)
If the was a way to group properties into sets. Then it would be easy to adapt the filterset
interface to use property sets. So then there would be only one mechanism for producing the
name/value pairs just several for using them.
I might give a little background on the problem I solved to help clarify.
I have a build system that compiles all our projects and packages them up into tar gz's.
This includes jars, html, css, docs, and deployment tools.
It also creates from template the install scripts to install the project once its on site (ie copied to
the appropriate server.)
Two of these product are themselves templates for creating instances of applications.
There is heaps of configuration stuff that gets installed and this all needs filtering but with
separate sets of filters. Mostly defined in separate properties files.
So I have a script or two full of copy's and sql commands with the filtering.
It used to be achieved with perl scripts but it was simpler for me to show people how to use ant
scripts than how to read/write perl :)
I think I rambling now so am going to do some work( I can ramble in my work instead of on this
list :).
regards
Michael
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Stefan Bodewig <bo...@apache.org>.
Jose Alberto Fernandez <j_...@yahoo.com> wrote:
> Here we are starting with the premise that you have SQL files for a
> particular 3rd party tool, that you want to incorporate in your
> build. Some of them may be yours to modify, but others may have
> been provided to you by others.
And you need to replace @foo@ or &&foo with something meaningful
inside Ant as the scripting tool of your RDBMS would do, right? OK.
The filter mechanism wouldn't work for &&foo anyway, would it?
OK, I see that <sql> could and should be improved to support something
like this, but don't think we should couple it to filters at all.
I agree that simple ${} expansion is not enough, thank you and Michael
for explaining it to me.
Stefan
RE: Filter and Filtersets (was SQLExec and Properties)
Posted by Jose Alberto Fernandez <j_...@yahoo.com>.
OK, I will try to explain some of the issues here.
> From: Stefan Bodewig [mailto:bodewig@apache.org]
>
> Michael McCallum <mi...@spinsoftware.com> wrote:
>
> > It is not enough either to just use expansion of ${foobar} it has to
> > be flexible. ie need to be able to replace @foobar@ and #foobar# and
> > anything else you might consider.
>
> Where do these @foobar@ things come from? I.e. why can't they be
> ${foobar}?
>
The main point here is to be able to use, on the SQL task, SQL script files
that can work using the SQL shell scripts interpreters of major RDBMS
providers. (like MSSQL, ORACLE, etc.).
All of them use SQL as the basis for their language, but they all have their
own conventions about parameter passing in the scripts.
If we want to maintain just ONE set of files, we need to be able to pass
parameters to those scripts from ANT and we will need to be able to do
parameter substitution based on the particulars of the file syntax.
So since every sqltool has its own consistent way one needs something
flexible enough to support all of them. Hence the need to support &&foo,
&foo, @foo@, #foo#, or whatever else is needed.
> >> > I do not believe that property replacement should be carried out
> >> > on file loaded sql.
> > Yes it should it is absolutely necessary when running the scripts
> > using other tools like sqlplus they have their own file substitution
> > patterns that ant needs to be flexible enough to work with.
>
> I don't understand this, sorry. You say, you have a tool that
> performs some substitution that follows either @foobar@ or #foobar#
> convention (but not consistently) and want Ant to adjust to this tool,
> right? Excuse me, if this is a dumb question, I've never used sqlplus
> - I'm a DB2 guy with some occasional project using Interbase or MS SQL
> Server.
>
Each tool is consistent on its own, but the task needs to be flexible enough
to allows for different users to use files consistent with different tools.
The point here is that you need a task that is flexible enough.
This may not be the only approach, I grant you that.
Here we are starting with the premise that you have SQL files for a
particular 3rd party tool, that you want to incorporate in your build.
Some of them may be yours to modify, but others may have been provided to
you by others.
We could say, NO, the correct approach is to start with files specific for
the ANT SQL task, (i.e., using ${foo} syntax) and then use some replacement
task to generate files specific to a particular tool. But, what do we do
with files provided by third parties? Do we need to maintain our own copies?
And how do we do the pattern replacement? The last I saw, replacement tokens
have a syntax that is incompatible with ${foo}.
So, what is for us SQL users to do?
> > Plus you have a solution that may not be perfect but people will get
> > to try it out and give feedback. So you can either improve the
> > concept or discard it for Ant2. But sitting the code on the fence
> > does not achieve anything except we sit around and discuss it merits
> > and then the users use it in completely unexpected way later.
>
Can your changes define a task of its own? Or do they affect the core of
ANT? Why can't your changes be put as a new task on its own? A subclass of
SqlExec.java, something like InprovedSqlExec.java and let people try it.
After all, I can always replace one definition with another by just using
<taskdef name"sqlexec" classname="....InprovedSqlExec" />.
Jose Alberto
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Stefan Bodewig <bo...@apache.org>.
Michael McCallum <mi...@spinsoftware.com> wrote:
> It is not enough either to just use expansion of ${foobar} it has to
> be flexible. ie need to be able to replace @foobar@ and #foobar# and
> anything else you might consider.
Where do these @foobar@ things come from? I.e. why can't they be
${foobar}?
> On 2 May 2001, at 14:23, Stefan Bodewig wrote:
>> > I do not believe that property replacement should be carried out
>> > on file loaded sql.
> Yes it should it is absolutely necessary when running the scripts
> using other tools like sqlplus they have their own file substitution
> patterns that ant needs to be flexible enough to work with.
I don't understand this, sorry. You say, you have a tool that
performs some substitution that follows either @foobar@ or #foobar#
convention (but not consistently) and want Ant to adjust to this tool,
right? Excuse me, if this is a dumb question, I've never used sqlplus
- I'm a DB2 guy with some occasional project using Interbase or MS SQL
Server.
> I guess what Im saying is the the use of a token replacement
> mechanism should be standard. Which we could use properties for (I
> just thought that the filter task was hackish so improved it).
Agreed on everything you've said (including the "filter was hackish"
part 8-).
>> If we add filtersets in Ant 1.4 but decide to drop filters
>> completely, change the concept in some other way, we are going to
>> confuse users more than we need to do. I'd simply prefer to see
>> how we are going to change it in Ant2 - and I'd like to take your
>> filterset concept as one proposal to achieve it.
> One thing I cant stand is spoon feeding.
Sorry, my English language parser has left me - "untranslatable metaphor"
8-)
> Plus you have a solution that may not be perfect but people will get
> to try it out and give feedback. So you can either improve the
> concept or discard it for Ant2. But sitting the code on the fence
> does not achieve anything except we sit around and discuss it merits
> and then the users use it in completely unexpected way later.
I guess I see us much closer to agree on how we want something to work
in Ant2 than you do. I expect this to happen in a few weeks (prior to
an Ant 1.4 release), but I may be wrong. I'll be pushing for it.
> if I copy files from root /usr/local/src/foobar to
> /usr/local/build/foobar and then there are compile errors all ides i
> have seen will let me easily open the file in
> /usr/local/build/foobar and not the file in /usr/local/src/foobar.
Use case understood, thanks.
Stefan
Re: Filter and Filtersets (was SQLExec and Properties)
Posted by Michael McCallum <mi...@spinsoftware.com>.
On 2 May 2001, at 14:23, Stefan Bodewig wrote:
> Michael McCallum <gh...@xtra.co.nz> wrote:
>
> > On Wednesday 02 May 2001 08:31, you wrote:
> >> Michael McCallum <gh...@xtra.co.nz> wrote:
>
> >> > The sql patch does not do anything execpt add a call to
> >> > filterset.replaceTokens( sql )
> >>
> >> Why use filters instead of property substitution for this?
> >
> > Because the substitution occurs for all sql statements even those
> > coming from file.
>
> Why couldn't you do this with properties? I think we are consistently
> using property expansions except when copying files and I'd like to
> keep it this way.
I see what you are saying. But you still need some way of specifying which set of properties you
wish to use. Just saying do a property expansion is not enough as i may want ot expand X
tokens in file A and X + C tokens in file B.
It is not enough either to just use expansion of ${foobar} it has to be flexible. ie need to be able
to replace @foobar@ and #foobar# and anything else you might consider.
>
> When you invoke filterset.replaceTokens(sql), how is this different
> from ProjectHelper.replaceProperties(project, sql, project.getProperties())
> - from the point of functionality, you are replacing one pattern with another.
>
> I guess, I'm missing something obvious.
no good point.
>
> > I do not believe that property replacement should be carried out on
> > file loaded sql.
Yes it should it is absolutely necessary when running the scripts using other tools like sqlplus
they have their own file substitution patterns that ant needs to be flexible enough to work with.
>
> Why not? To me filters are restricted to the filtered copying of
> files - and I'm even thinking about removing them for this purpose as
> well, like
I guess what Im saying is the the use of a token replacement mechanism should be standard.
Which we could use properties for (I just thought that the filter task was hackish so improved it).
Using property sets in a token replacement mechanism would be fine.
>
> <copy ...>
> <filterset>
> <property name="prop1" />
> <property name="prop2" />
> </filterset>
> </copy>
>
> where each occurrence of @prop1@ would be replaced with the expanded
> ${prop1}. This is just a random thought.
Thats fine, as long as the power is there. ie must have the ability to specifu @ and @.
You would need to be able to group all filters that came from a file as one set as well.
>
> Here we are getting at the point where my main concern with your
> filterset patch originates from. We have decided to give people more
> control over the filtering that happens when copying files - we've not
> decided how we would do that yet.
>
> If we add filtersets in Ant 1.4 but decide to drop filters completely,
> change the concept in some other way, we are going to confuse users
> more than we need to do. I'd simply prefer to see how we are going to
> change it in Ant2 - and I'd like to take your filterset concept as one
> proposal to achieve it.
I think that the users have the idea that they will need to do a little bit of rethinking before they
move to Ant 2. I always believe that too much information can never hurt and if they have to
backtrack a little or find something works the same but is now called something different they
will get there and benefit from finding the way.
One thing I cant stand is spoon feeding. I require that people always think about the process
and if a little confusion makes them think a little more next time well thats all good.
Plus you have a solution that may not be perfect but people will get to try it out and give
feedback. So you can either improve the concept or discard it for Ant2. But sitting the code on
the fence does not achieve anything except we sit around and discuss it merits and then the
users use it in completely unexpected way later.
>
> >> > I could split it all up into patches for Project, copy,
> >> > DefaultCompilerAdapter, Filter, SQLExec etc if you would like.
>
> I've just browsed over the original patch to find out what you've been
> doing with <javac> here (as filtering has gone away with the copying
> of support files in Ant 1.2).
>
> You want to apply filtering to the output from javac? Why? I'd
> prefer to see what an error message really looks like 8-)
Because if I copy files from root /usr/local/src/foobar to /usr/local/build/foobar and then there are
compile errors all ides i have seen will let me easily open the file in /usr/local/build/foobar and
not the file in /usr/local/src/foobar.
So i made a replacement of the /usr/local/build with /usr/local/src/ possible with the filtersets.
Ok its hacky but it solved my problem. Anyone trying to build a subset of a source tree will have
problems like I did with compilers dragging in sources not specfied for compilation and without
dependencies. (just decided they were in a package so should be compiled as well. I have seen
javac and jikes do this.)
Michael