You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Olivier Lamy <ol...@apache.org> on 2011/11/27 16:18:25 UTC
Re: svn commit: r1206698 - in /incubator/sqoop/trunk: pom.xml src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
IMHO I think it will be better and more maintenable to use filtering
stuff from maven and generate a .properties file with the needed infos
instead of modifying a .java file.
The SqoopVersion class could read this .properties file.
And will make the build pass on windauze by example :-)
2011/11/27 <ja...@apache.org>:
> Author: jarcec
> Date: Sun Nov 27 11:12:37 2011
> New Revision: 1206698
>
> URL: http://svn.apache.org/viewvc?rev=1206698&view=rev
> Log:
> SQOOP-392. Build using maven is broken due to missing class generated by script write-version-info.sh
>
> (Arvind Prabhakar via Jarek Jarcec Cecho)
>
>
> Modified:
> incubator/sqoop/trunk/pom.xml
> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>
> Modified: incubator/sqoop/trunk/pom.xml
> URL: http://svn.apache.org/viewvc/incubator/sqoop/trunk/pom.xml?rev=1206698&r1=1206697&r2=1206698&view=diff
> ==============================================================================
> --- incubator/sqoop/trunk/pom.xml (original)
> +++ incubator/sqoop/trunk/pom.xml Sun Nov 27 11:12:37 2011
> @@ -261,12 +261,23 @@ limitations under the License.
> </dependencies>
>
> <build>
> + <scriptSourceDirectory>src/scripts</scriptSourceDirectory>
> <sourceDirectory>src/java</sourceDirectory>
> <testSourceDirectory>src/test</testSourceDirectory>
> <pluginManagement>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> + <artifactId>maven-antrun-plugin</artifactId>
> + <version>1.7</version>
> + </plugin>
> + <plugin>
> + <groupId>org.codehaus.mojo</groupId>
> + <artifactId>build-helper-maven-plugin</artifactId>
> + <version>1.7</version>
> + </plugin>
> + <plugin>
> + <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-clean-plugin</artifactId>
> <version>2.4.1</version>
> </plugin>
> @@ -300,6 +311,47 @@ limitations under the License.
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> + <artifactId>maven-antrun-plugin</artifactId>
> + <executions>
> + <execution>
> + <id>generate-version-tool</id>
> + <phase>generate-sources</phase>
> + <goals><goal>run</goal></goals>
> + <configuration>
> + <target>
> + <echo>Generating version tool</echo>
> + <mkdir dir="${basedir}/target/generated-sources"/>
> + <exec executable="${basedir}/src/scripts/write-version-info.sh"
> + dir="${basedir}" failonerror="true">
> + <arg value="${basedir}/target/generated-sources"/>
> + <arg value="${version}" />
> + <arg value="${gitHash}" />
> + </exec>
> + </target>
> + </configuration>
> + </execution>
> + </executions>
> + </plugin>
> + <plugin>
> + <groupId>org.codehaus.mojo</groupId>
> + <artifactId>build-helper-maven-plugin</artifactId>
> + <executions>
> + <execution>
> + <id>add-generated-sources</id>
> + <phase>generate-sources</phase>
> + <goals>
> + <goal>add-source</goal>
> + </goals>
> + <configuration>
> + <sources>
> + <source>${basedir}/target/generated-sources/src</source>
> + </sources>
> + </configuration>
> + </execution>
> + </executions>
> + </plugin>
> + <plugin>
> + <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <configuration>
> <systemPropertyVariables>
>
> Modified: incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
> URL: http://svn.apache.org/viewvc/incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java?rev=1206698&r1=1206697&r2=1206698&view=diff
> ==============================================================================
> --- incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java (original)
> +++ incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java Sun Nov 27 11:12:37 2011
> @@ -37,7 +37,6 @@ import org.apache.hadoop.util.StringUtil
>
> import com.cloudera.sqoop.config.ConfigurationHelper;
> import com.cloudera.sqoop.mapreduce.db.DBConfiguration;
> -import com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter;
>
> /**
> * A OutputFormat that sends the reduce output to a SQL table.
> @@ -119,7 +118,8 @@ public class DBOutputFormat<K extends DB
>
> statement = connection.prepareStatement(
> constructQuery(tableName, fieldNames));
> - return new DBRecordWriter(connection, statement);
> + return new com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter(
> + connection, statement);
> } catch (Exception ex) {
> throw new IOException(ex);
> }
>
>
>
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: svn commit: r1206698 - in /incubator/sqoop/trunk: pom.xml src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
Posted by Arvind Prabhakar <ar...@apache.org>.
On Tue, Nov 29, 2011 at 8:00 AM, Olivier Lamy <ol...@apache.org> wrote:
> 2011/11/28 Arvind Prabhakar <ar...@apache.org>:
>> On Sun, Nov 27, 2011 at 4:04 PM, Olivier Lamy <ol...@apache.org> wrote:
>>> 2011/11/27 arvind@cloudera.com <ar...@cloudera.com>:
>>>> Thanks Olivier, that is indeed the appropriate solution for this problem.
>>>> However, this fix was an interim fix to make sure that the builds come back
>>>> to normal. We certainly need to clean these up going forward.
>>>
>>> I can work on that an provide a solution based on build number maven plugin [1]
>>>
>>> But with that the ant build won't work anymore (except using filtering
>>> in the ant build too).
>>>
>>> I wonder about using only one build tool ?
>>
>> Thanks for the offer Olivier - I agree that we should have one build
>> tool only. However, ant has been the major tool for build and
>> development so far on the trunk and changing it now will impact the
>> majority of developers.
>>
>> A better bet would be to instead focus on maven migration with the
>> next version of Sqoop (SQOOP-365). That would not impact existing
>> developers and would help us achieve migration to a better build
>> environment soon as well.
>
> you mean I can hack that on sqoop2 ? Just have a quick look on: I like
> the new layout (more maven style :-) )
Correct. The goal is to do this layout right and make sure that
releasing and working with this codebase is as straightforward as
possible.
>
>>
>> Thanks,
>> Arvind
>>
>>>
>>> --
>>> Olivier Lamy
>>> Talend: http://coders.talend.com
>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>
>>> [1] http://mojo.codehaus.org/buildnumber-maven-plugin/
>>>
>>>>
>>>> Thanks,
>>>> Arvind
>>>>
>>>> On Sun, Nov 27, 2011 at 7:18 AM, Olivier Lamy <ol...@apache.org> wrote:
>>>>
>>>>> IMHO I think it will be better and more maintenable to use filtering
>>>>> stuff from maven and generate a .properties file with the needed infos
>>>>> instead of modifying a .java file.
>>>>> The SqoopVersion class could read this .properties file.
>>>>>
>>>>> And will make the build pass on windauze by example :-)
>>>>>
>>>>> 2011/11/27 <ja...@apache.org>:
>>>>> > Author: jarcec
>>>>> > Date: Sun Nov 27 11:12:37 2011
>>>>> > New Revision: 1206698
>>>>> >
>>>>> > URL: http://svn.apache.org/viewvc?rev=1206698&view=rev
>>>>> > Log:
>>>>> > SQOOP-392. Build using maven is broken due to missing class generated by
>>>>> script write-version-info.sh
>>>>> >
>>>>> > (Arvind Prabhakar via Jarek Jarcec Cecho)
>>>>> >
>>>>> >
>>>>> > Modified:
>>>>> > incubator/sqoop/trunk/pom.xml
>>>>> >
>>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>>> >
>>>>> > Modified: incubator/sqoop/trunk/pom.xml
>>>>> > URL:
>>>>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/pom.xml?rev=1206698&r1=1206697&r2=1206698&view=diff
>>>>> >
>>>>> ==============================================================================
>>>>> > --- incubator/sqoop/trunk/pom.xml (original)
>>>>> > +++ incubator/sqoop/trunk/pom.xml Sun Nov 27 11:12:37 2011
>>>>> > @@ -261,12 +261,23 @@ limitations under the License.
>>>>> > </dependencies>
>>>>> >
>>>>> > <build>
>>>>> > + <scriptSourceDirectory>src/scripts</scriptSourceDirectory>
>>>>> > <sourceDirectory>src/java</sourceDirectory>
>>>>> > <testSourceDirectory>src/test</testSourceDirectory>
>>>>> > <pluginManagement>
>>>>> > <plugins>
>>>>> > <plugin>
>>>>> > <groupId>org.apache.maven.plugins</groupId>
>>>>> > + <artifactId>maven-antrun-plugin</artifactId>
>>>>> > + <version>1.7</version>
>>>>> > + </plugin>
>>>>> > + <plugin>
>>>>> > + <groupId>org.codehaus.mojo</groupId>
>>>>> > + <artifactId>build-helper-maven-plugin</artifactId>
>>>>> > + <version>1.7</version>
>>>>> > + </plugin>
>>>>> > + <plugin>
>>>>> > + <groupId>org.apache.maven.plugins</groupId>
>>>>> > <artifactId>maven-clean-plugin</artifactId>
>>>>> > <version>2.4.1</version>
>>>>> > </plugin>
>>>>> > @@ -300,6 +311,47 @@ limitations under the License.
>>>>> > <plugins>
>>>>> > <plugin>
>>>>> > <groupId>org.apache.maven.plugins</groupId>
>>>>> > + <artifactId>maven-antrun-plugin</artifactId>
>>>>> > + <executions>
>>>>> > + <execution>
>>>>> > + <id>generate-version-tool</id>
>>>>> > + <phase>generate-sources</phase>
>>>>> > + <goals><goal>run</goal></goals>
>>>>> > + <configuration>
>>>>> > + <target>
>>>>> > + <echo>Generating version tool</echo>
>>>>> > + <mkdir dir="${basedir}/target/generated-sources"/>
>>>>> > + <exec
>>>>> executable="${basedir}/src/scripts/write-version-info.sh"
>>>>> > + dir="${basedir}" failonerror="true">
>>>>> > + <arg value="${basedir}/target/generated-sources"/>
>>>>> > + <arg value="${version}" />
>>>>> > + <arg value="${gitHash}" />
>>>>> > + </exec>
>>>>> > + </target>
>>>>> > + </configuration>
>>>>> > + </execution>
>>>>> > + </executions>
>>>>> > + </plugin>
>>>>> > + <plugin>
>>>>> > + <groupId>org.codehaus.mojo</groupId>
>>>>> > + <artifactId>build-helper-maven-plugin</artifactId>
>>>>> > + <executions>
>>>>> > + <execution>
>>>>> > + <id>add-generated-sources</id>
>>>>> > + <phase>generate-sources</phase>
>>>>> > + <goals>
>>>>> > + <goal>add-source</goal>
>>>>> > + </goals>
>>>>> > + <configuration>
>>>>> > + <sources>
>>>>> > + <source>${basedir}/target/generated-sources/src</source>
>>>>> > + </sources>
>>>>> > + </configuration>
>>>>> > + </execution>
>>>>> > + </executions>
>>>>> > + </plugin>
>>>>> > + <plugin>
>>>>> > + <groupId>org.apache.maven.plugins</groupId>
>>>>> > <artifactId>maven-surefire-plugin</artifactId>
>>>>> > <configuration>
>>>>> > <systemPropertyVariables>
>>>>> >
>>>>> > Modified:
>>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>>> > URL:
>>>>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java?rev=1206698&r1=1206697&r2=1206698&view=diff
>>>>> >
>>>>> ==============================================================================
>>>>> > ---
>>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>>> (original)
>>>>> > +++
>>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>>> Sun Nov 27 11:12:37 2011
>>>>> > @@ -37,7 +37,6 @@ import org.apache.hadoop.util.StringUtil
>>>>> >
>>>>> > import com.cloudera.sqoop.config.ConfigurationHelper;
>>>>> > import com.cloudera.sqoop.mapreduce.db.DBConfiguration;
>>>>> > -import com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter;
>>>>> >
>>>>> > /**
>>>>> > * A OutputFormat that sends the reduce output to a SQL table.
>>>>> > @@ -119,7 +118,8 @@ public class DBOutputFormat<K extends DB
>>>>> >
>>>>> > statement = connection.prepareStatement(
>>>>> > constructQuery(tableName, fieldNames));
>>>>> > - return new DBRecordWriter(connection, statement);
>>>>> > + return new
>>>>> com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter(
>>>>> > + connection, statement);
>>>>> > } catch (Exception ex) {
>>>>> > throw new IOException(ex);
>>>>> > }
>>>>> >
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Olivier Lamy
>>>>> Talend: http://coders.talend.com
>>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>>
>>>>
>>>
>
>
>
> --
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: svn commit: r1206698 - in /incubator/sqoop/trunk: pom.xml src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
Posted by Olivier Lamy <ol...@apache.org>.
2011/11/28 Arvind Prabhakar <ar...@apache.org>:
> On Sun, Nov 27, 2011 at 4:04 PM, Olivier Lamy <ol...@apache.org> wrote:
>> 2011/11/27 arvind@cloudera.com <ar...@cloudera.com>:
>>> Thanks Olivier, that is indeed the appropriate solution for this problem.
>>> However, this fix was an interim fix to make sure that the builds come back
>>> to normal. We certainly need to clean these up going forward.
>>
>> I can work on that an provide a solution based on build number maven plugin [1]
>>
>> But with that the ant build won't work anymore (except using filtering
>> in the ant build too).
>>
>> I wonder about using only one build tool ?
>
> Thanks for the offer Olivier - I agree that we should have one build
> tool only. However, ant has been the major tool for build and
> development so far on the trunk and changing it now will impact the
> majority of developers.
>
> A better bet would be to instead focus on maven migration with the
> next version of Sqoop (SQOOP-365). That would not impact existing
> developers and would help us achieve migration to a better build
> environment soon as well.
you mean I can hack that on sqoop2 ? Just have a quick look on: I like
the new layout (more maven style :-) )
>
> Thanks,
> Arvind
>
>>
>> --
>> Olivier Lamy
>> Talend: http://coders.talend.com
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
>> [1] http://mojo.codehaus.org/buildnumber-maven-plugin/
>>
>>>
>>> Thanks,
>>> Arvind
>>>
>>> On Sun, Nov 27, 2011 at 7:18 AM, Olivier Lamy <ol...@apache.org> wrote:
>>>
>>>> IMHO I think it will be better and more maintenable to use filtering
>>>> stuff from maven and generate a .properties file with the needed infos
>>>> instead of modifying a .java file.
>>>> The SqoopVersion class could read this .properties file.
>>>>
>>>> And will make the build pass on windauze by example :-)
>>>>
>>>> 2011/11/27 <ja...@apache.org>:
>>>> > Author: jarcec
>>>> > Date: Sun Nov 27 11:12:37 2011
>>>> > New Revision: 1206698
>>>> >
>>>> > URL: http://svn.apache.org/viewvc?rev=1206698&view=rev
>>>> > Log:
>>>> > SQOOP-392. Build using maven is broken due to missing class generated by
>>>> script write-version-info.sh
>>>> >
>>>> > (Arvind Prabhakar via Jarek Jarcec Cecho)
>>>> >
>>>> >
>>>> > Modified:
>>>> > incubator/sqoop/trunk/pom.xml
>>>> >
>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>> >
>>>> > Modified: incubator/sqoop/trunk/pom.xml
>>>> > URL:
>>>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/pom.xml?rev=1206698&r1=1206697&r2=1206698&view=diff
>>>> >
>>>> ==============================================================================
>>>> > --- incubator/sqoop/trunk/pom.xml (original)
>>>> > +++ incubator/sqoop/trunk/pom.xml Sun Nov 27 11:12:37 2011
>>>> > @@ -261,12 +261,23 @@ limitations under the License.
>>>> > </dependencies>
>>>> >
>>>> > <build>
>>>> > + <scriptSourceDirectory>src/scripts</scriptSourceDirectory>
>>>> > <sourceDirectory>src/java</sourceDirectory>
>>>> > <testSourceDirectory>src/test</testSourceDirectory>
>>>> > <pluginManagement>
>>>> > <plugins>
>>>> > <plugin>
>>>> > <groupId>org.apache.maven.plugins</groupId>
>>>> > + <artifactId>maven-antrun-plugin</artifactId>
>>>> > + <version>1.7</version>
>>>> > + </plugin>
>>>> > + <plugin>
>>>> > + <groupId>org.codehaus.mojo</groupId>
>>>> > + <artifactId>build-helper-maven-plugin</artifactId>
>>>> > + <version>1.7</version>
>>>> > + </plugin>
>>>> > + <plugin>
>>>> > + <groupId>org.apache.maven.plugins</groupId>
>>>> > <artifactId>maven-clean-plugin</artifactId>
>>>> > <version>2.4.1</version>
>>>> > </plugin>
>>>> > @@ -300,6 +311,47 @@ limitations under the License.
>>>> > <plugins>
>>>> > <plugin>
>>>> > <groupId>org.apache.maven.plugins</groupId>
>>>> > + <artifactId>maven-antrun-plugin</artifactId>
>>>> > + <executions>
>>>> > + <execution>
>>>> > + <id>generate-version-tool</id>
>>>> > + <phase>generate-sources</phase>
>>>> > + <goals><goal>run</goal></goals>
>>>> > + <configuration>
>>>> > + <target>
>>>> > + <echo>Generating version tool</echo>
>>>> > + <mkdir dir="${basedir}/target/generated-sources"/>
>>>> > + <exec
>>>> executable="${basedir}/src/scripts/write-version-info.sh"
>>>> > + dir="${basedir}" failonerror="true">
>>>> > + <arg value="${basedir}/target/generated-sources"/>
>>>> > + <arg value="${version}" />
>>>> > + <arg value="${gitHash}" />
>>>> > + </exec>
>>>> > + </target>
>>>> > + </configuration>
>>>> > + </execution>
>>>> > + </executions>
>>>> > + </plugin>
>>>> > + <plugin>
>>>> > + <groupId>org.codehaus.mojo</groupId>
>>>> > + <artifactId>build-helper-maven-plugin</artifactId>
>>>> > + <executions>
>>>> > + <execution>
>>>> > + <id>add-generated-sources</id>
>>>> > + <phase>generate-sources</phase>
>>>> > + <goals>
>>>> > + <goal>add-source</goal>
>>>> > + </goals>
>>>> > + <configuration>
>>>> > + <sources>
>>>> > + <source>${basedir}/target/generated-sources/src</source>
>>>> > + </sources>
>>>> > + </configuration>
>>>> > + </execution>
>>>> > + </executions>
>>>> > + </plugin>
>>>> > + <plugin>
>>>> > + <groupId>org.apache.maven.plugins</groupId>
>>>> > <artifactId>maven-surefire-plugin</artifactId>
>>>> > <configuration>
>>>> > <systemPropertyVariables>
>>>> >
>>>> > Modified:
>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>> > URL:
>>>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java?rev=1206698&r1=1206697&r2=1206698&view=diff
>>>> >
>>>> ==============================================================================
>>>> > ---
>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>> (original)
>>>> > +++
>>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>>> Sun Nov 27 11:12:37 2011
>>>> > @@ -37,7 +37,6 @@ import org.apache.hadoop.util.StringUtil
>>>> >
>>>> > import com.cloudera.sqoop.config.ConfigurationHelper;
>>>> > import com.cloudera.sqoop.mapreduce.db.DBConfiguration;
>>>> > -import com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter;
>>>> >
>>>> > /**
>>>> > * A OutputFormat that sends the reduce output to a SQL table.
>>>> > @@ -119,7 +118,8 @@ public class DBOutputFormat<K extends DB
>>>> >
>>>> > statement = connection.prepareStatement(
>>>> > constructQuery(tableName, fieldNames));
>>>> > - return new DBRecordWriter(connection, statement);
>>>> > + return new
>>>> com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter(
>>>> > + connection, statement);
>>>> > } catch (Exception ex) {
>>>> > throw new IOException(ex);
>>>> > }
>>>> >
>>>> >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Olivier Lamy
>>>> Talend: http://coders.talend.com
>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>
>>>
>>
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: svn commit: r1206698 - in /incubator/sqoop/trunk: pom.xml src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
Posted by Arvind Prabhakar <ar...@apache.org>.
On Sun, Nov 27, 2011 at 4:04 PM, Olivier Lamy <ol...@apache.org> wrote:
> 2011/11/27 arvind@cloudera.com <ar...@cloudera.com>:
>> Thanks Olivier, that is indeed the appropriate solution for this problem.
>> However, this fix was an interim fix to make sure that the builds come back
>> to normal. We certainly need to clean these up going forward.
>
> I can work on that an provide a solution based on build number maven plugin [1]
>
> But with that the ant build won't work anymore (except using filtering
> in the ant build too).
>
> I wonder about using only one build tool ?
Thanks for the offer Olivier - I agree that we should have one build
tool only. However, ant has been the major tool for build and
development so far on the trunk and changing it now will impact the
majority of developers.
A better bet would be to instead focus on maven migration with the
next version of Sqoop (SQOOP-365). That would not impact existing
developers and would help us achieve migration to a better build
environment soon as well.
Thanks,
Arvind
>
> --
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
> [1] http://mojo.codehaus.org/buildnumber-maven-plugin/
>
>>
>> Thanks,
>> Arvind
>>
>> On Sun, Nov 27, 2011 at 7:18 AM, Olivier Lamy <ol...@apache.org> wrote:
>>
>>> IMHO I think it will be better and more maintenable to use filtering
>>> stuff from maven and generate a .properties file with the needed infos
>>> instead of modifying a .java file.
>>> The SqoopVersion class could read this .properties file.
>>>
>>> And will make the build pass on windauze by example :-)
>>>
>>> 2011/11/27 <ja...@apache.org>:
>>> > Author: jarcec
>>> > Date: Sun Nov 27 11:12:37 2011
>>> > New Revision: 1206698
>>> >
>>> > URL: http://svn.apache.org/viewvc?rev=1206698&view=rev
>>> > Log:
>>> > SQOOP-392. Build using maven is broken due to missing class generated by
>>> script write-version-info.sh
>>> >
>>> > (Arvind Prabhakar via Jarek Jarcec Cecho)
>>> >
>>> >
>>> > Modified:
>>> > incubator/sqoop/trunk/pom.xml
>>> >
>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>> >
>>> > Modified: incubator/sqoop/trunk/pom.xml
>>> > URL:
>>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/pom.xml?rev=1206698&r1=1206697&r2=1206698&view=diff
>>> >
>>> ==============================================================================
>>> > --- incubator/sqoop/trunk/pom.xml (original)
>>> > +++ incubator/sqoop/trunk/pom.xml Sun Nov 27 11:12:37 2011
>>> > @@ -261,12 +261,23 @@ limitations under the License.
>>> > </dependencies>
>>> >
>>> > <build>
>>> > + <scriptSourceDirectory>src/scripts</scriptSourceDirectory>
>>> > <sourceDirectory>src/java</sourceDirectory>
>>> > <testSourceDirectory>src/test</testSourceDirectory>
>>> > <pluginManagement>
>>> > <plugins>
>>> > <plugin>
>>> > <groupId>org.apache.maven.plugins</groupId>
>>> > + <artifactId>maven-antrun-plugin</artifactId>
>>> > + <version>1.7</version>
>>> > + </plugin>
>>> > + <plugin>
>>> > + <groupId>org.codehaus.mojo</groupId>
>>> > + <artifactId>build-helper-maven-plugin</artifactId>
>>> > + <version>1.7</version>
>>> > + </plugin>
>>> > + <plugin>
>>> > + <groupId>org.apache.maven.plugins</groupId>
>>> > <artifactId>maven-clean-plugin</artifactId>
>>> > <version>2.4.1</version>
>>> > </plugin>
>>> > @@ -300,6 +311,47 @@ limitations under the License.
>>> > <plugins>
>>> > <plugin>
>>> > <groupId>org.apache.maven.plugins</groupId>
>>> > + <artifactId>maven-antrun-plugin</artifactId>
>>> > + <executions>
>>> > + <execution>
>>> > + <id>generate-version-tool</id>
>>> > + <phase>generate-sources</phase>
>>> > + <goals><goal>run</goal></goals>
>>> > + <configuration>
>>> > + <target>
>>> > + <echo>Generating version tool</echo>
>>> > + <mkdir dir="${basedir}/target/generated-sources"/>
>>> > + <exec
>>> executable="${basedir}/src/scripts/write-version-info.sh"
>>> > + dir="${basedir}" failonerror="true">
>>> > + <arg value="${basedir}/target/generated-sources"/>
>>> > + <arg value="${version}" />
>>> > + <arg value="${gitHash}" />
>>> > + </exec>
>>> > + </target>
>>> > + </configuration>
>>> > + </execution>
>>> > + </executions>
>>> > + </plugin>
>>> > + <plugin>
>>> > + <groupId>org.codehaus.mojo</groupId>
>>> > + <artifactId>build-helper-maven-plugin</artifactId>
>>> > + <executions>
>>> > + <execution>
>>> > + <id>add-generated-sources</id>
>>> > + <phase>generate-sources</phase>
>>> > + <goals>
>>> > + <goal>add-source</goal>
>>> > + </goals>
>>> > + <configuration>
>>> > + <sources>
>>> > + <source>${basedir}/target/generated-sources/src</source>
>>> > + </sources>
>>> > + </configuration>
>>> > + </execution>
>>> > + </executions>
>>> > + </plugin>
>>> > + <plugin>
>>> > + <groupId>org.apache.maven.plugins</groupId>
>>> > <artifactId>maven-surefire-plugin</artifactId>
>>> > <configuration>
>>> > <systemPropertyVariables>
>>> >
>>> > Modified:
>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>> > URL:
>>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java?rev=1206698&r1=1206697&r2=1206698&view=diff
>>> >
>>> ==============================================================================
>>> > ---
>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>> (original)
>>> > +++
>>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>>> Sun Nov 27 11:12:37 2011
>>> > @@ -37,7 +37,6 @@ import org.apache.hadoop.util.StringUtil
>>> >
>>> > import com.cloudera.sqoop.config.ConfigurationHelper;
>>> > import com.cloudera.sqoop.mapreduce.db.DBConfiguration;
>>> > -import com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter;
>>> >
>>> > /**
>>> > * A OutputFormat that sends the reduce output to a SQL table.
>>> > @@ -119,7 +118,8 @@ public class DBOutputFormat<K extends DB
>>> >
>>> > statement = connection.prepareStatement(
>>> > constructQuery(tableName, fieldNames));
>>> > - return new DBRecordWriter(connection, statement);
>>> > + return new
>>> com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter(
>>> > + connection, statement);
>>> > } catch (Exception ex) {
>>> > throw new IOException(ex);
>>> > }
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Olivier Lamy
>>> Talend: http://coders.talend.com
>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>
>>
>
Re: svn commit: r1206698 - in /incubator/sqoop/trunk: pom.xml src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
Posted by Olivier Lamy <ol...@apache.org>.
2011/11/27 arvind@cloudera.com <ar...@cloudera.com>:
> Thanks Olivier, that is indeed the appropriate solution for this problem.
> However, this fix was an interim fix to make sure that the builds come back
> to normal. We certainly need to clean these up going forward.
I can work on that an provide a solution based on build number maven plugin [1]
But with that the ant build won't work anymore (except using filtering
in the ant build too).
I wonder about using only one build tool ?
--
Olivier Lamy
Talend: http://coders.talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
[1] http://mojo.codehaus.org/buildnumber-maven-plugin/
>
> Thanks,
> Arvind
>
> On Sun, Nov 27, 2011 at 7:18 AM, Olivier Lamy <ol...@apache.org> wrote:
>
>> IMHO I think it will be better and more maintenable to use filtering
>> stuff from maven and generate a .properties file with the needed infos
>> instead of modifying a .java file.
>> The SqoopVersion class could read this .properties file.
>>
>> And will make the build pass on windauze by example :-)
>>
>> 2011/11/27 <ja...@apache.org>:
>> > Author: jarcec
>> > Date: Sun Nov 27 11:12:37 2011
>> > New Revision: 1206698
>> >
>> > URL: http://svn.apache.org/viewvc?rev=1206698&view=rev
>> > Log:
>> > SQOOP-392. Build using maven is broken due to missing class generated by
>> script write-version-info.sh
>> >
>> > (Arvind Prabhakar via Jarek Jarcec Cecho)
>> >
>> >
>> > Modified:
>> > incubator/sqoop/trunk/pom.xml
>> >
>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>> >
>> > Modified: incubator/sqoop/trunk/pom.xml
>> > URL:
>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/pom.xml?rev=1206698&r1=1206697&r2=1206698&view=diff
>> >
>> ==============================================================================
>> > --- incubator/sqoop/trunk/pom.xml (original)
>> > +++ incubator/sqoop/trunk/pom.xml Sun Nov 27 11:12:37 2011
>> > @@ -261,12 +261,23 @@ limitations under the License.
>> > </dependencies>
>> >
>> > <build>
>> > + <scriptSourceDirectory>src/scripts</scriptSourceDirectory>
>> > <sourceDirectory>src/java</sourceDirectory>
>> > <testSourceDirectory>src/test</testSourceDirectory>
>> > <pluginManagement>
>> > <plugins>
>> > <plugin>
>> > <groupId>org.apache.maven.plugins</groupId>
>> > + <artifactId>maven-antrun-plugin</artifactId>
>> > + <version>1.7</version>
>> > + </plugin>
>> > + <plugin>
>> > + <groupId>org.codehaus.mojo</groupId>
>> > + <artifactId>build-helper-maven-plugin</artifactId>
>> > + <version>1.7</version>
>> > + </plugin>
>> > + <plugin>
>> > + <groupId>org.apache.maven.plugins</groupId>
>> > <artifactId>maven-clean-plugin</artifactId>
>> > <version>2.4.1</version>
>> > </plugin>
>> > @@ -300,6 +311,47 @@ limitations under the License.
>> > <plugins>
>> > <plugin>
>> > <groupId>org.apache.maven.plugins</groupId>
>> > + <artifactId>maven-antrun-plugin</artifactId>
>> > + <executions>
>> > + <execution>
>> > + <id>generate-version-tool</id>
>> > + <phase>generate-sources</phase>
>> > + <goals><goal>run</goal></goals>
>> > + <configuration>
>> > + <target>
>> > + <echo>Generating version tool</echo>
>> > + <mkdir dir="${basedir}/target/generated-sources"/>
>> > + <exec
>> executable="${basedir}/src/scripts/write-version-info.sh"
>> > + dir="${basedir}" failonerror="true">
>> > + <arg value="${basedir}/target/generated-sources"/>
>> > + <arg value="${version}" />
>> > + <arg value="${gitHash}" />
>> > + </exec>
>> > + </target>
>> > + </configuration>
>> > + </execution>
>> > + </executions>
>> > + </plugin>
>> > + <plugin>
>> > + <groupId>org.codehaus.mojo</groupId>
>> > + <artifactId>build-helper-maven-plugin</artifactId>
>> > + <executions>
>> > + <execution>
>> > + <id>add-generated-sources</id>
>> > + <phase>generate-sources</phase>
>> > + <goals>
>> > + <goal>add-source</goal>
>> > + </goals>
>> > + <configuration>
>> > + <sources>
>> > + <source>${basedir}/target/generated-sources/src</source>
>> > + </sources>
>> > + </configuration>
>> > + </execution>
>> > + </executions>
>> > + </plugin>
>> > + <plugin>
>> > + <groupId>org.apache.maven.plugins</groupId>
>> > <artifactId>maven-surefire-plugin</artifactId>
>> > <configuration>
>> > <systemPropertyVariables>
>> >
>> > Modified:
>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>> > URL:
>> http://svn.apache.org/viewvc/incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java?rev=1206698&r1=1206697&r2=1206698&view=diff
>> >
>> ==============================================================================
>> > ---
>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>> (original)
>> > +++
>> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
>> Sun Nov 27 11:12:37 2011
>> > @@ -37,7 +37,6 @@ import org.apache.hadoop.util.StringUtil
>> >
>> > import com.cloudera.sqoop.config.ConfigurationHelper;
>> > import com.cloudera.sqoop.mapreduce.db.DBConfiguration;
>> > -import com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter;
>> >
>> > /**
>> > * A OutputFormat that sends the reduce output to a SQL table.
>> > @@ -119,7 +118,8 @@ public class DBOutputFormat<K extends DB
>> >
>> > statement = connection.prepareStatement(
>> > constructQuery(tableName, fieldNames));
>> > - return new DBRecordWriter(connection, statement);
>> > + return new
>> com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter(
>> > + connection, statement);
>> > } catch (Exception ex) {
>> > throw new IOException(ex);
>> > }
>> >
>> >
>> >
>>
>>
>>
>> --
>> Olivier Lamy
>> Talend: http://coders.talend.com
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
>
Re: svn commit: r1206698 - in /incubator/sqoop/trunk: pom.xml src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
Posted by "arvind@cloudera.com" <ar...@cloudera.com>.
Thanks Olivier, that is indeed the appropriate solution for this problem.
However, this fix was an interim fix to make sure that the builds come back
to normal. We certainly need to clean these up going forward.
Thanks,
Arvind
On Sun, Nov 27, 2011 at 7:18 AM, Olivier Lamy <ol...@apache.org> wrote:
> IMHO I think it will be better and more maintenable to use filtering
> stuff from maven and generate a .properties file with the needed infos
> instead of modifying a .java file.
> The SqoopVersion class could read this .properties file.
>
> And will make the build pass on windauze by example :-)
>
> 2011/11/27 <ja...@apache.org>:
> > Author: jarcec
> > Date: Sun Nov 27 11:12:37 2011
> > New Revision: 1206698
> >
> > URL: http://svn.apache.org/viewvc?rev=1206698&view=rev
> > Log:
> > SQOOP-392. Build using maven is broken due to missing class generated by
> script write-version-info.sh
> >
> > (Arvind Prabhakar via Jarek Jarcec Cecho)
> >
> >
> > Modified:
> > incubator/sqoop/trunk/pom.xml
> >
> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
> >
> > Modified: incubator/sqoop/trunk/pom.xml
> > URL:
> http://svn.apache.org/viewvc/incubator/sqoop/trunk/pom.xml?rev=1206698&r1=1206697&r2=1206698&view=diff
> >
> ==============================================================================
> > --- incubator/sqoop/trunk/pom.xml (original)
> > +++ incubator/sqoop/trunk/pom.xml Sun Nov 27 11:12:37 2011
> > @@ -261,12 +261,23 @@ limitations under the License.
> > </dependencies>
> >
> > <build>
> > + <scriptSourceDirectory>src/scripts</scriptSourceDirectory>
> > <sourceDirectory>src/java</sourceDirectory>
> > <testSourceDirectory>src/test</testSourceDirectory>
> > <pluginManagement>
> > <plugins>
> > <plugin>
> > <groupId>org.apache.maven.plugins</groupId>
> > + <artifactId>maven-antrun-plugin</artifactId>
> > + <version>1.7</version>
> > + </plugin>
> > + <plugin>
> > + <groupId>org.codehaus.mojo</groupId>
> > + <artifactId>build-helper-maven-plugin</artifactId>
> > + <version>1.7</version>
> > + </plugin>
> > + <plugin>
> > + <groupId>org.apache.maven.plugins</groupId>
> > <artifactId>maven-clean-plugin</artifactId>
> > <version>2.4.1</version>
> > </plugin>
> > @@ -300,6 +311,47 @@ limitations under the License.
> > <plugins>
> > <plugin>
> > <groupId>org.apache.maven.plugins</groupId>
> > + <artifactId>maven-antrun-plugin</artifactId>
> > + <executions>
> > + <execution>
> > + <id>generate-version-tool</id>
> > + <phase>generate-sources</phase>
> > + <goals><goal>run</goal></goals>
> > + <configuration>
> > + <target>
> > + <echo>Generating version tool</echo>
> > + <mkdir dir="${basedir}/target/generated-sources"/>
> > + <exec
> executable="${basedir}/src/scripts/write-version-info.sh"
> > + dir="${basedir}" failonerror="true">
> > + <arg value="${basedir}/target/generated-sources"/>
> > + <arg value="${version}" />
> > + <arg value="${gitHash}" />
> > + </exec>
> > + </target>
> > + </configuration>
> > + </execution>
> > + </executions>
> > + </plugin>
> > + <plugin>
> > + <groupId>org.codehaus.mojo</groupId>
> > + <artifactId>build-helper-maven-plugin</artifactId>
> > + <executions>
> > + <execution>
> > + <id>add-generated-sources</id>
> > + <phase>generate-sources</phase>
> > + <goals>
> > + <goal>add-source</goal>
> > + </goals>
> > + <configuration>
> > + <sources>
> > + <source>${basedir}/target/generated-sources/src</source>
> > + </sources>
> > + </configuration>
> > + </execution>
> > + </executions>
> > + </plugin>
> > + <plugin>
> > + <groupId>org.apache.maven.plugins</groupId>
> > <artifactId>maven-surefire-plugin</artifactId>
> > <configuration>
> > <systemPropertyVariables>
> >
> > Modified:
> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
> > URL:
> http://svn.apache.org/viewvc/incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java?rev=1206698&r1=1206697&r2=1206698&view=diff
> >
> ==============================================================================
> > ---
> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
> (original)
> > +++
> incubator/sqoop/trunk/src/java/org/apache/sqoop/mapreduce/db/DBOutputFormat.java
> Sun Nov 27 11:12:37 2011
> > @@ -37,7 +37,6 @@ import org.apache.hadoop.util.StringUtil
> >
> > import com.cloudera.sqoop.config.ConfigurationHelper;
> > import com.cloudera.sqoop.mapreduce.db.DBConfiguration;
> > -import com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter;
> >
> > /**
> > * A OutputFormat that sends the reduce output to a SQL table.
> > @@ -119,7 +118,8 @@ public class DBOutputFormat<K extends DB
> >
> > statement = connection.prepareStatement(
> > constructQuery(tableName, fieldNames));
> > - return new DBRecordWriter(connection, statement);
> > + return new
> com.cloudera.sqoop.mapreduce.db.DBOutputFormat.DBRecordWriter(
> > + connection, statement);
> > } catch (Exception ex) {
> > throw new IOException(ex);
> > }
> >
> >
> >
>
>
>
> --
> Olivier Lamy
> Talend: http://coders.talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>