You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nutch.apache.org by jn...@apache.org on 2011/05/31 17:08:24 UTC
svn commit: r1129734 - in /nutch/branches/branch-1.3: CHANGES.txt build.xml
default.properties ivy/ivy-2.1.0.jar ivy/ivy.xml ivy/mvn.template pom.xml
Author: jnioche
Date: Tue May 31 15:08:24 2011
New Revision: 1129734
URL: http://svn.apache.org/viewvc?rev=1129734&view=rev
Log:
Undoing excessive commit
Added:
nutch/branches/branch-1.3/ivy/ivy-2.1.0.jar
- copied unchanged from r1126417, nutch/branches/branch-1.3/ivy/ivy-2.1.0.jar
nutch/branches/branch-1.3/pom.xml
- copied unchanged from r1126417, nutch/branches/branch-1.3/pom.xml
Removed:
nutch/branches/branch-1.3/ivy/mvn.template
Modified:
nutch/branches/branch-1.3/CHANGES.txt
nutch/branches/branch-1.3/build.xml
nutch/branches/branch-1.3/default.properties
nutch/branches/branch-1.3/ivy/ivy.xml
Modified: nutch/branches/branch-1.3/CHANGES.txt
URL: http://svn.apache.org/viewvc/nutch/branches/branch-1.3/CHANGES.txt?rev=1129734&r1=1129733&r2=1129734&view=diff
==============================================================================
--- nutch/branches/branch-1.3/CHANGES.txt (original)
+++ nutch/branches/branch-1.3/CHANGES.txt Tue May 31 15:08:24 2011
@@ -1,6 +1,6 @@
Nutch Change Log
-Release 1.3 - 5/31/2011
+Release 1.3 - 4/21/2011
* NUTCH-994 Fine tune Solr schema (markus)
@@ -68,75 +68,53 @@ Release 1.3 - 5/31/2011
* NUTCH-936 LanguageIdentifier should not set empty lang field on NutchDocument (Markus Jelsma via jnioche)
-* NUTCH-787 ScoringFilters should not override the injected score (jnioche)
-
-* NUTCH-949 Conflicting ANT jars in classpath (jnioche)
-
-* NUTCH-863 Benchmark and a testbed proxy server (ab)
-
-* NUTCH-844 Improve NutchConfiguration (ab)
-
-* NUTCH-845 Native hadoop libs not available through maven (ab)
-
-* NUTCH-843 Separate the build and runtime environments (ab)
-
-* NUTCH-821 Use ivy in nutch builds (Enis Soztutar, jnioche)
+* NUTCH-855 ScoringFilter and IndexingFilter: To allow for the propagation of URL Metatags and their subsequent indexing (Scott Gonyea via mattmann)
-* NUTCH-837 Remove search servers and Lucene dependencies (ab)
-
-* NUTCH-836 Remove deprecated parse plugins (jnioche)
-
-* NUTCH-939 Added -dir command line option to SolrIndexer (Claudio Martella via ab)
-
-* NUTCH-948 Remove Lucene dependencies (ab)
+* NUTCH-901 Make index-more plug-in configurable (Markus Jelsma, mattmann)
-Release 1.2 - 09/18/2010
-
-* NUTCH-901 Make index-more plug-in configurable (Markus Jelsma via mattmann)
-
-* NUTCH-908 Infinite Loop and Null Pointer Bugs in Searching (kubes via mattmann)
-
-* NUTCH-906 Nutch OpenSearch sometimes raises DOMExceptions (Asheesh Laroia via ab)
-
-* NUTCH-862 HttpClient null pointer exception (Sebastian Nagel via ab)
+* NUTCH-716 Make subcollection index filed multivalued (Dmitry Lihachev via jnioche)
* NUTCH-905 Configurable file protocol parent directory crawling (Thorsten Scherler, mattmann, ab)
-* NUTCH-877 Allow setting of slop values for non-quote phrase queries on query-basic plugin (kubes via jnioche)
+* NUTCH-787 ScoringFilters should not override the injected score (jnioche)
-* NUTCH-716 Make subcollection index filed multivalued (Dmitry Lihachev via jnioche)
+* NUTCH-949 Conflicting ANT jars in classpath (jnioche)
-* NUTCH-878 ScoringFilters should not override the injected score
+* NUTCH-862 HttpClient null pointer exception (Sebastian Nagel via ab)
* NUTCH-870 Injector should add the metadata before calling injectedScore (jnioche via mattmann)
-* NUTCH-858 No longer able to set per-field boosts on lucene documents (ab)
-
* NUTCH-869 Add parse-html back (jnioche)
-* NUTCH-871 MoreIndexingFilter missing date format (Max Lynch via mattmann)
+* NUTCH-871 MoreIndexingFilter missing date format (Max Lynch via mattmann)
* NUTCH-696 Timeout for Parser (ab, jnioche)
-* NUTCH-857 DistributedBeans should not close their RPC counterparts (kubes)
-
-* NUTCH-855 ScoringFilter and IndexingFilter: To allow for the propagation of URL Metatags
- and their subsequent indexing (Scott Gonyea via mattmann)
+* NUTCH-863 Benchmark and a testbed proxy server (ab)
-* NUTCH-677 Segment merge filering based on segment content (Marcin Okraszewski via mattmann)
+* NUTCH-677 Segment merge filtering based on segment content (Marcin Okraszewski via mattmann)
* NUTCH-774 Retry interval in crawl date is set to 0 (Reinhard Schwab via mattmann)
* NUTCH-697 Generate log output for solr indexer and dedup (Dmitry Lihachev, Jeroen van Vianen via mattmann)
+* NUTCH-844 Improve NutchConfiguration (ab)
+
* NUTCH-850 SolrDeleteDuplicates needs to clone the SolrRecord objects (jnioche)
+* NUTCH-845 Native hadoop libs not available through maven (ab)
+
+* NUTCH-843 Separate the build and runtime environments (ab)
+
+* NUTCH-821 Use ivy in nutch builds (Enis Soztutar, jnioche)
+
* NUTCH-838 Add timing information to all Tool classes (Jeroen van Vianen, mattmann)
-* NUTCH-835 Document deduplication failed using MD5Signature (Sebastian Nagel via ab)
+* NUTCH-837 Remove search servers and Lucene dependencies (ab)
+
+* NUTCH-836 Remove deprecated parse plugins (jnioche)
-* NUTCH-831 Allow configuration of how fields crawled by Nutch are stored / indexed /
- tokenized (Jeroen van Vianen via mattmann)
+* NUTCH-835 Document deduplication failed using MD5Signature (Sebastian Nagel via ab)
* NUTCH-278 Fetcher-status might need clarification: kbit/s instead of kb/s shown (Alex McLintock via mattmann)
@@ -144,6 +122,11 @@ Release 1.2 - 09/18/2010
* NUTCH-832 Website menu has lots of broken links - in particular the API docs (Alex McLintock via mattmann)
+* NUTCH-939 Added -dir command line option to SolrIndexer (Claudio Martella via ab)
+
+* NUTCH-948 Remove Lucene dependencies (ab)
+
+
Release 1.1 - 2010-06-06
* NUTCH-819 Included Solr schema.xml and solrindex-mapping.xml don't play together (ab)
Modified: nutch/branches/branch-1.3/build.xml
URL: http://svn.apache.org/viewvc/nutch/branches/branch-1.3/build.xml?rev=1129734&r1=1129733&r2=1129734&view=diff
==============================================================================
--- nutch/branches/branch-1.3/build.xml (original)
+++ nutch/branches/branch-1.3/build.xml Tue May 31 15:08:24 2011
@@ -17,663 +17,708 @@
-->
<project name="Nutch" default="runtime" xmlns:ivy="antlib:org.apache.ivy.ant" xmlns:artifact="antlib:org.apache.maven.artifact.ant">
- <!-- Load all the default properties, and any the user wants -->
- <!-- to contribute (without having to type -D or edit this file -->
- <property file="${user.home}/build.properties" />
- <property file="${basedir}/build.properties" />
- <property file="${basedir}/default.properties" />
- <property name="test.junit.output.format" value="plain" />
- <property name="release.dir" value="${build.dir}/release" />
-
- <!-- define Maven coordinates, repository url and artifacts name etc -->
- <property name="groupId" value="org.apache.nutch" />
- <property name="artifactId" value="nutch" />
- <property name="maven-repository-url" value="https://repository.apache.org/service/local/staging/deploy/maven2" />
- <property name="maven-repository-id" value="apache.releases.https" />
- <property name="maven-jar" value="${release.dir}/${artifactId}-${version}.jar" />
- <property name="maven-javadoc-jar" value="${release.dir}/${artifactId}-${version}-javadoc.jar" />
- <property name="maven-sources-jar" value="${release.dir}/${artifactId}-${version}-sources.jar" />
-
-
- <!-- the normal classpath -->
- <path id="classpath">
- <pathelement location="${build.classes}" />
- <fileset dir="${build.lib.dir}">
- <include name="*.jar" />
- </fileset>
- </path>
-
- <!-- the unit test classpath -->
- <dirname property="plugins.classpath.dir" file="${build.plugins}" />
- <path id="test.classpath">
- <pathelement location="${test.build.classes}" />
- <pathelement location="${conf.dir}" />
- <pathelement location="${test.src.dir}" />
- <pathelement location="${plugins.classpath.dir}" />
- <path refid="classpath" />
- <pathelement location="${build.dir}/${final.name}.job" />
- <fileset dir="${build.lib.dir}">
- <include name="*.jar" />
- </fileset>
- </path>
-
- <!-- ====================================================== -->
- <!-- Stuff needed by all targets -->
- <!-- ====================================================== -->
- <target name="init" depends="ivy-init">
- <mkdir dir="${build.dir}" />
- <mkdir dir="${build.classes}" />
- <mkdir dir="${release.dir}" />
-
- <mkdir dir="${test.build.dir}" />
- <mkdir dir="${test.build.classes}" />
-
- <touch datetime="01/25/1971 2:00 pm">
- <fileset dir="${conf.dir}" includes="**/*.template" />
- </touch>
-
- <copy todir="${conf.dir}" verbose="true">
- <fileset dir="${conf.dir}" includes="**/*.template" />
- <mapper type="glob" from="*.template" to="*" />
- </copy>
- </target>
-
- <!-- ====================================================== -->
- <!-- Compile the Java files -->
- <!-- ====================================================== -->
- <target name="compile" depends="compile-core, compile-plugins" />
-
- <target name="compile-core" depends="init, resolve-default">
- <javac encoding="${build.encoding}" srcdir="${src.dir}" includes="org/apache/nutch/**/*.java" destdir="${build.classes}" debug="${javac.debug}" optimize="${javac.optimize}" target="${javac.version}" source="${javac.version}" deprecation="${javac.deprecation}">
- <classpath refid="classpath" />
- </javac>
- </target>
-
- <target name="compile-plugins" depends="init, resolve-default">
- <ant dir="src/plugin" target="deploy" inheritAll="false" />
- </target>
-
- <!-- ================================================================== -->
- <!-- Make nutch.jar -->
- <!-- ================================================================== -->
- <!-- -->
- <!-- ================================================================== -->
- <target name="jar" depends="compile-core">
- <copy file="${conf.dir}/nutch-default.xml" todir="${build.classes}" />
- <copy file="${conf.dir}/nutch-site.xml" todir="${build.classes}" />
- <jar jarfile="${build.dir}/${final.name}.jar" basedir="${build.classes}">
- <manifest>
- </manifest>
- </jar>
- </target>
-
- <!-- ================================================================== -->
- <!-- Make Maven Central Release -->
- <!-- ================================================================== -->
- <!-- -->
- <!-- ================================================================== -->
- <target name="release" depends="compile-core" description="generate the release distribution">
- <copy file="${conf.dir}/nutch-default.xml" todir="${build.classes}" />
- <copy file="${conf.dir}/nutch-site.xml" todir="${build.classes}" />
-
- <!-- build the main artifact -->
- <jar jarfile="${maven-jar}" basedir="${build.classes}" />
-
- <!-- build the javadoc artifact -->
- <javadoc destdir="${release.dir}/javadoc" overview="${src.dir}/overview.html" author="true" version="true" use="true" windowtitle="${Name} ${version} API" doctitle="${Name} ${version} API" bottom="Copyright &copy; ${year} The Apache Software Foundation">
- <arg value="${javadoc.proxy.host}" />
- <arg value="${javadoc.proxy.port}" />
-
- <packageset dir="${src.dir}" />
- <packageset dir="${plugins.dir}/lib-http/src/java" />
- <packageset dir="${plugins.dir}/lib-regex-filter/src/java" />
- <packageset dir="${plugins.dir}/microformats-reltag/src/java" />
- <packageset dir="${plugins.dir}/protocol-file/src/java" />
- <packageset dir="${plugins.dir}/protocol-ftp/src/java" />
- <packageset dir="${plugins.dir}/protocol-http/src/java" />
- <packageset dir="${plugins.dir}/protocol-httpclient/src/java" />
- <packageset dir="${plugins.dir}/parse-tika/src/java" />
- <packageset dir="${plugins.dir}/parse-ext/src/java" />
- <packageset dir="${plugins.dir}/parse-js/src/java" />
- <packageset dir="${plugins.dir}/parse-swf/src/java" />
- <packageset dir="${plugins.dir}/parse-zip/src/java" />
- <packageset dir="${plugins.dir}/index-basic/src/java" />
- <packageset dir="${plugins.dir}/index-more/src/java" />
- <packageset dir="${plugins.dir}/scoring-opic/src/java" />
- <packageset dir="${plugins.dir}/urlfilter-automaton/src/java" />
- <packageset dir="${plugins.dir}/urlfilter-regex/src/java" />
- <packageset dir="${plugins.dir}/urlfilter-prefix/src/java" />
- <packageset dir="${plugins.dir}/creativecommons/src/java" />
- <packageset dir="${plugins.dir}/languageidentifier/src/java" />
-
- <link href="${javadoc.link.java}" />
- <link href="${javadoc.link.lucene}" />
- <link href="${javadoc.link.hadoop}" />
-
- <classpath refid="classpath" />
- <classpath>
- <fileset dir="${plugins.dir}">
- <include name="**/*.jar" />
- </fileset>
- </classpath>
-
- <group title="Core" packages="org.apache.nutch.*" />
- <group title="Plugins API" packages="${plugins.api}" />
- <group title="Protocol Plugins" packages="${plugins.protocol}" />
- <group title="URL Filter Plugins" packages="${plugins.urlfilter}" />
- <group title="Scoring Plugins" packages="${plugins.scoring}" />
- <group title="Parse Plugins" packages="${plugins.parse}" />
- <group title="Indexing Filter Plugins" packages="${plugins.index}" />
- <group title="Misc. Plugins" packages="${plugins.misc}" />
- </javadoc>
- <jar jarfile="${maven-javadoc-jar}">
- <fileset dir="${release.dir}/javadoc" />
- </jar>
-
- <!-- build the sources artifact -->
- <jar jarfile="${maven-sources-jar}">
- <fileset dir="${src.dir}" />
- </jar>
- </target>
-
- <!-- ================================================================== -->
- <!-- Generate POM file -->
- <!-- The Ivy lib must be > 2.2.0. Run the tasks with "ant -lib ivy" to -->
- <!-- force it to use the right version -->
- <!-- ================================================================== -->
-
- <target name="generate-pom" description="Generate a pom file from ivy">
- <ivy:makepom ivyfile="${ivy.file}" pomfile="${basedir}/pom.xml" templatefile="ivy/mvn.template">
- <mapping conf="default" scope="compile" />
- <mapping conf="runtime" scope="runtime" />
- </ivy:makepom>
- </target>
-
- <!-- ================================================================== -->
- <!-- Deploy to Apache Nexus -->
- <!-- ================================================================== -->
- <!-- Note : the Maven jars must be in ANT lib directory -->
- <!-- See http://codehaus.org/~jvanzyl/maven2/ant-tasks.html -->
- <!-- ================================================================== -->
-
- <target name="deploy" depends="release" description="deploy to Apache Nexus">
-
- <antcall target="generate-pom" />
-
- <!-- sign and deploy the main artifact -->
- <artifact:mvn>
- <arg value="org.apache.maven.plugins:maven-gpg-plugin:1.1:sign-and-deploy-file" />
- <arg value="-Durl=${maven-repository-url}" />
- <arg value="-DrepositoryId=${maven-repository-id}" />
- <arg value="-DpomFile=pom.xml" />
- <arg value="-Dfile=${maven-jar}" />
- <arg value="-Papache-release" />
- </artifact:mvn>
-
- <!-- sign and deploy the sources artifact -->
- <artifact:mvn>
- <arg value="org.apache.maven.plugins:maven-gpg-plugin:1.1:sign-and-deploy-file" />
- <arg value="-Durl=${maven-repository-url}" />
- <arg value="-DrepositoryId=${maven-repository-id}" />
- <arg value="-DpomFile=pom.xml" />
- <arg value="-Dfile=${maven-sources-jar}" />
- <arg value="-Dclassifier=sources" />
- <arg value="-Papache-release" />
- </artifact:mvn>
-
- <!-- sign and deploy the javadoc artifact -->
- <artifact:mvn>
- <arg value="org.apache.maven.plugins:maven-gpg-plugin:1.1:sign-and-deploy-file" />
- <arg value="-Durl=${maven-repository-url}" />
- <arg value="-DrepositoryId=${maven-repository-id}" />
- <arg value="-DpomFile=pom.xml" />
- <arg value="-Dfile=${maven-javadoc-jar}" />
- <arg value="-Dclassifier=javadoc" />
- <arg value="-Papache-release" />
- </artifact:mvn>
- </target>
-
- <!-- ================================================================== -->
- <!-- Make job jar -->
- <!-- ================================================================== -->
- <!-- -->
- <!-- ================================================================== -->
- <target name="job" depends="compile">
- <jar jarfile="${build.dir}/${final.name}.job">
- <!-- If the build.classes has the nutch config files because the jar
+ <!-- Load all the default properties, and any the user wants -->
+ <!-- to contribute (without having to type -D or edit this file -->
+ <property file="${user.home}/build.properties" />
+ <property file="${basedir}/build.properties" />
+ <property file="${basedir}/default.properties" />
+ <property name="test.junit.output.format" value="plain"/>
+ <property name="release.dir" value="${build.dir}/release"/>
+
+ <!-- define Maven coordinates, repository url and artifacts name etc -->
+ <property name="groupId" value="org.apache.nutch" />
+ <property name="artifactId" value="nutch" />
+ <property name="maven-repository-url" value="https://repository.apache.org/service/local/staging/deploy/maven2" />
+ <property name="maven-repository-id" value="apache.releases.https" />
+ <property name="maven-jar" value="${release.dir}/${artifactId}-${version}.jar" />
+ <property name="maven-javadoc-jar" value="${release.dir}/${artifactId}-${version}-javadoc.jar" />
+ <property name="maven-sources-jar" value="${release.dir}/${artifactId}-${version}-sources.jar" />
+
+
+ <!-- the normal classpath -->
+ <path id="classpath">
+ <pathelement location="${build.classes}"/>
+ <fileset dir="${build.lib.dir}">
+ <include name="*.jar" />
+ </fileset>
+ </path>
+
+ <!-- the unit test classpath -->
+ <dirname property="plugins.classpath.dir" file="${build.plugins}"/>
+ <path id="test.classpath">
+ <pathelement location="${test.build.classes}" />
+ <pathelement location="${conf.dir}"/>
+ <pathelement location="${test.src.dir}"/>
+ <pathelement location="${plugins.classpath.dir}"/>
+ <path refid="classpath"/>
+ <pathelement location="${build.dir}/${final.name}.job" />
+ <fileset dir="${build.lib.dir}">
+ <include name="*.jar" />
+ </fileset>
+ </path>
+
+ <!-- ====================================================== -->
+ <!-- Stuff needed by all targets -->
+ <!-- ====================================================== -->
+ <target name="init" depends="ivy-init">
+ <mkdir dir="${build.dir}"/>
+ <mkdir dir="${build.classes}"/>
+ <mkdir dir="${release.dir}"/>
+
+ <mkdir dir="${test.build.dir}"/>
+ <mkdir dir="${test.build.classes}"/>
+
+ <touch datetime="01/25/1971 2:00 pm">
+ <fileset dir="${conf.dir}" includes="**/*.template"/>
+ </touch>
+
+ <copy todir="${conf.dir}" verbose="true">
+ <fileset dir="${conf.dir}" includes="**/*.template"/>
+ <mapper type="glob" from="*.template" to="*"/>
+ </copy>
+ </target>
+
+ <!-- ====================================================== -->
+ <!-- Compile the Java files -->
+ <!-- ====================================================== -->
+ <target name="compile" depends="compile-core, compile-plugins"/>
+
+ <target name="compile-core" depends="init, resolve-default">
+ <javac
+ encoding="${build.encoding}"
+ srcdir="${src.dir}"
+ includes="org/apache/nutch/**/*.java"
+ destdir="${build.classes}"
+ debug="${javac.debug}"
+ optimize="${javac.optimize}"
+ target="${javac.version}"
+ source="${javac.version}"
+ deprecation="${javac.deprecation}">
+ <classpath refid="classpath"/>
+ </javac>
+ </target>
+
+ <target name="compile-plugins" depends="init, resolve-default">
+ <ant dir="src/plugin" target="deploy" inheritAll="false"/>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Make nutch.jar -->
+ <!-- ================================================================== -->
+ <!-- -->
+ <!-- ================================================================== -->
+ <target name="jar" depends="compile-core">
+ <copy file="${conf.dir}/nutch-default.xml"
+ todir="${build.classes}"/>
+ <copy file="${conf.dir}/nutch-site.xml"
+ todir="${build.classes}"/>
+ <jar jarfile="${build.dir}/${final.name}.jar"
+ basedir="${build.classes}">
+ <manifest>
+ </manifest>
+ </jar>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Make Maven Central Release -->
+ <!-- ================================================================== -->
+ <!-- -->
+ <!-- ================================================================== -->
+ <target name="release" depends="compile-core" description="generate the release distribution">
+ <copy file="${conf.dir}/nutch-default.xml"
+ todir="${build.classes}"/>
+ <copy file="${conf.dir}/nutch-site.xml"
+ todir="${build.classes}"/>
+
+ <!-- build the main artifact -->
+ <jar jarfile="${maven-jar}" basedir="${build.classes}" />
+
+ <!-- build the javadoc artifact -->
+ <javadoc
+ destdir="${release.dir}/javadoc"
+ overview="${src.dir}/overview.html"
+ author="true"
+ version="true"
+ use="true"
+ windowtitle="${Name} ${version} API"
+ doctitle="${Name} ${version} API"
+ bottom="Copyright &copy; ${year} The Apache Software Foundation"
+ >
+ <arg value="${javadoc.proxy.host}"/>
+ <arg value="${javadoc.proxy.port}"/>
+
+ <packageset dir="${src.dir}"/>
+ <packageset dir="${plugins.dir}/lib-http/src/java"/>
+ <packageset dir="${plugins.dir}/lib-regex-filter/src/java"/>
+ <packageset dir="${plugins.dir}/microformats-reltag/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-file/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-ftp/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-http/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-httpclient/src/java"/>
+ <packageset dir="${plugins.dir}/parse-tika/src/java"/>
+ <packageset dir="${plugins.dir}/parse-ext/src/java"/>
+ <packageset dir="${plugins.dir}/parse-js/src/java"/>
+ <packageset dir="${plugins.dir}/parse-swf/src/java"/>
+ <packageset dir="${plugins.dir}/parse-zip/src/java"/>
+ <packageset dir="${plugins.dir}/index-basic/src/java"/>
+ <packageset dir="${plugins.dir}/index-more/src/java"/>
+ <packageset dir="${plugins.dir}/scoring-opic/src/java"/>
+ <packageset dir="${plugins.dir}/urlfilter-automaton/src/java"/>
+ <packageset dir="${plugins.dir}/urlfilter-regex/src/java"/>
+ <packageset dir="${plugins.dir}/urlfilter-prefix/src/java"/>
+ <packageset dir="${plugins.dir}/creativecommons/src/java"/>
+ <packageset dir="${plugins.dir}/languageidentifier/src/java"/>
+
+ <link href="${javadoc.link.java}"/>
+ <link href="${javadoc.link.lucene}"/>
+ <link href="${javadoc.link.hadoop}"/>
+
+ <classpath refid="classpath"/>
+ <classpath>
+ <fileset dir="${plugins.dir}" >
+ <include name="**/*.jar"/>
+ </fileset>
+ </classpath>
+
+ <group title="Core" packages="org.apache.nutch.*"/>
+ <group title="Plugins API" packages="${plugins.api}"/>
+ <group title="Protocol Plugins" packages="${plugins.protocol}"/>
+ <group title="URL Filter Plugins" packages="${plugins.urlfilter}"/>
+ <group title="Scoring Plugins" packages="${plugins.scoring}"/>
+ <group title="Parse Plugins" packages="${plugins.parse}"/>
+ <group title="Indexing Filter Plugins" packages="${plugins.index}"/>
+ <group title="Misc. Plugins" packages="${plugins.misc}"/>
+ </javadoc>
+ <jar jarfile="${maven-javadoc-jar}">
+ <fileset dir="${release.dir}/javadoc" />
+ </jar>
+
+ <!-- build the sources artifact -->
+ <jar jarfile="${maven-sources-jar}">
+ <fileset dir="${src.dir}" />
+ </jar>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Deploy to Apache Nexus -->
+ <!-- ================================================================== -->
+ <!-- -->
+ <!-- ================================================================== -->
+ <target name="deploy" depends="release" description="deploy to Apache Nexus">
+ <!-- sign and deploy the main artifact -->
+ <artifact:mvn>
+ <arg value="org.apache.maven.plugins:maven-gpg-plugin:1.1:sign-and-deploy-file" />
+ <arg value="-Durl=${maven-repository-url}" />
+ <arg value="-DrepositoryId=${maven-repository-id}" />
+ <arg value="-DpomFile=pom.xml" />
+ <arg value="-Dfile=${maven-jar}" />
+ <arg value="-Papache-release" />
+ </artifact:mvn>
+
+ <!-- sign and deploy the sources artifact -->
+ <artifact:mvn>
+ <arg value="org.apache.maven.plugins:maven-gpg-plugin:1.1:sign-and-deploy-file" />
+ <arg value="-Durl=${maven-repository-url}" />
+ <arg value="-DrepositoryId=${maven-repository-id}" />
+ <arg value="-DpomFile=pom.xml" />
+ <arg value="-Dfile=${maven-sources-jar}" />
+ <arg value="-Dclassifier=sources" />
+ <arg value="-Papache-release" />
+ </artifact:mvn>
+
+ <!-- sign and deploy the javadoc artifact -->
+ <artifact:mvn>
+ <arg value="org.apache.maven.plugins:maven-gpg-plugin:1.1:sign-and-deploy-file" />
+ <arg value="-Durl=${maven-repository-url}" />
+ <arg value="-DrepositoryId=${maven-repository-id}" />
+ <arg value="-DpomFile=pom.xml" />
+ <arg value="-Dfile=${maven-javadoc-jar}" />
+ <arg value="-Dclassifier=javadoc" />
+ <arg value="-Papache-release" />
+ </artifact:mvn>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Make job jar -->
+ <!-- ================================================================== -->
+ <!-- -->
+ <!-- ================================================================== -->
+ <target name="job" depends="compile">
+ <jar jarfile="${build.dir}/${final.name}.job">
+ <!-- If the build.classes has the nutch config files because the jar
command command has run, exclude them. The conf directory has
them.
-->
- <zipfileset dir="${build.classes}" excludes="nutch-default.xml,nutch-site.xml" />
- <zipfileset dir="${conf.dir}" excludes="*.template,hadoop*.*" />
- <zipfileset dir="${build.lib.dir}" prefix="lib" includes="**/*.jar" excludes="hadoop-*.jar" />
- <zipfileset dir="${build.plugins}" prefix="plugins" />
- </jar>
- </target>
-
- <target name="runtime" depends="jar, job">
- <mkdir dir="${runtime.dir}" />
- <mkdir dir="${runtime.local}" />
- <mkdir dir="${runtime.deploy}" />
- <!-- deploy area -->
- <copy file="${build.dir}/${final.name}.job" todir="${runtime.deploy}" />
- <copy todir="${runtime.deploy}/bin">
- <fileset dir="src/bin" />
- </copy>
- <chmod perm="ugo+x" type="file">
- <fileset dir="${runtime.deploy}/bin" />
- </chmod>
- <!-- local area -->
- <copy file="${build.dir}/${final.name}.jar" todir="${runtime.local}/lib" />
- <copy todir="${runtime.local}/lib/native">
- <fileset dir="lib/native" />
- </copy>
- <copy todir="${runtime.local}/conf">
- <fileset dir="conf" excludes="*.template" />
- </copy>
- <copy todir="${runtime.local}/bin">
- <fileset dir="src/bin" />
- </copy>
- <chmod perm="ugo+x" type="file">
- <fileset dir="${runtime.local}/bin" />
- </chmod>
- <copy todir="${runtime.local}/lib">
- <fileset dir="${build.dir}/lib" />
- </copy>
- <copy todir="${runtime.local}/plugins">
- <fileset dir="${build.dir}/plugins" />
- </copy>
- </target>
-
- <!-- ================================================================== -->
- <!-- Compile test code -->
- <!-- ================================================================== -->
- <target name="compile-core-test" depends="compile-core, resolve-test">
- <javac encoding="${build.encoding}" srcdir="${test.src.dir}" includes="org/apache/nutch/**/*.java" destdir="${test.build.classes}" debug="${javac.debug}" optimize="${javac.optimize}" target="${javac.version}" source="${javac.version}" deprecation="${javac.deprecation}">
- <classpath refid="test.classpath" />
- </javac>
- </target>
-
- <!-- ================================================================== -->
- <!-- Run code checks (PMD) -->
- <!-- ================================================================== -->
- <target name="pmd" depends="compile">
- <property name="pmd.report" location="${build.dir}/pmd-report.html" />
- <taskdef name="pmd" classname="net.sourceforge.pmd.ant.PMDTask">
- <classpath>
- <fileset dir="${lib.dir}">
- <include name="pmd-ext/*.jar" />
- <include name="xerces*.jar" />
- </fileset>
- </classpath>
- </taskdef>
- <pmd shortFilenames="true" failonerror="true" failOnRuleViolation="false" encoding="${build.encoding}" failuresPropertyName="pmd.failures">
- <ruleset>unusedcode</ruleset>
- <!--ruleset>basic</ruleset-->
- <!--ruleset>optimizations</ruleset-->
- <formatter type="html" toFile="${pmd.report}" />
- <!-- <formatter type="xml" toFile="${tempbuild}/$report_pmd.xml"/> -->
- <fileset dir="${basedir}/src">
- <include name="java/**/*.java" />
- <include name="plugin/**/*.java" />
- </fileset>
- </pmd>
- <condition property="pmd.stop" value="true">
- <and>
- <isset property="pmd.failures" />
- <not>
- <equals arg1="0" arg2="${pmd.failures}" trim="true" />
- </not>
- </and>
- </condition>
- <fail if="pmd.stop">FAILURE: PMD shows ${pmd.failures} rule violations. See ${pmd.report} for details.</fail>
- </target>
-
- <target name="proxy" depends="job, compile-core-test">
- <java classname="org.apache.nutch.tools.proxy.TestbedProxy" fork="true">
- <classpath refid="test.classpath" />
- <arg value="-fake" />
- <!--
+ <zipfileset dir="${build.classes}"
+ excludes="nutch-default.xml,nutch-site.xml"/>
+ <zipfileset dir="${conf.dir}" excludes="*.template,hadoop*.*"/>
+ <zipfileset dir="${build.lib.dir}" prefix="lib"
+ includes="**/*.jar" excludes="hadoop-*.jar"/>
+ <zipfileset dir="${build.plugins}" prefix="plugins"/>
+ </jar>
+ </target>
+
+ <target name="runtime" depends="jar, job">
+ <mkdir dir="${runtime.dir}"/>
+ <mkdir dir="${runtime.local}"/>
+ <mkdir dir="${runtime.deploy}"/>
+ <!-- deploy area -->
+ <copy file="${build.dir}/${final.name}.job"
+ todir="${runtime.deploy}"/>
+ <copy todir="${runtime.deploy}/bin">
+ <fileset dir="src/bin"/>
+ </copy>
+ <chmod perm="ugo+x" type="file">
+ <fileset dir="${runtime.deploy}/bin"/>
+ </chmod>
+ <!-- local area -->
+ <copy file="${build.dir}/${final.name}.jar"
+ todir="${runtime.local}/lib"/>
+ <copy todir="${runtime.local}/lib/native">
+ <fileset dir="lib/native"/>
+ </copy>
+ <copy todir="${runtime.local}/conf">
+ <fileset dir="conf" excludes="*.template"/>
+ </copy>
+ <copy todir="${runtime.local}/bin">
+ <fileset dir="src/bin"/>
+ </copy>
+ <chmod perm="ugo+x" type="file">
+ <fileset dir="${runtime.local}/bin"/>
+ </chmod>
+ <copy todir="${runtime.local}/lib">
+ <fileset dir="${build.dir}/lib"/>
+ </copy>
+ <copy todir="${runtime.local}/plugins">
+ <fileset dir="${build.dir}/plugins"/>
+ </copy>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Compile test code -->
+ <!-- ================================================================== -->
+ <target name="compile-core-test" depends="compile-core, resolve-test">
+ <javac
+ encoding="${build.encoding}"
+ srcdir="${test.src.dir}"
+ includes="org/apache/nutch/**/*.java"
+ destdir="${test.build.classes}"
+ debug="${javac.debug}"
+ optimize="${javac.optimize}"
+ target="${javac.version}"
+ source="${javac.version}"
+ deprecation="${javac.deprecation}">
+ <classpath refid="test.classpath"/>
+ </javac>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Run code checks (PMD) -->
+ <!-- ================================================================== -->
+ <target name="pmd" depends="compile">
+ <property name="pmd.report" location="${build.dir}/pmd-report.html" />
+ <taskdef name="pmd" classname="net.sourceforge.pmd.ant.PMDTask">
+ <classpath>
+ <fileset dir="${lib.dir}">
+ <include name="pmd-ext/*.jar" />
+ <include name="xerces*.jar" />
+ </fileset>
+ </classpath>
+ </taskdef>
+ <pmd shortFilenames="true" failonerror="true" failOnRuleViolation="false"
+ encoding="${build.encoding}" failuresPropertyName="pmd.failures">
+ <ruleset>unusedcode</ruleset>
+ <!--ruleset>basic</ruleset-->
+ <!--ruleset>optimizations</ruleset-->
+ <formatter type="html" toFile="${pmd.report}" />
+ <!-- <formatter type="xml" toFile="${tempbuild}/$report_pmd.xml"/> -->
+ <fileset dir="${basedir}/src">
+ <include name="java/**/*.java"/>
+ <include name="plugin/**/*.java"/>
+ </fileset>
+ </pmd>
+ <condition property="pmd.stop" value="true">
+ <and>
+ <isset property="pmd.failures" />
+ <not>
+ <equals arg1="0" arg2="${pmd.failures}" trim="true" />
+ </not>
+ </and>
+ </condition>
+ <fail if="pmd.stop">FAILURE: PMD shows ${pmd.failures} rule violations. See ${pmd.report} for details.</fail>
+ </target>
+
+ <target name="proxy" depends="job, compile-core-test">
+ <java classname="org.apache.nutch.tools.proxy.TestbedProxy" fork="true">
+ <classpath refid="test.classpath"/>
+ <arg value="-fake"/>
+<!--
<arg value="-delay"/>
<arg value="-200"/>
-->
- <jvmarg line="-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl" />
- </java>
- </target>
-
- <target name="benchmark">
- <java classname="org.apache.nutch.tools.Benchmark" fork="true">
- <classpath refid="test.classpath" />
- <jvmarg line="-Xmx512m -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl" />
- <arg value="-maxPerHost" />
- <arg value="10" />
- <arg value="-seeds" />
- <arg value="1" />
- <arg value="-depth" />
- <arg value="5" />
- </java>
- </target>
-
- <!-- ================================================================== -->
- <!-- Run unit tests -->
- <!-- ================================================================== -->
- <target name="test" depends="test-core, test-plugins" />
-
- <target name="test-core" depends="job, compile-core-test">
-
- <delete dir="${test.build.data}" />
- <mkdir dir="${test.build.data}" />
- <!--
+ <jvmarg line="-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl"/>
+ </java>
+ </target>
+
+ <target name="benchmark">
+ <java classname="org.apache.nutch.tools.Benchmark" fork="true">
+ <classpath refid="test.classpath"/>
+ <jvmarg line="-Xmx512m -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl"/>
+ <arg value="-maxPerHost"/>
+ <arg value="10"/>
+ <arg value="-seeds"/>
+ <arg value="1"/>
+ <arg value="-depth"/>
+ <arg value="5"/>
+ </java>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Run unit tests -->
+ <!-- ================================================================== -->
+ <target name="test" depends="test-core, test-plugins"/>
+
+ <target name="test-core" depends="job, compile-core-test">
+
+ <delete dir="${test.build.data}"/>
+ <mkdir dir="${test.build.data}"/>
+ <!--
copy resources needed in junit tests
-->
- <copy todir="${test.build.data}">
- <fileset dir="src/testresources" includes="**/*" />
- </copy>
- <copy file="${test.src.dir}/nutch-site.xml" todir="${test.build.classes}" />
-
- <copy file="${test.src.dir}/log4j.properties" todir="${test.build.classes}" />
-
- <junit printsummary="yes" haltonfailure="no" fork="yes" dir="${basedir}" errorProperty="tests.failed" failureProperty="tests.failed" maxmemory="1000m">
- <sysproperty key="test.build.data" value="${test.build.data}" />
- <sysproperty key="test.src.dir" value="${test.src.dir}" />
- <sysproperty key="javax.xml.parsers.DocumentBuilderFactory" value="com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl" />
- <classpath refid="test.classpath" />
- <formatter type="${test.junit.output.format}" />
- <batchtest todir="${test.build.dir}" unless="testcase">
- <fileset dir="${test.src.dir}" includes="**/Test*.java" excludes="**/${test.exclude}.java" />
- </batchtest>
- <batchtest todir="${test.build.dir}" if="testcase">
- <fileset dir="${test.src.dir}" includes="**/${testcase}.java" />
- </batchtest>
- </junit>
-
- <fail if="tests.failed">Tests failed!</fail>
-
- </target>
-
- <target name="test-plugins" depends="compile">
- <ant dir="src/plugin" target="test" inheritAll="false" />
- </target>
-
- <target name="nightly" depends="test, tar">
- </target>
-
- <!-- ================================================================== -->
- <!-- Ivy targets -->
- <!-- ================================================================== -->
-
- <!-- target: resolve ================================================= -->
- <target name="resolve-default" depends="clean-lib" description="--> resolve and retrieve dependencies with ivy">
- <ivy:resolve file="${ivy.file}" conf="default" log="download-only" />
- <ivy:retrieve pattern="${build.lib.dir}/[artifact]-[revision].[ext]" symlink="false" log="quiet" />
- <antcall target="copy-libs" />
- </target>
-
- <target name="resolve-test" depends="clean-lib" description="--> resolve and retrieve dependencies with ivy">
- <ivy:resolve file="${ivy.file}" conf="test" log="download-only" />
- <ivy:retrieve pattern="${build.lib.dir}/[artifact]-[revision].[ext]" symlink="false" log="quiet" />
- <antcall target="copy-libs" />
- </target>
-
- <target name="copy-libs">
- <!-- copy the libs in lib, which are not ivy enabled-->
- <copy todir="${build.lib.dir}/" failonerror="false">
- <fileset dir="${lib.dir}" includes="**/*.jar" />
- </copy>
- </target>
-
- <!-- target: publish-local =========================================== -->
- <target name="publish-local" depends="jar" description="--> publish this project in the local ivy repository">
- <ivy:publish artifactspattern="${build.dir}/[artifact]-${version}.[ext]" resolver="local" pubrevision="${version}" pubdate="${now}" status="integration" forcedeliver="true" overwrite="true" />
- <echo message="project ${ant.project.name} published locally with version ${version}" />
- </target>
-
- <!-- target: report ================================================== -->
- <target name="report" depends="resolve-test" description="--> generates a report of dependencies">
- <ivy:report todir="${build.dir}" />
- </target>
-
- <!-- target: ivy-init ================================================ -->
- <target name="ivy-init" depends="ivy-probe-antlib, ivy-init-antlib">
- <ivy:settings file="${ivy.dir}/ivysettings.xml" />
- </target>
-
- <!-- target: ivy-probe-antlib ======================================== -->
- <target name="ivy-probe-antlib">
- <condition property="ivy.found">
- <typefound uri="antlib:org.apache.ivy.ant" name="cleancache" />
- </condition>
- </target>
-
- <!-- target: ivy-download ============================================ -->
- <target name="ivy-download" description="Download ivy">
- <available file="${ivy.jar}" property="ivy.jar.found" />
- <antcall target="-ivy-download-unchecked" />
- </target>
-
- <!-- target: ivy-download-unchecked ================================== -->
- <target name="-ivy-download-unchecked" unless="ivy.jar.found">
- <get src="${ivy.repo.url}" dest="${ivy.jar}" usetimestamp="true" />
- </target>
-
- <!-- target: ivy-init-antlib ========================================= -->
- <target name="ivy-init-antlib" depends="ivy-download" unless="ivy.found">
- <typedef uri="antlib:org.apache.ivy.ant" onerror="fail" loaderRef="ivyLoader">
- <classpath>
- <pathelement location="${ivy.jar}" />
- </classpath>
- </typedef>
- <fail>
- <condition>
- <not>
- <typefound uri="antlib:org.apache.ivy.ant" name="cleancache" />
- </not>
- </condition>
- You need Apache Ivy 2.2.0 or later from http://ant.apache.org/
+ <copy todir="${test.build.data}">
+ <fileset dir="src/testresources" includes="**/*"/>
+ </copy>
+ <copy file="${test.src.dir}/nutch-site.xml"
+ todir="${test.build.classes}"/>
+
+ <copy file="${test.src.dir}/log4j.properties"
+ todir="${test.build.classes}"/>
+
+ <junit printsummary="yes" haltonfailure="no" fork="yes" dir="${basedir}"
+ errorProperty="tests.failed" failureProperty="tests.failed" maxmemory="1000m">
+ <sysproperty key="test.build.data" value="${test.build.data}"/>
+ <sysproperty key="test.src.dir" value="${test.src.dir}"/>
+ <sysproperty key="javax.xml.parsers.DocumentBuilderFactory" value="com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl"/>
+ <classpath refid="test.classpath"/>
+ <formatter type="${test.junit.output.format}" />
+ <batchtest todir="${test.build.dir}" unless="testcase">
+ <fileset dir="${test.src.dir}"
+ includes="**/Test*.java" excludes="**/${test.exclude}.java" />
+ </batchtest>
+ <batchtest todir="${test.build.dir}" if="testcase">
+ <fileset dir="${test.src.dir}" includes="**/${testcase}.java"/>
+ </batchtest>
+ </junit>
+
+ <fail if="tests.failed">Tests failed!</fail>
+
+ </target>
+
+ <target name="test-plugins" depends="compile">
+ <ant dir="src/plugin" target="test" inheritAll="false"/>
+ </target>
+
+ <target name="nightly" depends="test, tar">
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Ivy targets -->
+ <!-- ================================================================== -->
+
+ <!-- target: resolve ================================================= -->
+ <target name="resolve-default" depends="clean-lib" description="--> resolve and retrieve dependencies with ivy">
+ <ivy:resolve file="${ivy.file}" conf="default" log="download-only"/>
+ <ivy:retrieve pattern="${build.lib.dir}/[artifact]-[revision].[ext]" symlink="false" log="quiet"/>
+ <antcall target="copy-libs"/>
+ </target>
+
+ <target name="resolve-test" depends="clean-lib" description="--> resolve and retrieve dependencies with ivy">
+ <ivy:resolve file="${ivy.file}" conf="test" log="download-only"/>
+ <ivy:retrieve pattern="${build.lib.dir}/[artifact]-[revision].[ext]" symlink="false" log="quiet"/>
+ <antcall target="copy-libs"/>
+ </target>
+
+ <target name="copy-libs">
+ <!-- copy the libs in lib, which are not ivy enabled-->
+ <copy todir="${build.lib.dir}/" failonerror="false">
+ <fileset dir="${lib.dir}" includes="**/*.jar"/>
+ </copy>
+ </target>
+
+ <!-- target: publish-local =========================================== -->
+ <target name="publish-local" depends="jar" description="--> publish this project in the local ivy repository">
+ <ivy:publish artifactspattern="${build.dir}/[artifact]-${version}.[ext]"
+ resolver="local"
+ pubrevision="${version}"
+ pubdate="${now}"
+ status="integration"
+ forcedeliver="true"
+ overwrite="true"
+ />
+ <echo message="project ${ant.project.name} published locally with version ${version}" />
+ </target>
+
+ <!-- target: report ================================================== -->
+ <target name="report" depends="resolve-test" description="--> generates a report of dependencies">
+ <ivy:report todir="${build.dir}"/>
+ </target>
+
+ <!-- target: ivy-init ================================================ -->
+ <target name="ivy-init" depends="ivy-probe-antlib, ivy-init-antlib">
+ <ivy:settings file="${ivy.dir}/ivysettings.xml" />
+ </target>
+
+ <!-- target: ivy-probe-antlib ======================================== -->
+ <target name="ivy-probe-antlib">
+ <condition property="ivy.found">
+ <typefound uri="antlib:org.apache.ivy.ant" name="cleancache" />
+ </condition>
+ </target>
+
+ <!-- target: ivy-download ============================================ -->
+ <target name="ivy-download" description="Download ivy">
+ <available file="${ivy.jar}" property="ivy.jar.found"/>
+ <antcall target="-ivy-download-unchecked"/>
+ </target>
+
+ <!-- target: ivy-download-unchecked ================================== -->
+ <target name="-ivy-download-unchecked" unless="ivy.jar.found">
+ <get src="${ivy.repo.url}" dest="${ivy.jar}" usetimestamp="true" />
+ </target>
+
+ <!-- target: ivy-init-antlib ========================================= -->
+ <target name="ivy-init-antlib" depends="ivy-download" unless="ivy.found">
+ <typedef uri="antlib:org.apache.ivy.ant" onerror="fail" loaderRef="ivyLoader">
+ <classpath>
+ <pathelement location="${ivy.jar}" />
+ </classpath>
+ </typedef>
+ <fail>
+ <condition>
+ <not>
+ <typefound uri="antlib:org.apache.ivy.ant" name="cleancache" />
+ </not>
+ </condition>
+ You need Apache Ivy 2.0 or later from http://ant.apache.org/
It could not be loaded from ${ivy.repo.url}
</fail>
- </target>
-
- <!-- ================================================================== -->
- <!-- Documentation -->
- <!-- ================================================================== -->
- <target name="javadoc" depends="compile">
- <mkdir dir="${build.javadoc}" />
- <javadoc overview="${src.dir}/overview.html" destdir="${build.javadoc}" author="true" version="true" use="true" windowtitle="${Name} ${version} API" doctitle="${Name} ${version} API" bottom="Copyright &copy; ${year} The Apache Software Foundation">
- <arg value="${javadoc.proxy.host}" />
- <arg value="${javadoc.proxy.port}" />
-
- <packageset dir="${src.dir}" />
- <packageset dir="${plugins.dir}/lib-http/src/java" />
- <packageset dir="${plugins.dir}/lib-regex-filter/src/java" />
- <packageset dir="${plugins.dir}/microformats-reltag/src/java" />
- <packageset dir="${plugins.dir}/protocol-file/src/java" />
- <packageset dir="${plugins.dir}/protocol-ftp/src/java" />
- <packageset dir="${plugins.dir}/protocol-http/src/java" />
- <packageset dir="${plugins.dir}/protocol-httpclient/src/java" />
- <packageset dir="${plugins.dir}/parse-tika/src/java" />
- <packageset dir="${plugins.dir}/parse-ext/src/java" />
- <packageset dir="${plugins.dir}/parse-js/src/java" />
- <packageset dir="${plugins.dir}/parse-swf/src/java" />
- <packageset dir="${plugins.dir}/parse-zip/src/java" />
- <packageset dir="${plugins.dir}/index-basic/src/java" />
- <packageset dir="${plugins.dir}/index-more/src/java" />
- <packageset dir="${plugins.dir}/scoring-opic/src/java" />
- <packageset dir="${plugins.dir}/urlfilter-automaton/src/java" />
- <packageset dir="${plugins.dir}/urlfilter-regex/src/java" />
- <packageset dir="${plugins.dir}/urlfilter-prefix/src/java" />
- <packageset dir="${plugins.dir}/creativecommons/src/java" />
- <packageset dir="${plugins.dir}/languageidentifier/src/java" />
-
- <link href="${javadoc.link.java}" />
- <link href="${javadoc.link.lucene}" />
- <link href="${javadoc.link.hadoop}" />
-
- <classpath refid="classpath" />
- <classpath>
- <fileset dir="${plugins.dir}">
- <include name="**/*.jar" />
- </fileset>
- </classpath>
-
- <group title="Core" packages="org.apache.nutch.*" />
- <group title="Plugins API" packages="${plugins.api}" />
- <group title="Protocol Plugins" packages="${plugins.protocol}" />
- <group title="URL Filter Plugins" packages="${plugins.urlfilter}" />
- <group title="Scoring Plugins" packages="${plugins.scoring}" />
- <group title="Parse Plugins" packages="${plugins.parse}" />
- <group title="Indexing Filter Plugins" packages="${plugins.index}" />
- <group title="Misc. Plugins" packages="${plugins.misc}" />
- </javadoc>
- <!-- Copy the plugin.dtd file to the plugin doc-files dir -->
- <copy file="${plugins.dir}/plugin.dtd" todir="${build.javadoc}/org/apache/nutch/plugin/doc-files" />
- </target>
-
- <target name="default-doc">
- <style basedir="${conf.dir}" destdir="${docs.dir}" includes="nutch-default.xml" style="conf/nutch-conf.xsl" />
- </target>
-
- <!-- ================================================================== -->
- <!-- D I S T R I B U T I O N -->
- <!-- ================================================================== -->
- <!-- -->
- <!-- ================================================================== -->
- <target name="package" depends="runtime, javadoc">
- <mkdir dir="${dist.version.dir}" />
- <mkdir dir="${dist.version.dir}/lib" />
- <mkdir dir="${dist.version.dir}/runtime" />
- <mkdir dir="${dist.version.dir}/docs" />
- <mkdir dir="${dist.version.dir}/docs/api" />
- <mkdir dir="${dist.version.dir}/plugins" />
-
- <copy todir="${dist.version.dir}/lib" includeEmptyDirs="false">
- <fileset dir="${build.lib.dir}" />
- </copy>
-
- <copy todir="${dist.version.dir}/plugins">
- <fileset dir="${build.plugins}" />
- </copy>
-
- <copy file="${build.dir}/${final.name}.jar" todir="${dist.version.dir}" />
- <copy file="${build.dir}/${final.name}.job" todir="${dist.version.dir}" />
-
- <copy todir="${dist.version.dir}/runtime">
- <fileset dir="runtime" />
- </copy>
-
- <chmod perm="ugo+x" type="file">
- <fileset dir="${dist.version.dir}/runtime/deploy/bin" />
- <fileset dir="${dist.version.dir}/runtime/local/bin" />
- </chmod>
-
- <copy todir="${dist.version.dir}/conf">
- <fileset dir="${conf.dir}" excludes="**/*.template" />
- </copy>
-
- <copy todir="${dist.version.dir}/docs/api">
- <fileset dir="${build.javadoc}" />
- </copy>
-
- <copy todir="${dist.version.dir}">
- <fileset dir=".">
- <include name="*.txt" />
- <include name="KEYS" />
- </fileset>
- </copy>
-
- <copy todir="${dist.version.dir}/src" includeEmptyDirs="true">
- <fileset dir="src" />
- </copy>
-
- <copy todir="${dist.version.dir}/" file="build.xml" />
- <copy todir="${dist.version.dir}/" file="default.properties" />
-
- </target>
-
- <!-- ================================================================== -->
- <!-- Make release tarball -->
- <!-- ================================================================== -->
- <target name="tar" depends="package">
- <tar compression="gzip" longfile="gnu" destfile="${dist.dir}/${final.name}.tar.gz">
- <tarfileset dir="${dist.dir}" mode="664">
- <exclude name="${final.name}/bin/*" />
- <include name="${final.name}/**" />
- </tarfileset>
- <tarfileset dir="${dist.dir}" mode="755">
- <include name="${final.name}/bin/*" />
- </tarfileset>
- </tar>
- </target>
-
- <!-- ================================================================== -->
- <!-- Make release tarball -->
- <!-- ================================================================== -->
- <target name="zip" depends="package">
- <zip compress="true" casesensitive="yes" destfile="${dist.dir}/${final.name}.zip">
- <zipfileset dir="${dist.dir}" filemode="664">
- <exclude name="${final.name}/bin/*" />
- <include name="${final.name}/**" />
- </zipfileset>
- <zipfileset dir="${dist.dir}" filemode="755">
- <include name="${final.name}/bin/*" />
- </zipfileset>
- </zip>
- </target>
-
- <!-- ================================================================== -->
- <!-- Clean. Delete the build files, and their directories -->
- <!-- ================================================================== -->
-
- <!-- target: clean =================================================== -->
- <target name="clean" depends="clean-build, clean-lib, clean-dist, clean-runtime" description="--> clean the project" />
-
- <!-- target: clean-local ============================================= -->
- <target name="clean-local" depends="" description="--> cleans the local repository for the current module">
- <delete dir="${ivy.local.default.root}/${ivy.organisation}/${ivy.module}" />
- </target>
-
- <!-- target: clean-lib =============================================== -->
- <target name="clean-lib" description="--> clean the project libraries directory (dependencies)">
- <delete includeemptydirs="true" dir="${build.lib.dir}" />
- </target>
-
- <!-- target: clean-build ============================================= -->
- <target name="clean-build" description="--> clean the project built files">
- <delete includeemptydirs="true" dir="${build.dir}" />
- </target>
-
- <!-- target: clean-dist ============================================= -->
- <target name="clean-dist" description="--> clean the project dist files">
- <delete includeemptydirs="true" dir="${dist.dir}" />
- </target>
-
- <!-- target: clean-cache ============================================= -->
- <target name="clean-cache" depends="" description="delete ivy cache">
- <ivy:cleancache />
- </target>
-
- <target name="clean-runtime" description="--> clean the project runtime area">
- <delete includeemptydirs="true" dir="${runtime.dir}" />
- </target>
-
- <!-- ================================================================== -->
- <!-- RAT targets -->
- <!-- ================================================================== -->
- <target name="rat-sources-typedef">
- <typedef resource="org/apache/rat/anttasks/antlib.xml">
- <classpath>
- <fileset dir="." includes="rat*.jar" />
- </classpath>
- </typedef>
- </target>
-
- <target name="rat-sources" depends="rat-sources-typedef" description="runs the tasks over src/java">
- <rat:report xmlns:rat="antlib:org.apache.rat.anttasks">
- <fileset dir="src">
- <include name="java/**/*" />
- <include name="plugin/**/src/**/*" />
- </fileset>
- </rat:report>
- </target>
+ </target>
+ <!-- ================================================================== -->
+ <!-- Documentation -->
+ <!-- ================================================================== -->
+ <target name="javadoc" depends="compile">
+ <mkdir dir="${build.javadoc}"/>
+ <javadoc
+ overview="${src.dir}/overview.html"
+ destdir="${build.javadoc}"
+ author="true"
+ version="true"
+ use="true"
+ windowtitle="${Name} ${version} API"
+ doctitle="${Name} ${version} API"
+ bottom="Copyright &copy; ${year} The Apache Software Foundation"
+ >
+ <arg value="${javadoc.proxy.host}"/>
+ <arg value="${javadoc.proxy.port}"/>
+
+ <packageset dir="${src.dir}"/>
+ <packageset dir="${plugins.dir}/lib-http/src/java"/>
+ <packageset dir="${plugins.dir}/lib-regex-filter/src/java"/>
+ <packageset dir="${plugins.dir}/microformats-reltag/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-file/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-ftp/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-http/src/java"/>
+ <packageset dir="${plugins.dir}/protocol-httpclient/src/java"/>
+ <packageset dir="${plugins.dir}/parse-tika/src/java"/>
+ <packageset dir="${plugins.dir}/parse-ext/src/java"/>
+ <packageset dir="${plugins.dir}/parse-js/src/java"/>
+ <packageset dir="${plugins.dir}/parse-swf/src/java"/>
+ <packageset dir="${plugins.dir}/parse-zip/src/java"/>
+ <packageset dir="${plugins.dir}/index-basic/src/java"/>
+ <packageset dir="${plugins.dir}/index-more/src/java"/>
+ <packageset dir="${plugins.dir}/scoring-opic/src/java"/>
+ <packageset dir="${plugins.dir}/urlfilter-automaton/src/java"/>
+ <packageset dir="${plugins.dir}/urlfilter-regex/src/java"/>
+ <packageset dir="${plugins.dir}/urlfilter-prefix/src/java"/>
+ <packageset dir="${plugins.dir}/creativecommons/src/java"/>
+ <packageset dir="${plugins.dir}/languageidentifier/src/java"/>
+
+ <link href="${javadoc.link.java}"/>
+ <link href="${javadoc.link.lucene}"/>
+ <link href="${javadoc.link.hadoop}"/>
+
+ <classpath refid="classpath"/>
+ <classpath>
+ <fileset dir="${plugins.dir}" >
+ <include name="**/*.jar"/>
+ </fileset>
+ </classpath>
+
+ <group title="Core" packages="org.apache.nutch.*"/>
+ <group title="Plugins API" packages="${plugins.api}"/>
+ <group title="Protocol Plugins" packages="${plugins.protocol}"/>
+ <group title="URL Filter Plugins" packages="${plugins.urlfilter}"/>
+ <group title="Scoring Plugins" packages="${plugins.scoring}"/>
+ <group title="Parse Plugins" packages="${plugins.parse}"/>
+ <group title="Indexing Filter Plugins" packages="${plugins.index}"/>
+ <group title="Misc. Plugins" packages="${plugins.misc}"/>
+ </javadoc>
+ <!-- Copy the plugin.dtd file to the plugin doc-files dir -->
+ <copy file="${plugins.dir}/plugin.dtd"
+ todir="${build.javadoc}/org/apache/nutch/plugin/doc-files"/>
+ </target>
+
+ <target name="default-doc">
+ <style basedir="${conf.dir}" destdir="${docs.dir}"
+ includes="nutch-default.xml" style="conf/nutch-conf.xsl"/>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- D I S T R I B U T I O N -->
+ <!-- ================================================================== -->
+ <!-- -->
+ <!-- ================================================================== -->
+ <target name="package" depends="runtime, javadoc">
+ <mkdir dir="${dist.version.dir}"/>
+ <mkdir dir="${dist.version.dir}/lib"/>
+ <mkdir dir="${dist.version.dir}/runtime"/>
+ <mkdir dir="${dist.version.dir}/docs"/>
+ <mkdir dir="${dist.version.dir}/docs/api"/>
+ <mkdir dir="${dist.version.dir}/plugins"/>
+
+ <copy todir="${dist.version.dir}/lib" includeEmptyDirs="false">
+ <fileset dir="${build.lib.dir}"/>
+ </copy>
+
+ <copy todir="${dist.version.dir}/plugins">
+ <fileset dir="${build.plugins}"/>
+ </copy>
+
+ <copy file="${build.dir}/${final.name}.jar" todir="${dist.version.dir}"/>
+ <copy file="${build.dir}/${final.name}.job" todir="${dist.version.dir}"/>
+
+ <copy todir="${dist.version.dir}/runtime">
+ <fileset dir="runtime"/>
+ </copy>
+
+ <chmod perm="ugo+x" type="file">
+ <fileset dir="${dist.version.dir}/runtime/deploy/bin"/>
+ <fileset dir="${dist.version.dir}/runtime/local/bin"/>
+ </chmod>
+
+ <copy todir="${dist.version.dir}/conf">
+ <fileset dir="${conf.dir}" excludes="**/*.template"/>
+ </copy>
+
+ <copy todir="${dist.version.dir}/docs/api">
+ <fileset dir="${build.javadoc}"/>
+ </copy>
+
+ <copy todir="${dist.version.dir}">
+ <fileset dir=".">
+ <include name="*.txt" />
+ <include name="KEYS" />
+ </fileset>
+ </copy>
+
+ <copy todir="${dist.version.dir}/src" includeEmptyDirs="true">
+ <fileset dir="src"/>
+ </copy>
+
+ <copy todir="${dist.version.dir}/" file="build.xml"/>
+ <copy todir="${dist.version.dir}/" file="default.properties"/>
+
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Make release tarball -->
+ <!-- ================================================================== -->
+ <target name="tar" depends="package">
+ <tar compression="gzip" longfile="gnu"
+ destfile="${dist.dir}/${final.name}.tar.gz">
+ <tarfileset dir="${dist.dir}" mode="664">
+ <exclude name="${final.name}/bin/*" />
+ <include name="${final.name}/**" />
+ </tarfileset>
+ <tarfileset dir="${dist.dir}" mode="755">
+ <include name="${final.name}/bin/*" />
+ </tarfileset>
+ </tar>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Make release tarball -->
+ <!-- ================================================================== -->
+ <target name="zip" depends="package">
+ <zip compress="true" casesensitive="yes" destfile="${dist.dir}/${final.name}.zip">
+ <zipfileset dir="${dist.dir}" filemode="664">
+ <exclude name="${final.name}/bin/*" />
+ <include name="${final.name}/**" />
+ </zipfileset>
+ <zipfileset dir="${dist.dir}" filemode="755">
+ <include name="${final.name}/bin/*" />
+ </zipfileset>
+ </zip>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- Clean. Delete the build files, and their directories -->
+ <!-- ================================================================== -->
+
+ <!-- target: clean =================================================== -->
+ <target name="clean" depends="clean-build, clean-lib, clean-dist, clean-runtime" description="--> clean the project" />
+
+ <!-- target: clean-local ============================================= -->
+ <target name="clean-local" depends=""
+ description="--> cleans the local repository for the current module">
+ <delete dir="${ivy.local.default.root}/${ivy.organisation}/${ivy.module}"/>
+ </target>
+
+ <!-- target: clean-lib =============================================== -->
+ <target name="clean-lib" description="--> clean the project libraries directory (dependencies)">
+ <delete includeemptydirs="true" dir="${build.lib.dir}"/>
+ </target>
+
+ <!-- target: clean-build ============================================= -->
+ <target name="clean-build" description="--> clean the project built files">
+ <delete includeemptydirs="true" dir="${build.dir}"/>
+ </target>
+
+ <!-- target: clean-dist ============================================= -->
+ <target name="clean-dist" description="--> clean the project dist files">
+ <delete includeemptydirs="true" dir="${dist.dir}"/>
+ </target>
+
+ <!-- target: clean-cache ============================================= -->
+ <target name="clean-cache" depends=""
+ description="delete ivy cache">
+ <ivy:cleancache />
+ </target>
+
+ <target name="clean-runtime" description="--> clean the project runtime area">
+ <delete includeemptydirs="true" dir="${runtime.dir}"/>
+ </target>
+
+ <!-- ================================================================== -->
+ <!-- RAT targets -->
+ <!-- ================================================================== -->
+ <target name="rat-sources-typedef">
+ <typedef resource="org/apache/rat/anttasks/antlib.xml" >
+ <classpath>
+ <fileset dir="." includes="rat*.jar"/>
+ </classpath>
+ </typedef>
+ </target>
+
+ <target name="rat-sources" depends="rat-sources-typedef"
+ description="runs the tasks over src/java">
+ <rat:report xmlns:rat="antlib:org.apache.rat.anttasks">
+ <fileset dir="src">
+ <include name="java/**/*"/>
+ <include name="plugin/**/src/**/*"/>
+ </fileset>
+ </rat:report>
+ </target>
+
</project>
Modified: nutch/branches/branch-1.3/default.properties
URL: http://svn.apache.org/viewvc/nutch/branches/branch-1.3/default.properties?rev=1129734&r1=1129733&r2=1129734&view=diff
==============================================================================
--- nutch/branches/branch-1.3/default.properties (original)
+++ nutch/branches/branch-1.3/default.properties Tue May 31 15:08:24 2011
@@ -1,3 +1,4 @@
+Name=Nutch
name=nutch
version=1.3
final.name=${name}-${version}
@@ -43,7 +44,7 @@ runtime.dir=./runtime
runtime.deploy=${runtime.dir}/deploy
runtime.local=${runtime.dir}/local
-ivy.version=2.2.0
+ivy.version=2.1.0
ivy.dir=${basedir}/ivy
ivy.file=${ivy.dir}/ivy.xml
ivy.jar=${ivy.dir}/ivy-${ivy.version}.jar
Modified: nutch/branches/branch-1.3/ivy/ivy.xml
URL: http://svn.apache.org/viewvc/nutch/branches/branch-1.3/ivy/ivy.xml?rev=1129734&r1=1129733&r2=1129734&view=diff
==============================================================================
--- nutch/branches/branch-1.3/ivy/ivy.xml (original)
+++ nutch/branches/branch-1.3/ivy/ivy.xml Tue May 31 15:08:24 2011
@@ -12,12 +12,12 @@
language governing permissions and limitations under the License. -->
<ivy-module version="1.0">
- <info organisation="org.apache.nutch" module="nutch">
- <license name="Apache 2.0" url="http://www.apache.org/licenses/LICENSE-2.0.txt/"/>
+ <info organisation="org.apache.nutch" module="${ant.project.name}">
+ <license name="Apache 2.0" />
<ivyauthor name="Apache Nutch Team" url="http://nutch.apache.org" />
- <description homepage="http://nutch.apache.org">Nutch is an open source web-search software. It builds on
- Hadoop, Tika and Solr, adding web-specifics, such as a crawler, a link-graph database etc.
- </description>
+ <description>
+ Apache Nutch
+ </description>
</info>
<configurations>