You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-commits@db.apache.org by kr...@apache.org on 2009/12/08 14:04:26 UTC
svn commit: r888385 - in /db/derby/code/trunk/maven2: README.txt
SetDerbyVersion.java settings.xml
Author: kristwaa
Date: Tue Dec 8 13:04:25 2009
New Revision: 888385
URL: http://svn.apache.org/viewvc?rev=888385&view=rev
Log:
DERBY-4400: Document the process of producing Maven 2 artifacts for Derby
Added a utility class that obtains the Derby version from the jars that will
be put into the artifacts. It also does a few sanity checks of the jars, for
instance checking if they are all there.
Updated the README and changed/added some comments in settings.xml.
Patch file: derby-4400-1c-SetDerbyVersion.diff
Added:
db/derby/code/trunk/maven2/SetDerbyVersion.java (with props)
Modified:
db/derby/code/trunk/maven2/README.txt
db/derby/code/trunk/maven2/settings.xml
Modified: db/derby/code/trunk/maven2/README.txt
URL: http://svn.apache.org/viewvc/db/derby/code/trunk/maven2/README.txt?rev=888385&r1=888384&r2=888385&view=diff
==============================================================================
--- db/derby/code/trunk/maven2/README.txt (original)
+++ db/derby/code/trunk/maven2/README.txt Tue Dec 8 13:04:25 2009
@@ -1,8 +1,8 @@
GENERATING MAVEN 2 ARTIFACTS FOR APACHE DERBY
=============================================
-The POMs in the maven2 directory are able to generate Maven 2 artifacts for
-Apache Derby. The following software is required for deploying a release:
+The POMs in the maven2 directory enable you to generate Maven 2 artifacts
+for Apache Derby. The following software is required for deploying a release:
1. Maven 2
2. GnuPG (for signing the artifacts)
3. ssh/scp (for site deployment)
@@ -11,20 +11,38 @@
you run it. They will be cached locally, so they are not downloaded again the
next time.
+All commands below are to be executed from the directory 'maven2' within the
+Derby source code repository.
+
+WARNING: The Maven repository is write-once. This means that you have only one
+ chance to deploy artifacts with a given version string. Once they are
+ deployed, you cannot overwrite them. The only way to deprecate a set
+ of deployed artifacts is to deploy a new set of artifacts with a
+ different version string.
+
+
Short description of the required steps:
+
a) Generate the Derby jar files.
- For releases, generate the insane jars. You can specify which jars to use
+ For releases, generate the insane jars. You can override which jars to use
with the property 'sanity' in the top-level POM.
The jars are expected to be found in 'jars/[in]sane' relative to the
checked out code repository.
- b) Specify required information for one or all of the following steps.
- To generate and deploy release artifacts, these pieces of information must
- be specified:
+ b) Specify required information for one or all of the following sub-steps.
+ To successfully generate and deploy release artifacts, all of these
+ must be specified:
- The Derby release version, which must be specified in all POMs.
- One way to do this, is to use search and replace (i.e. Perl or sed).
- - Passphrase for your GPG signing key, see top level POM and step (d).
- - User credentials for deployment, see 'settings.xml'.
+ Compile and execute the Java program SetDerbyVersion, i.e.:
+ javac SetDerbyVersion && java -cp . SetDerbyVersion
+ Alternatively, use search and replace (i.e. Perl or sed) - make sure
+ you don't replace version tags that aren't supposed to modified.
+ Make sure you diff the POMs to verify the changes.
+ Note that the Java program performs some extra sanity checks.
+ - Passphrase for your GPG signing key. Required for step (c) and (d).
+ See the top level POM for details, brief instructions in (c).
+ - User credentials for deployment. Required for step (d).
+ See 'settings.xml' for details.
c) 'mvn clean install'
Generates the artifacts, signatures for the artifacts using GnuPG and
@@ -32,12 +50,17 @@
You are required to provide your private key and the passphrase to GnuPG.
Using a passphrase agent is recommended, but you can also specify it on
the command line when invoking maven with -Dgpg.passphrase=PASSPHRASE.
- There are other ways to achieve this too, but please do not specify you
- passphrase in the POM that is deployed on the Maven repositories!
+
+ WARNING: Do not specify your passphrase in the POM that is deployed on
+ the Maven repositories!
+
The local repository is typically found in '~/.m2/repository/', and the
- Derby artifacts are located under "org/apache/derby/".
+ Derby artifacts are located under 'org/apache/derby/'.
The clean target is included to avoid unintentionally installing/deploying
artifacts not supposed to be deployed.
+ If you just want to build the artifacts, use 'mvn package' or 'mvn verify'.
+ The former will generate the artifact jar, the latter will additionally
+ generate/include the POM to be deployed and the signatures.
NOTE: Do not run 'mvn package|verify install', that is to combine either
package or verify with install, as this causes the
@@ -45,17 +68,42 @@
'./engine/target/derby-trunk-alpha.jar.asc.asc'.
d) 'mvn deploy' or 'mvn clean deploy'
- NOTE: This step has been reported not to work. Deploy manually until fixed.
- Deploys the artifacts, including signatures and checksum files, to the
- Apache Maven 2 repository. The files will then be distributed to mirrors.
+ Deploys the artifacts, including signatures and checksum files, to the
+ Apache Maven 2 repository. The files will then be distributed to mirrors.
+
+ NOTE: This step has been reported to not work when using username and
+ password authentication. Unless you prefer to deploy manually, use a
+ public key to log into the remote host (people.apache.org).
-Basically, for each project, the following files should be found in the
-various 'maven2/[project]/target' directories:
+For each project, the following files should be found in the
+various 'maven2/[project]/target' directories after 'verify' or 'install':
- ARTIFACT-VERSION.jar
- ARTIFACT-VERSION.jar.asc
- ARTIFACT-VERSION.pom
- ARTIFACT-VERSION.pom.asc
-When these are deployed or installed locally, there will be a md5 and a sha1
-file for each artifact. The 'derbywar' project will have a war file instead
-of a jar file.
+When these are deployed, or installed locally, checksum files (a md5 and a sha1
+file for each artifact) will be generated by Maven. Check your local
+repository to confirm this (i.e. '~/.m2/repository').
+The 'derbywar' project will have a war file instead of a jar file.
+
+Some time after you have deployed the artifacts to the Apache staging
+repository (happens when you run 'mvn deploy'), they should appear in the
+central Maven repository.
+Try one of these to confirm that your artifacts are available:
+http://repo1.maven.org/maven2/org/apache/derby/
+http://mvnrepository.com/artifact/org.apache.derby
+
+
+Release history for Maven 2 artifacts
+=====================================
+
+The list below shows the Apache Derby artifacts published by the Apache Derby
+community.
+The dates are when the artifacts were written to the central Maven repository
+(repo1.maven.org/maven2 or repo2.maven.org/maven2).
+
+2009-10-07 10.5.3.0_1 OK
+2009-08-26 10.5.3.0 BROKEN
+ An error in all the POMs made these artifacts unusable (DERBY-4390).
+ Use version 10.5.3.0_1 instead.
Added: db/derby/code/trunk/maven2/SetDerbyVersion.java
URL: http://svn.apache.org/viewvc/db/derby/code/trunk/maven2/SetDerbyVersion.java?rev=888385&view=auto
==============================================================================
--- db/derby/code/trunk/maven2/SetDerbyVersion.java (added)
+++ db/derby/code/trunk/maven2/SetDerbyVersion.java Tue Dec 8 13:04:25 2009
@@ -0,0 +1,447 @@
+/*
+
+ Derby - Class SetDerbyVersion
+
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ */
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.EOFException;
+import java.io.File;
+import java.io.FileReader;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.Driver;
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Checks the current Derby jars in the source tree directory, obtains the
+ * Derby version from them, and replaces the value of the placeholder
+ * version tags in the POM files.
+ * <p>
+ * After this method has been successfully run you should be ready to
+ * generate the Maven 2 artifacts for Derby.
+ * <p>
+ * The main task of this class is to replace the version tags in the Maven
+ * POM files. The can be done manually, but exact process would vary from
+ * platform to platform. Also, running a search-and-replace could potentially
+ * replace tags not supposed to be replaced. To make the Maven 2 artifact
+ * publish process simpler, this class was written.
+ */
+//@NotThreadSafe
+public class SetDerbyVersion {
+
+ private static final String PROMPT_CONT_WARN =
+ "Do you want to continue despite the warnings?";
+ private static final String PROMPT_USE_SANE =
+ "Do you want to generate artifacts with SANE jars?";
+ private static final String JDBC_URL =
+ "jdbc:derby:memory:testDB;create=true";
+ private static final String REL_JAR_PATH = "../jars";
+ private static final File SANE = new File(REL_JAR_PATH, "sane");
+ private static final File INSANE = new File(REL_JAR_PATH, "insane");
+ /** List of required jar files the Maven 2 Derby artifacts. */
+ private static final String[] JARS = new String[] {
+ "derby.jar",
+ "derby.war",
+ "derbynet.jar",
+ "derbyclient.jar",
+ "derbytools.jar",
+ // Ignore derbyTesting.jar, not part of the Maven 2 artifacts.
+ // "derbyTesting.jar",
+ // Ignore derbyrun.jar, not part of the Maven 2 artifacts.
+ //"derbyrun.jar",
+ // The various locale files.
+ "derbyLocale_cs.jar",
+ "derbyLocale_de_DE.jar",
+ "derbyLocale_es.jar",
+ "derbyLocale_fr.jar",
+ "derbyLocale_hu.jar",
+ "derbyLocale_it.jar",
+ "derbyLocale_ja_JP.jar",
+ "derbyLocale_ko_KR.jar",
+ "derbyLocale_pl.jar",
+ "derbyLocale_pt_BR.jar",
+ "derbyLocale_ru.jar",
+ "derbyLocale_zh_CN.jar",
+ "derbyLocale_zh_TW.jar",
+ };
+
+ /**
+ * Displays a prompt and obtains a yes / no answer from standard in.
+ *
+ * @param prompt the prompt to display
+ * @return {@code true} if the answer is yes, {@code false} if the answer
+ * is no.
+ * @throws IOException if reading from standard in fails
+ */
+ private static boolean getYesNoInput(String prompt)
+ throws IOException {
+ // We don't care about keeping the objects around, this method will
+ // only be used a few times.
+ BufferedReader bIn = new BufferedReader(
+ new InputStreamReader(System.in));
+ while (true) {
+ System.out.print(">> " + prompt + " (yes/no) ");
+ String answer = bIn.readLine();
+ if (answer == null) {
+ // We don't know what do to here, so just fail.
+ throw new EOFException("Input stream closed.");
+ }
+ if (answer.equals("yes")) {
+ return true;
+ } else if (answer.equals("no")) {
+ return false;
+ }
+ }
+ }
+
+ /**
+ * Set to {@code true} if a warning message is printed. Must be manually
+ * reset if used to check for warnings in a part of the code.
+ */
+ private boolean warnings = false;
+ /** The version string inserted into the POMs. */
+ private String versionString = "ALPHA_VERSION";
+ private File PREFIX;
+
+ /**
+ * Prints a warning message and sets the internal warning flag.
+ *
+ * @param msg the message to print
+ */
+ private void warn(String msg) {
+ warnings = true;
+ System.out.println("WARNING! " + msg);
+ }
+
+ private void info(String msg) {
+ System.out.println(msg);
+ }
+
+ /**
+ * Checks that all required jars are found in the jar directory.
+ *
+ * @return {@code true} if all required jars exist, {@code false} otherwise.
+ */
+ public boolean checkJars()
+ throws Exception {
+ if (!SANE.exists() && !INSANE.exists()) {
+ warn("No jars exist. Produce a Derby release build.");
+ return false;
+ }
+ if (SANE.exists() && INSANE.exists()) {
+ warn("Both SANE and INSANE jars exist.");
+ return false;
+ }
+ PREFIX = SANE.exists() ? SANE : INSANE;
+ if (SANE.exists()) {
+ warn("Only SANE jars exist. Normally INSANE jars are used for a " +
+ "release.");
+ boolean answer = getYesNoInput(PROMPT_USE_SANE);
+ if (!answer) {
+ return false;
+ }
+ }
+ URL[] URLS = new URL[JARS.length];
+ for (int i=0; i < JARS.length; i++) {
+ URLS[i] = new File(PREFIX, JARS[i]).toURI().toURL();
+ }
+
+ warnings = false; // Reuse the warnings flag.
+ // Make sure the files are there.
+ for (URL url : URLS) {
+ File f = new File(url.toURI());
+ info(String.format(
+ "Checking file: %-30s %,12d bytes",
+ f.getName(), f.length()));
+ if (!f.exists()) {
+ warn("Missing file: " + f.getCanonicalPath());
+ } else if (f.length() == 0) {
+ warn("Empty file: " + f.getCanonicalPath());
+ }
+ }
+ info("");
+ if (warnings) {
+ // Fail here.
+ warn("There are missing or empty jar files.");
+ return false;
+ }
+
+ // The class loader used for the Derby jars.
+ URLClassLoader cl = new URLClassLoader(URLS, null);
+
+ // Extra sanity check for the sanity...
+ try {
+ Class.forName(
+ "org.apache.derby.shared.common.sanity.SanityManager",
+ true ,cl);
+ if (PREFIX == INSANE) {
+ warn("Found SanityManager in INSANE build. Aborting.");
+ return false;
+ }
+ } catch (ClassNotFoundException cnfe) {
+ if (PREFIX == SANE) {
+ warn("Unable to load SanityManager in SANE build. Aborting.");
+ return false;
+ }
+ }
+
+ // Fire up Derby to get the version string.
+ Class driverClass =
+ Class.forName("org.apache.derby.jdbc.EmbeddedDriver", true, cl);
+ Driver driver = (Driver)driverClass.newInstance();
+ Connection con = driver.connect(JDBC_URL, null);
+ DatabaseMetaData meta = con.getMetaData();
+ con.close();
+ // Delete the derby.log file.
+ new File("derby.log").delete();
+
+ // I.e.: 10.6.0.0 alpha - (882129M)
+ String fullVersion = meta.getDatabaseProductVersion();
+ String[] components = fullVersion.split(" - ");
+ versionString = components[0].replaceAll(" ", "_");
+ String srcRevision = components[1].replaceAll("\\(|\\)", "");
+ info("Obtained product version string: " + fullVersion);
+ info("(version=" + versionString + ", revision=" + srcRevision + ")");
+ if (versionString.contains("beta")) {
+ warn("This is a BETA build.");
+ }
+ if (versionString.contains("alpha")) {
+ warn("This is an ALPHA build.");
+ }
+ if (srcRevision.endsWith("M")) {
+ warn("The sources had been modified when the jars were built.");
+ warnings = true;
+ }
+ if (warnings) {
+ if (!getYesNoInput(PROMPT_CONT_WARN)) {
+ return false;
+ }
+ }
+ return true;
+ }
+
+ /**
+ * Replaces the relevant version tags in the various POM files.
+ *
+ * @throws IOException if accessing a POM file fails
+ */
+ public boolean setPOMVersionTags()
+ throws IOException {
+ File curDir = new File(".");
+ boolean gotWarnings = false;
+ // We only descend one level, no need for a recursive method.
+ for (File topLevel : curDir.listFiles()) {
+ if (topLevel.getName().equals("pom.xml")) {
+ gotWarnings |= setVersionTag(topLevel);
+ }
+ if (topLevel.isDirectory()) {
+ for (File l1 : topLevel.listFiles()) {
+ if (l1.getName().equals("pom.xml")) {
+ gotWarnings |= setVersionTag(l1);
+ // There is only one POM in each sub-directory.
+ break;
+ }
+ }
+ }
+ }
+
+ // See if we ran into problems when replacing the version tags.
+ info("");
+ if (gotWarnings) {
+ warn("There were errors replacing the POM version tags.");
+ } else {
+ info("POM version tags replacement succeeded.");
+ info("It is recommended that you verify the POM diffs " +
+ "before running Maven.");
+ }
+ return (warnings == false);
+ }
+
+ public void printSanityNote()
+ throws IOException {
+ if (PREFIX == SANE) {
+ info("");
+ info("NOTE: Remember to change the <sanity> tag in the top-level");
+ info(" POM, setting it to 'sane'.");
+ }
+ }
+
+ /**
+ * Verifies that the correct number of tags were replaced in the POM.
+ *
+ * @param replaceCount the number of tags replaced
+ * @param pom the POM modified
+ * @return {@code 0} if the check passed, a negative value if too few tags
+ * were replaced, and a positive value if too many tags were replaced.
+ */
+ private int checkResult(int replaceCount, File pom) {
+ // The locales requires two replacements, due to the dependency.
+ String parent = pom.getParent();
+ if (parent.contains("derbyLocale")) {
+ return (replaceCount - 2);
+ // derbynet also requries two replacements (derby.jar dependency)
+ } else if (parent.contains("net")) {
+ return (replaceCount - 2);
+ } else {
+ return (replaceCount - 1);
+ }
+ }
+
+ /**
+ * Replaces all qualifying version tags in the specified POM.
+ *
+ * @param pom the POM to modify
+ * @return {@code false} if warnings were produced when replacing,
+ * {@code true} if all seemed to go well.
+ *
+ * @throws IOException if reading or writing to the POM file fails
+ */
+ private boolean setVersionTag(File pom)
+ throws IOException {
+ // Clear internal warning flag.
+ warnings = false;
+ // Just read the whole file into memory.
+ List<String> lines = readFile(pom);
+
+ // Start writing the file back out, and search for tags to replace.
+ BufferedWriter bOut = new BufferedWriter(new FileWriter(pom, false));
+ int replaced = 0;
+ boolean artifactIdOk = false;
+ boolean groupIdOk = false;
+ // Could have used XML, but keep it simple.
+ // artifactId and groupId are used to qualify the version tag, as it can
+ // be used in more places than those we want to replace.
+ for (String line : lines) {
+ // Look for tags for qualification.
+ if (line.trim().startsWith("<artifactId>") &&
+ line.contains("derby")) {
+ artifactIdOk = true;
+ } else if (line.trim().startsWith("<groupId>") &&
+ line.contains("org.apache.derby")) {
+ groupIdOk = true;
+ }
+
+ // Change line if a qualified version tag, echo otherwise.
+ if (line.trim().startsWith("<version>") &&
+ artifactIdOk && groupIdOk) {
+ // Replace tag.
+ int whitespaceTag = line.indexOf(">");
+ bOut.write(line.substring(0, whitespaceTag +1));
+ bOut.write(versionString);
+ int tagEnd = line.indexOf("<", whitespaceTag);
+ bOut.write(line.substring(tagEnd));
+ bOut.newLine();
+ replaced++;
+ artifactIdOk = groupIdOk = false;
+ } else {
+ bOut.write(line);
+ bOut.newLine();
+ }
+ }
+ try {
+ bOut.flush();
+ bOut.close();
+ } catch (IOException ioe) {
+ warn("Flushing/closing stream for " + pom.getCanonicalPath() +
+ " failed: " + ioe.getMessage());
+ }
+ int result = checkResult(replaced, pom);
+ if (result == 0) {
+ info("Replaced " + replaced + " version tag(s) in " +
+ pom.getParentFile().getName() + "/" + pom.getName());
+ } else if (result > 0) {
+ warn("Too many version tags (" + replaced + " > " +
+ (replaced - result) + ") replaced in " + pom.getPath());
+ } else {
+ warn("Too few version tags (" + replaced + " < " +
+ (replaced - result) + ") replaced in " + pom.getPath());
+ }
+ return warnings;
+ }
+
+ /**
+ * Reads the contents of a text file.
+ *
+ * @param f the file to read
+ * @return A list containing the lines of the file.
+ * @throws IOException if reading the file fails
+ */
+ private List<String> readFile(File f)
+ throws IOException {
+ ArrayList<String> lines = new ArrayList<String>();
+ BufferedReader bIn = new BufferedReader(new FileReader(f));
+ String lineIn;
+ try {
+ while ((lineIn = bIn.readLine()) != null) {
+ lines.add(lineIn);
+ }
+ } finally {
+ try {
+ bIn.close();
+ } catch (IOException ioe) {
+ // Just print a warning.
+ warn("Failed to close input stream for " + f.getPath() + ": " +
+ ioe.getMessage());
+ }
+ }
+ return lines;
+ }
+
+ /**
+ * Quits the JVM if a failure is detected.
+ *
+ * @param success the return value to check
+ */
+ private static void ensureSuccess(boolean success) {
+ if (!success) {
+ System.out.println();
+ System.out.println(
+ "!! The process failed or was aborted by the user.");
+ System.out.println(
+ " Read the above output and take corrective measures.");
+ System.exit(1);
+ }
+ }
+
+ /**
+ * Checks the current Derby jars in the source tree directory, obtains the
+ * Derby version from them, and replaces the value of the placeholder
+ * version tags in the POM files.
+ * <p>
+ * After this method has been successfully run you should be ready to
+ * generate the Maven 2 artifacts for Derby.
+ *
+ * @param args ignored
+ * @throws Exception if something goes wrong
+ */
+ public static void main(String[] args)
+ throws Exception {
+ SetDerbyVersion sdv = new SetDerbyVersion();
+ ensureSuccess(sdv.checkJars());
+ ensureSuccess(sdv.setPOMVersionTags());
+ sdv.printSanityNote();
+ }
+}
Propchange: db/derby/code/trunk/maven2/SetDerbyVersion.java
------------------------------------------------------------------------------
svn:eol-style = native
Modified: db/derby/code/trunk/maven2/settings.xml
URL: http://svn.apache.org/viewvc/db/derby/code/trunk/maven2/settings.xml?rev=888385&r1=888384&r2=888385&view=diff
==============================================================================
--- db/derby/code/trunk/maven2/settings.xml (original)
+++ db/derby/code/trunk/maven2/settings.xml Tue Dec 8 13:04:25 2009
@@ -7,21 +7,39 @@
<server>
<id>apache.releases</id>
<!-- RELEASE_DATA: User credentials required for site deployment.
- Replace the user name and password with your Apache credentials.
+ Fill in your Apache user name, and uncomment the line to specify
+ a private key passphrase if required.
It is highly recommended to only do this when actually deploying
a release, and then revert the changes immediately afterwards to
avoid unintended deployments!
+
+ Note that all required information can be specified as properties on
+ the command line instead of in this file. For instance:
+ mvn -Dusername=asf_user -Dpassphrase=asf_users_passphrase deploy
+
+ If you get the credentials wrong and try to deploy to the Apache
+ site, your IP may be blacklisted. Contact the Apache infrastructure
+ team to get the ban lifted.
-->
<username>my_login</username>
+ <!--
+ Password authentication doesn't work (unknown reason).
+ Use a private key (see below).
<password>my_password</password>
- <!-- Make sure only one of password and privateKey is used.
- If both are present, privateKey will be ignored.
+ -->
+ <!-- Make sure only one of 'password' and 'privateKey' is specified.
+ If both are present, 'privateKey' will be ignored.
+ -->
<privateKey>${user.home}/.ssh/id_dsa</privateKey>
+ <!-- Specify the key passphrase if you have one (uncomment the line).
<passphrase>some_passphrase</passphrase>
-->
+
+ <!-- These settings don't work, but maybe they will one day? -->
<filePermissions>664</filePermissions>
<directoryPermissions>775</directoryPermissions>
<configuration></configuration>
+
</server>
</servers>
</settings>